Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
ee89cf5ae5
yacy_search_server/source/de/anomic/crawler
History
Lotus ee89cf5ae5 fix must match filter for full domain crawl 
allow:
http://www.example.com
http://www.example.com/
http://www.example.com/abc.html?xyz=q
block:
http://www.example.com.cn
http://www.example.com.cn/dsf
2012-02-07 16:13:13 +01:00
..
retrieval Fixed a lot CHMOD rights (no need for execute flag on *.java/*.html) and introduced local/remote crawl size ratio based check 2011-12-29 00:33:16 +01:00
Balancer.java complete redesign of crawl queue monitoring: do not look at a 2012-02-02 21:33:42 +01:00
CrawlProfile.java fix must match filter for full domain crawl 2012-02-07 16:13:13 +01:00
CrawlQueues.java complete redesign of crawl queue monitoring: do not look at a 2012-02-02 21:33:42 +01:00
CrawlStacker.java complete redesign of crawl queue monitoring: do not look at a 2012-02-02 21:33:42 +01:00
CrawlSwitchboard.java - less automatic indexing after a search (needs to reset the default 2011-12-05 16:22:11 +01:00
ImporterException.java
Latency.java complete redesign of crawl queue monitoring: do not look at a 2012-02-02 21:33:42 +01:00
NoticedURL.java complete redesign of crawl queue monitoring: do not look at a 2012-02-02 21:33:42 +01:00
ResourceObserver.java refactoring: 2011-09-25 16:59:06 +00:00
ResultImages.java Fixed a lot CHMOD rights (no need for execute flag on *.java/*.html) and introduced local/remote crawl size ratio based check 2011-12-29 00:33:16 +01:00
ResultURLs.java Removed handling of components objects for URIMetadataRows. 2011-12-17 01:27:08 +01:00
RobotsTxt.java produce a bookmark entry from every crawl start. these bookmarks are always private. 2011-11-21 23:10:29 +00:00
RobotsTxtEntry.java - enhanced logging in robots.txt parser for remote debugging 2011-11-16 01:03:49 +00:00
RobotsTxtParser.java fix for robot parser 2011-11-16 13:12:46 +00:00
RSSLoader.java refactoring: 2011-09-25 16:59:06 +00:00
SitemapImporter.java refactoring: 2011-09-25 16:59:06 +00:00
ZURL.java Fixed a lot CHMOD rights (no need for execute flag on *.java/*.html) and introduced local/remote crawl size ratio based check 2011-12-29 00:33:16 +01:00