yacy_search_server/source/de/anomic/crawler
2009-06-06 16:20:27 +00:00
..
AbstractImporter.java refactoring of logging 2009-01-30 23:33:47 +00:00
Balancer.java omit most of forced crawl delays by using a separat delay table which flushes delayed URLs at the correct time 2009-06-06 16:20:27 +00:00
CrawlEntry.java - added writing of temporary file names and renaming to final file name when index dump/merge are done. Interrupted merges can be cleaned up. 2009-04-01 12:39:11 +00:00
CrawlProfile.java migration of all databases that use the deprecated BLOBTree format into the BLOBHeap format. Old databases are migrated automatically. 2009-05-27 15:04:04 +00:00
CrawlQueues.java - fixed a not working selection rule in balancer 2009-06-06 08:46:59 +00:00
CrawlStacker.java more refactoring to make the segment object easier to use and to be prepared to integrate author navigation 2009-05-29 10:03:35 +00:00
CrawlSwitchboard.java new fully redesigned balancer and bugfixes regarding lost profile handles and killed crawls 2009-06-06 01:56:31 +00:00
FTPLoader.java refactoring of plasmaWordIndex: 2009-05-28 14:26:05 +00:00
HTTPLoader.java refactoring of plasmaWordIndex: 2009-05-28 14:26:05 +00:00
Importer.java
ImporterException.java added final where possible 2008-08-02 12:12:04 +00:00
ImporterManager.java more memory leak fixing hacks 2009-02-11 13:31:10 +00:00
IndexingStack.java new fully redesigned balancer and bugfixes regarding lost profile handles and killed crawls 2009-06-06 01:56:31 +00:00
Latency.java - fixed a not working selection rule in balancer 2009-06-06 08:46:59 +00:00
LoaderMessage.java refactoring: better abstraction of reference and metadata prototypes. 2009-04-03 13:23:45 +00:00
NoticedURL.java new fully redesigned balancer and bugfixes regarding lost profile handles and killed crawls 2009-06-06 01:56:31 +00:00
NoticeURLImporter.java refactoring of plasmaWordIndex: 2009-05-28 14:26:05 +00:00
ProtocolLoader.java refactoring: better abstraction of reference and metadata prototypes. 2009-04-03 13:23:45 +00:00
ResourceObserver.java refactoring of plasmaWordIndex: 2009-05-28 14:26:05 +00:00
ResultImages.java * refactoring 2008-08-02 13:57:00 +00:00
ResultURLs.java refactoring: better abstraction of reference and metadata prototypes. 2009-04-03 13:23:45 +00:00
robotsParser.java * Robots.txt: don't interpret Crawl-Delays for other robots 2008-12-18 15:35:41 +00:00
RobotsTxt.java migration of all databases that use the deprecated BLOBTree format into the BLOBHeap format. Old databases are migrated automatically. 2009-05-27 15:04:04 +00:00
SitemapImporter.java more performance hacks 2008-12-04 12:54:16 +00:00
ZURL.java - added migration class to go from index collections to the index cell data structure. 2009-03-30 15:31:25 +00:00