Michael Peter Christen eca9380e3d bugfix for crawler double-check: if an url is redirected, the ... redirect-target was not double-checked. This is now done by replacing the redirect-URL on the crawl queue again (where it is double-checked)		2014-08-06 12:35:12 +02:00
..
data	fix for subpath crawl filter	2014-08-06 01:33:24 +02:00
retrieval	bugfix for crawler double-check: if an url is redirected, the	2014-08-06 12:35:12 +02:00
robots	do not store failure of loading of robots.txt into the index as a fail	2014-08-01 12:15:14 +02:00
Balancer.java	- added a new Crawler Balancer: HostBalancer and HostQueues:	2014-04-16 21:34:28 +02:00
CrawlStacker.java	bugfix for crawler double-check: if an url is redirected, the	2014-08-06 12:35:12 +02:00
CrawlSwitchboard.java	typo	2014-08-04 16:13:39 +02:00
HarvestProcess.java	fix for wrong display of error urls in HostBrowser	2012-12-07 00:31:10 +01:00
HostBalancer.java	npe fix in crawler	2014-08-05 21:31:59 +02:00
HostQueue.java	fix for malformed hostpath names in crawl balancer	2014-07-29 11:18:45 +02:00
LegacyBalancer.java	special strategy for balancer: do not remove targets with zero wait time	2014-04-18 06:50:07 +02:00