luccioman dcdea2d02f Fixed shutdown for crawler.MaxActiveThreads value greater than 200 ... Shutdown was hanging in CrawlQueues.close() at this.workerQueue.put(POISON_REQUEST) when config value crawler.MaxActiveThreads was greater than 200. Revealed by "Collision" Threads dumps in mantis 689 (http://mantis.tokeek.de/view.php?id=689#c1312) Fixed consistency between this.worker.length and this.workerQueue capacity, and made the process more reliable using non-blocking offer() function.		2016-09-29 10:33:11 +02:00
..
data	Fixed shutdown for crawler.MaxActiveThreads value greater than 200	2016-09-29 10:33:11 +02:00
retrieval	removed unused imports	2016-09-06 18:46:24 +02:00
robots	more ipv6 bugfixes	2014-10-08 15:21:49 +02:00
Balancer.java	removed warnings	2015-08-03 05:37:34 +02:00
CrawlStacker.java	enhanced timezone managament for indexed data:	2015-04-15 13:17:23 +02:00
CrawlSwitchboard.java	remove wrong log line in CrawlSwitchboard	2016-07-02 20:33:23 +02:00
HarvestProcess.java	fix for wrong display of error urls in HostBrowser	2012-12-07 00:31:10 +01:00
HostBalancer.java	one more replacement to use cached hosthash vs. calculated	2016-07-07 02:50:57 +02:00
HostQueue.java	to prevent crawler to concurrently access and alter same crawl queue	2016-07-05 23:22:35 +02:00
LegacyBalancer.java	use supplied url port to get robots.txt in crawlers hostqueue	2016-03-02 00:12:34 +01:00
RecrawlBusyThread.java	init Recrawl job chunk size to max crawl loader during job start, to use some system preferences	2015-10-16 03:05:39 +02:00