Michael Peter Christen 0fe8be7981 enhaced data structures for balancer and latency computation which ... should produce a bit better prognosis about forced waiting times.		2012-10-30 17:30:24 +01:00
..
data	enhaced data structures for balancer and latency computation which	2012-10-30 17:30:24 +01:00
retrieval	replaced the custom robots.txt loader by the standard http loader	2012-10-28 22:48:11 +01:00
robots	enhaced data structures for balancer and latency computation which	2012-10-30 17:30:24 +01:00
Balancer.java	enhaced data structures for balancer and latency computation which	2012-10-30 17:30:24 +01:00
CrawlStacker.java	- added concurrency for robots.txt loading	2012-10-29 21:08:45 +01:00
CrawlSwitchboard.java	removed options for stopwords which are not used	2012-10-30 12:36:36 +01:00