yacy_search_server/source/net/yacy/crawler
Michael Peter Christen 0b6566a389 optimizations when starting large crawl requests with many start urls in
one request:
- allow larger match-fields in html interface
- delete all host hashes at once from zurl
- when deleting by host, do not count size of deleted entries since that
was the reason it took so long
2013-01-31 13:15:28 +01:00
..
data optimizations when starting large crawl requests with many start urls in 2013-01-31 13:15:28 +01:00
retrieval small sanitary fixes 2013-01-02 01:59:47 +01:00
robots fix for wrong robots.txt loading for https protocol 2013-01-16 17:38:06 +01:00
Balancer.java fix for smb crawl situation (lost too many urls) 2012-12-26 19:15:11 +01:00
CrawlStacker.java fix for smb crawl situation (lost too many urls) 2012-12-26 19:15:11 +01:00
CrawlSwitchboard.java added separate delete commands for the local+remote solr index, the old 2013-01-04 16:39:34 +01:00
HarvestProcess.java fix for wrong display of error urls in HostBrowser 2012-12-07 00:31:10 +01:00