yacy_search_server/source/de/anomic/crawler/retrieval
orbiter 035ebfbf3b - performance hacks (should affect the crawl balancer and reduce CPU load during crawl stack re-fill)
- this may have also (good) performance side effects on other parts of YaCy


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7982 6c8d7289-2bf4-0310-a012-ef5d649a1542
2011-09-30 07:57:50 +00:00
..
FileLoader.java refactoring: 2011-09-25 16:59:06 +00:00
FTPLoader.java refactoring: 2011-09-25 16:59:06 +00:00
HTTPLoader.java refactoring: 2011-09-25 16:59:06 +00:00
Request.java - performance hacks (should affect the crawl balancer and reduce CPU load during crawl stack re-fill) 2011-09-30 07:57:50 +00:00
Response.java added new configuration property "crawler.embedLinksAsDocuments". If this is switched on (this is default now), the all embedded image, audio and video links from all parsed documents are added to the search index as individual document. This will increase the search index size dramatically but will also enable us to create a much faster image, audio and video search. If the flag is switched on, the index entries are also stored to a solr index, if this is also enabled. 2011-09-07 10:08:57 +00:00
SMBLoader.java refactoring: 2011-09-25 16:59:06 +00:00