yacy_search_server/source/de/anomic/crawler/retrieval
orbiter aa322bc6d0 fix
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8050 6c8d7289-2bf4-0310-a012-ef5d649a1542
2011-11-16 15:36:30 +00:00
..
FileLoader.java refactoring: 2011-09-25 16:59:06 +00:00
FTPLoader.java refactoring: 2011-09-25 16:59:06 +00:00
HTTPLoader.java fix 2011-11-16 15:36:30 +00:00
Request.java - performance hacks (should affect the crawl balancer and reduce CPU load during crawl stack re-fill) 2011-09-30 07:57:50 +00:00
Response.java added new configuration property "crawler.embedLinksAsDocuments". If this is switched on (this is default now), the all embedded image, audio and video links from all parsed documents are added to the search index as individual document. This will increase the search index size dramatically but will also enable us to create a much faster image, audio and video search. If the flag is switched on, the index entries are also stored to a solr index, if this is also enabled. 2011-09-07 10:08:57 +00:00
SMBLoader.java refactoring: 2011-09-25 16:59:06 +00:00