yacy_search_server/source/de/anomic/crawler/retrieval
2011-11-25 11:23:52 +00:00
..
FileLoader.java refactoring: 2011-09-25 16:59:06 +00:00
FTPLoader.java some last-minute performance hacks 2011-11-25 11:23:52 +00:00
HTTPLoader.java fix 2011-11-16 15:36:30 +00:00
Request.java - performance hacks (should affect the crawl balancer and reduce CPU load during crawl stack re-fill) 2011-09-30 07:57:50 +00:00
Response.java added new configuration property "crawler.embedLinksAsDocuments". If this is switched on (this is default now), the all embedded image, audio and video links from all parsed documents are added to the search index as individual document. This will increase the search index size dramatically but will also enable us to create a much faster image, audio and video search. If the flag is switched on, the index entries are also stored to a solr index, if this is also enabled. 2011-09-07 10:08:57 +00:00
SMBLoader.java refactoring: 2011-09-25 16:59:06 +00:00