yacy_search_server/source/de/anomic/crawler
orbiter 7bcfa033c9 more abstraction of the htcache when using the LoaderDispatcher:
a cache access shall not made directly to the cache any more, all loading attempts shall use the LoaderDispatcher.
To control the usage of the cache, a enum instance from CrawlProfile.CacheStrategy shall be used.
Some direct loading methods without the usage of a cache strategy have been removed. This affects also the verify-option
of the yacysearch servlet. If there is a 'verify=false' now after this commit this does not necessarily mean that no snippets
are generated. Instead, all snippets that can be retrieved using the cache only are presented. This still means that the search hit was not verified because the snippet was generated using the cache. If a cache-based generation of snippets is not possible, then the verify=false causes that the link is not rejected.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6936 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-06-21 14:54:54 +00:00
..
retrieval - scanned remote search process and enhanced some data structure and synchronizations here and there 2010-06-17 11:59:40 +00:00
AbstractImporter.java - cleanup, removed unused imports 2010-04-27 21:47:41 +00:00
Balancer.java fixes and enhancements for balancer: 2010-06-01 09:30:23 +00:00
CrawlProfile.java more abstraction of the htcache when using the LoaderDispatcher: 2010-06-21 14:54:54 +00:00
CrawlQueues.java more abstraction of the htcache when using the LoaderDispatcher: 2010-06-21 14:54:54 +00:00
CrawlStacker.java - added new protocol loader for 'file'-type URLs 2010-05-25 12:54:57 +00:00
CrawlSwitchboard.java fixed a bug in snippet fetch strategy: cache only does not help if resource can only be found in web 2010-06-18 15:25:25 +00:00
Importer.java
ImporterException.java
ImporterManager.java *) some minor changes for better code readability 2010-04-05 12:37:33 +00:00
Latency.java fixes to balancer when crawling filesystem (problem was: host == null) 2010-05-30 11:17:38 +00:00
NoticedURL.java continuing String-hash - to - byte[]-hash redesign that was started in SVN 6775 and continued in SVN 6790 2010-04-15 13:22:59 +00:00
ResourceObserver.java allow global search if res. observer disabled index transmission 2010-02-09 17:14:16 +00:00
ResultImages.java - added new protocol loader for 'file'-type URLs 2010-05-25 12:54:57 +00:00
ResultURLs.java - more abstraction (HashMap -> Map) 2010-06-01 13:02:11 +00:00
RobotsEntry.java redesign of remote proxy settings 2010-05-26 00:01:16 +00:00
robotsParser.java - fixed a bug in robots.txt parser 2010-03-04 11:58:07 +00:00
RobotsTxt.java better handling of OOM situations 2010-06-15 19:44:05 +00:00
SitemapImporter.java applied code changes that are recommended by PMD 2010-01-10 23:09:48 +00:00
ZURL.java possible fix for ZURL.list counter 2010-04-23 08:46:47 +00:00