yacy_search_server/source/de/anomic/crawler
sixcooler 661867923a ... migrating to HttpComponents-Client-4.x ...
The Client is dead, long live the Client!
(no references to the old client)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7060 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-08-22 17:38:27 +00:00
..
retrieval - better url double check in crawler 2010-08-11 09:54:18 +00:00
AbstractImporter.java - cleanup, removed unused imports 2010-04-27 21:47:41 +00:00
Balancer.java redirect uncaught exceptions to logging + small other changes 2010-08-16 12:33:06 +00:00
CrawlProfile.java added the new crawl scheduling function to the crawl start menu: 2010-08-19 23:52:38 +00:00
CrawlQueues.java ... migrating to HttpComponents-Client-4.x ... 2010-08-22 17:38:27 +00:00
CrawlStacker.java fixed crawler bug caused by NPE in logging 2010-08-12 01:29:56 +00:00
CrawlSwitchboard.java added the new crawl scheduling function to the crawl start menu: 2010-08-19 23:52:38 +00:00
Importer.java
ImporterException.java added final where possible 2008-08-02 12:12:04 +00:00
ImporterManager.java *) some minor changes for better code readability 2010-04-05 12:37:33 +00:00
Latency.java better (and corrected) recognition of intranet and internet-addresses. This corrects the isLocal property that is used by network definitions to restrict index ranges to local and global addresses. Address locations (intranet or internet) had been partly identified by the top level domain of the host address. Since intranet addresses can also be addressed using a host name that is in a country domain it is necessary to do a dns resolving for each check. The check is supported by a local dns cache so the intranet/internet check should not affect network traffic too much. To ensure that the cache works properly the cache class was upgraded to better concurrency data structures. 2010-07-18 20:14:20 +00:00
NoticedURL.java - better url double check in crawler 2010-08-11 09:54:18 +00:00
ResourceObserver.java allow global search if res. observer disabled index transmission 2010-02-09 17:14:16 +00:00
ResultImages.java redesign of parser interface: 2010-06-29 19:20:45 +00:00
ResultURLs.java - more abstraction (HashMap -> Map) 2010-06-01 13:02:11 +00:00
RobotsEntry.java redesign of remote proxy settings 2010-05-26 00:01:16 +00:00
robotsParser.java - fixed a bug in robots.txt parser 2010-03-04 11:58:07 +00:00
RobotsTxt.java - more logging 2010-08-21 10:16:05 +00:00
SitemapImporter.java applied code changes that are recommended by PMD 2010-01-10 23:09:48 +00:00
ZURL.java fixed crawler bug caused by NPE in logging 2010-08-12 01:29:56 +00:00