yacy_search_server/source/de/anomic/crawler
orbiter f3baaca920 - enhancements to DNS IP caching and crawler speed
- bugfixes (NPEs)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7619 6c8d7289-2bf4-0310-a012-ef5d649a1542
2011-03-22 09:34:10 +00:00
..
retrieval - replaced more direct string-based UTF-8 conversions by predefined UTF-8 conversion 2011-03-10 23:25:07 +00:00
Balancer.java - enhancements to DNS IP caching and crawler speed 2011-03-22 09:34:10 +00:00
CrawlProfile.java replaced all 'new String' with default encoding (missing) or UTF-8 encoding with a String generation method that uses a pre-defined Charset constant for UTF-8. This avoids a cache-lookup for the Charset object using String hashing of the String 'UTF-8'. 2011-03-07 20:36:40 +00:00
CrawlQueues.java fix for 'invisible entry', see http://forum.yacy-websuche.de/viewtopic.php?p=22133#p22133 2011-03-09 17:04:34 +00:00
CrawlStacker.java - removed file upload function in crawl start and replaced it with an input field for a file path where the crawl start file is loaded. This was necessary to support the API steering for file crawl starts, for two reasons: 2011-03-09 12:50:39 +00:00
CrawlSwitchboard.java - fixed document number limitation for crawls that restrict the number of documents per domain 2011-02-12 00:01:40 +00:00
ImporterException.java added final where possible 2008-08-02 12:12:04 +00:00
Latency.java more memory protection: auto-flush of caches in case of memory shortage 2011-03-09 16:32:34 +00:00
NoticedURL.java - removed file upload function in crawl start and replaced it with an input field for a file path where the crawl start file is loaded. This was necessary to support the API steering for file crawl starts, for two reasons: 2011-03-09 12:50:39 +00:00
ResourceObserver.java same units for memory observer configuration (MiB) 2011-01-02 20:38:01 +00:00
ResultImages.java more memory protection: auto-flush of caches in case of memory shortage 2011-03-09 16:32:34 +00:00
ResultURLs.java redesigned the ScoreMap classes: 2011-03-13 01:41:44 +00:00
RobotsEntry.java replaced all 'new String' with default encoding (missing) or UTF-8 encoding with a String generation method that uses a pre-defined Charset constant for UTF-8. This avoids a cache-lookup for the Charset object using String hashing of the String 'UTF-8'. 2011-03-07 20:36:40 +00:00
robotsParser.java added a sitemap entry parser and loader for sitemaps 2010-11-03 19:48:33 +00:00
RobotsTxt.java replaced all 'new String' with default encoding (missing) or UTF-8 encoding with a String generation method that uses a pre-defined Charset constant for UTF-8. This avoids a cache-lookup for the Charset object using String hashing of the String 'UTF-8'. 2011-03-07 20:36:40 +00:00
RSSLoader.java *) set SVN properties 2011-03-08 01:51:51 +00:00
SitemapImporter.java enhanced crawler: 2010-12-11 00:31:57 +00:00
ZURL.java moved getBytes() to UTF8.getBytes() to use a default String encoding 2011-03-10 12:35:32 +00:00