yacy_search_server/source/de/anomic/crawler
2011-03-09 16:32:34 +00:00
..
retrieval *) set SVN properties 2011-03-08 01:51:51 +00:00
Balancer.java more memory protection: auto-flush of caches in case of memory shortage 2011-03-09 16:32:34 +00:00
CrawlProfile.java replaced all 'new String' with default encoding (missing) or UTF-8 encoding with a String generation method that uses a pre-defined Charset constant for UTF-8. This avoids a cache-lookup for the Charset object using String hashing of the String 'UTF-8'. 2011-03-07 20:36:40 +00:00
CrawlQueues.java replaced all 'new String' with default encoding (missing) or UTF-8 encoding with a String generation method that uses a pre-defined Charset constant for UTF-8. This avoids a cache-lookup for the Charset object using String hashing of the String 'UTF-8'. 2011-03-07 20:36:40 +00:00
CrawlStacker.java - removed file upload function in crawl start and replaced it with an input field for a file path where the crawl start file is loaded. This was necessary to support the API steering for file crawl starts, for two reasons: 2011-03-09 12:50:39 +00:00
CrawlSwitchboard.java - fixed document number limitation for crawls that restrict the number of documents per domain 2011-02-12 00:01:40 +00:00
ImporterException.java added final where possible 2008-08-02 12:12:04 +00:00
Latency.java more memory protection: auto-flush of caches in case of memory shortage 2011-03-09 16:32:34 +00:00
NoticedURL.java - removed file upload function in crawl start and replaced it with an input field for a file path where the crawl start file is loaded. This was necessary to support the API steering for file crawl starts, for two reasons: 2011-03-09 12:50:39 +00:00
ResourceObserver.java same units for memory observer configuration (MiB) 2011-01-02 20:38:01 +00:00
ResultImages.java more memory protection: auto-flush of caches in case of memory shortage 2011-03-09 16:32:34 +00:00
ResultURLs.java more memory protection: auto-flush of caches in case of memory shortage 2011-03-09 16:32:34 +00:00
RobotsEntry.java replaced all 'new String' with default encoding (missing) or UTF-8 encoding with a String generation method that uses a pre-defined Charset constant for UTF-8. This avoids a cache-lookup for the Charset object using String hashing of the String 'UTF-8'. 2011-03-07 20:36:40 +00:00
robotsParser.java added a sitemap entry parser and loader for sitemaps 2010-11-03 19:48:33 +00:00
RobotsTxt.java replaced all 'new String' with default encoding (missing) or UTF-8 encoding with a String generation method that uses a pre-defined Charset constant for UTF-8. This avoids a cache-lookup for the Charset object using String hashing of the String 'UTF-8'. 2011-03-07 20:36:40 +00:00
RSSLoader.java *) set SVN properties 2011-03-08 01:51:51 +00:00
SitemapImporter.java enhanced crawler: 2010-12-11 00:31:57 +00:00
ZURL.java performance hack: replaced generic row.getColBytes() call with row.getPrimaryKeyBytes() where the column is 0 2011-02-25 12:41:27 +00:00