yacy_search_server/source/de/anomic/crawler
orbiter 6388a58fc7 better memory management and slightly less (in total and temporary) RAM allocation:
- confirm that database objects that are not supposed to grow do not have a index memory management that is designed for growth
- changed index sorting method in such a way that it allocates less objects during quicksort
- database classes classes renaming (shorter, naming addresses that objects hold in RAM)
- added a large number of asserts to check if objects actually take the RAM that they should have


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7019 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-08-04 13:33:12 +00:00
..
retrieval ... migrating to HttpComponents-Client-4.x ... 2010-07-27 01:16:26 +00:00
AbstractImporter.java - cleanup, removed unused imports 2010-04-27 21:47:41 +00:00
Balancer.java - enhanced concurrency in database index access for multicore 2010-08-03 04:58:48 +00:00
CrawlProfile.java more abstraction of the htcache when using the LoaderDispatcher: 2010-06-21 14:54:54 +00:00
CrawlQueues.java ... migrating to HttpComponents-Client-4.x ... 2010-07-27 01:16:26 +00:00
CrawlStacker.java ... migrating to HttpComponents-Client-4.x ... 2010-07-27 01:16:26 +00:00
CrawlSwitchboard.java fixed a bug in snippet fetch strategy: cache only does not help if resource can only be found in web 2010-06-18 15:25:25 +00:00
Importer.java
ImporterException.java
ImporterManager.java *) some minor changes for better code readability 2010-04-05 12:37:33 +00:00
Latency.java better (and corrected) recognition of intranet and internet-addresses. This corrects the isLocal property that is used by network definitions to restrict index ranges to local and global addresses. Address locations (intranet or internet) had been partly identified by the top level domain of the host address. Since intranet addresses can also be addressed using a host name that is in a country domain it is necessary to do a dns resolving for each check. The check is supported by a local dns cache so the intranet/internet check should not affect network traffic too much. To ensure that the cache works properly the cache class was upgraded to better concurrency data structures. 2010-07-18 20:14:20 +00:00
NoticedURL.java continuing String-hash - to - byte[]-hash redesign that was started in SVN 6775 and continued in SVN 6790 2010-04-15 13:22:59 +00:00
ResourceObserver.java allow global search if res. observer disabled index transmission 2010-02-09 17:14:16 +00:00
ResultImages.java redesign of parser interface: 2010-06-29 19:20:45 +00:00
ResultURLs.java - more abstraction (HashMap -> Map) 2010-06-01 13:02:11 +00:00
RobotsEntry.java redesign of remote proxy settings 2010-05-26 00:01:16 +00:00
robotsParser.java - fixed a bug in robots.txt parser 2010-03-04 11:58:07 +00:00
RobotsTxt.java ... migrating to HttpComponents-Client-4.x ... 2010-07-27 01:16:26 +00:00
SitemapImporter.java applied code changes that are recommended by PMD 2010-01-10 23:09:48 +00:00
ZURL.java better memory management and slightly less (in total and temporary) RAM allocation: 2010-08-04 13:33:12 +00:00