orbiter 161d2fd2ef redesign of access to the HTCache (now http.client.Cache): ... - better control to the cache by using combined request-header and content access methods - refactoring of many classes to comply to this new access method - make shure that the cache is always written if something was loaded - some redesign of the process how http response results are feeded into the new indexing queue - introduction of a cache read policy: * never use the cache * use the cache if entry exist * use the cache if the proxy freshness rule confirmes * use only the cache and go never online - added configuration options for the crawl profiles to use the new cache policies. There is not yet a input during crawl start to set the policy but this will be added in another step. - set the default policies for the existing crawl profiles. If you want them to appear in your default profiles you must delete the crawl profiles database; othervise the policy is 'proxy freshness rule' - enhanced some cache access methods in such a way that unnecessary retrievals are omitted (i.e. for size computation). That should reduce some IO but also a lot of CPU computation because sizes were computed after decompression of content after retrieval of the content from the disc. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6239 6c8d7289-2bf4-0310-a012-ef5d649a1542		2009-07-23 21:31:51 +00:00
..
list	*) Classes to handle import of lists (especially blacklists) from XML files, not used yet, but will be used soon.	2009-04-05 13:36:44 +00:00
wiki	refactoring:	2009-07-19 20:37:44 +00:00
AbstractBlacklist.java	*) added new method "contains()" to Blacklist interface	2009-04-18 16:27:17 +00:00
Blacklist.java	*) added new method "contains()" to Blacklist interface	2009-04-18 16:27:17 +00:00
blogBoard.java	removed strange debugging strings	2009-07-06 15:21:43 +00:00
blogBoardComments.java	removed strange debugging strings	2009-07-06 15:21:43 +00:00
bookmarksDB.java	redesign of access to the HTCache (now http.client.Cache):	2009-07-23 21:31:51 +00:00
DefaultBlacklist.java	refactoring: better abstraction of reference and metadata prototypes.	2009-04-03 13:23:45 +00:00
diff.java	refactoring of parsers and document processing	2009-07-08 21:48:08 +00:00
listManager.java	refactoring:	2009-07-19 20:37:44 +00:00
messageBoard.java	removed strange debugging strings	2009-07-06 15:21:43 +00:00
SitemapParser.java	redesign of access to the HTCache (now http.client.Cache):	2009-07-23 21:31:51 +00:00
translator.java	removed the indexing queue. This queue was superfluous since the introduction of the blocking queues last year, where documents are parsed, analysed and stored in the index with concurrency.	2009-07-17 13:59:21 +00:00
URLAnalysis.java	- refactoring of IntegerHandleIndex and LongHandleIndex: both classes had been merged into the new HandleMap class, which handles (key<byte[]>,n-byte-long) pairs with arbitraty key and value length. This will be useful to get a memory-enhanced/minimized database table indexing.	2009-06-07 21:48:01 +00:00
URLLicense.java	small corrections to image loading methods in result presentation	2009-03-06 15:39:02 +00:00
userDB.java	refactoring:	2009-07-19 20:37:44 +00:00