Michael Peter Christen d6b82840f8 added a feature to find similarities in documents. ... This uses an enhanced version of the Nutch/Solr TextProfileSignatue. As a result, a signature of the document is written to the solr search index. Additionally for each time when a signature is written, it is checked if the singature exists already in the index. If the signature does not exist, the document is marked as unique. The unique attribute can now be used to sort document lists and bring duplicates to the end of a result list. To enable this, a large portion of the search api to Solr had to be changed. This affected mainly caching of 'exists' searches to enhance the check for existing signatures and do this without actually doing a solr query. Because here the first time a long number is used as value in the Solr store, also the value naming in the YaCySchema had to be adopted and normalized. This caused that many files had to be changed.		2012-11-21 18:46:49 +01:00
..
FileLoader.java	added a feature to find similarities in documents.	2012-11-21 18:46:49 +01:00
FTPLoader.java	update to Balancer algorithm:	2012-10-28 13:24:49 +01:00
HTTPLoader.java	replaced the custom robots.txt loader by the standard http loader	2012-10-28 22:48:11 +01:00
ImporterException.java	refactoring	2012-09-21 15:48:16 +02:00
Request.java	replaced the custom robots.txt loader by the standard http loader	2012-10-28 22:48:11 +01:00
Response.java	added a feature to find similarities in documents.	2012-11-21 18:46:49 +01:00
RSSLoader.java	removed the option to prevent removal of & parts inside of the	2012-10-10 11:46:22 +02:00
SitemapImporter.java	- removed dependencies from URIMetadataRow and made direct access to	2012-10-16 18:11:57 +02:00
SMBLoader.java	added a feature to find similarities in documents.	2012-11-21 18:46:49 +01:00
URLRewriterLibrary.java	added an url rewriter which can be used to remove session ids from urls	2012-10-09 11:24:48 +02:00