yacy_search_server/source/de/anomic/yacy
orbiter 19fd13d3bc Added federated index storage to solr.
YaCy supports now the storage to remote solr indexes.
More federated storage (and search) methods may follow.

The remote index scheme is the same as produced by the SolrCell; see
http://wiki.apache.org/solr/ExtractingRequestHandler
Because this default scheme is used, the default example scheme can be used as solr configuration
This is also the same scheme that solr uses if documents are imported with apache tika.

federated solr storage is switched off by default.

To use this, do the following:
- set federated.service.solr.indexing.enabled = true
- download solr from http://www.apache.org/dyn/closer.cgi/lucene/solr/
- extract the solr (3.1) package, 'cd example' and start solr with 'java -jar start.jar'
- start yacy and then start a crawler. The crawler will fill both, YaCy and solr indexes.
- to check whats in solr after indexing, open http://localhost:8983/solr/admin/

Until now it is not possible to use the solr index to search with YaCy in that solr index.
This functionality is now available for two reasons:
1) to compare the functionality of Solr and YaCy and to compare the search speed
2) to use YaCy as a search appliance for people who need a crawler or other source harvesting methods
   that YaCy provides (like dublin core reading, wikimedia dump reading, rss feed reader etc) if people still
   want to use solr instead of YaCy.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7654 6c8d7289-2bf4-0310-a012-ef5d649a1542
2011-04-14 20:05:04 +00:00
..
dht more UTF8 getBytes() performance hacks 2011-04-12 05:02:36 +00:00
graphics removed the DNS resolving for web structure computation from the indexing queue and placed it in a concurrent computation queue that does not block the crawler. Makes crawling faster and less DNS-speed-dependent 2011-04-04 22:01:07 +00:00
seedUpload - replaced more direct string-based UTF-8 conversions by predefined UTF-8 conversion 2011-03-10 23:25:07 +00:00
Tray.java *) set SVN properties 2011-03-08 01:51:51 +00:00
yacyAccessible.java more UTF8 getBytes() performance hacks 2011-04-12 05:02:36 +00:00
yacyBuildProperties.java replaced more appearance of double values by float values 2011-02-02 00:06:29 +00:00
yacyChannel.java *) set SVN properties 2011-03-08 01:51:51 +00:00
yacyClient.java Added federated index storage to solr. 2011-04-14 20:05:04 +00:00
yacyCore.java replaced all 'new String' with default encoding (missing) or UTF-8 encoding with a String generation method that uses a pre-defined Charset constant for UTF-8. This avoids a cache-lookup for the Charset object using String hashing of the String 'UTF-8'. 2011-03-07 20:36:40 +00:00
yacyNetwork.java *) set SVN properties 2011-03-08 01:51:51 +00:00
yacyNewsDB.java more UTF8 getBytes() performance hacks 2011-04-12 05:02:36 +00:00
yacyNewsPool.java *) General private License? ;-) 2011-01-12 00:28:08 +00:00
yacyNewsQueue.java more UTF8 getBytes() performance hacks 2011-04-12 05:02:36 +00:00
yacyPeerActions.java more UTF8 getBytes() performance hacks 2011-04-12 05:02:36 +00:00
yacyRelease.java * fix bug #6 2011-03-20 17:59:58 +00:00
yacySearch.java more UTF8 getBytes() performance hacks 2011-04-12 05:02:36 +00:00
yacySeed.java more UTF8 getBytes() performance hacks 2011-04-12 05:02:36 +00:00
yacySeedDB.java more UTF8 getBytes() performance hacks 2011-04-12 05:02:36 +00:00
yacySeedUploader.java extended seed list uploader: do not only upload all active peers but also some more peers that are passive but had been active in the last 24 hours 2011-01-26 23:21:33 +00:00
yacyUpdateLocation.java *) set SVN properties 2011-03-08 01:51:51 +00:00
yacyVersion.java moved all Double values to Float values because of 2011-02-01 23:49:11 +00:00