orbiter
7cd302de3e
omit xml parsing when using the embedded solr server
2012-08-24 12:18:30 +02:00
orbiter
787e1c6836
added the
...
QueryResponse query(SolrParams params)
method to the SolrServerConnector which is necessary to use facets in
solr search.
2012-08-23 11:53:54 +02:00
orbiter
01a63ef595
redesign of YaCySchema and SolrDoc handling
2012-08-23 09:51:45 +02:00
orbiter
479bfca571
refctoring
2012-08-23 09:30:11 +02:00
Michael Peter Christen
48a82bc705
log queries anonymous from gsa+solr requests
2012-08-22 23:50:40 +02:00
Michael Peter Christen
ab6ec4ec52
added snippet computation to solr/rss and gsa result writer
2012-08-22 17:37:34 +02:00
Michael Peter Christen
4716546ef5
- reduced memory usage in index transmission using a transformation of
...
Node to Row objects
- removed peerDeparture in solr remote search in case that peer does not
answer (this may be normal because it is allowed to switch this off)
2012-08-22 16:30:33 +02:00
Michael Peter Christen
06b0081fdc
fix for NPE during host navigation computation
2012-08-22 01:55:39 +02:00
Michael Peter Christen
feb99bc291
fixed GSA format
2012-08-22 00:48:37 +02:00
Michael Peter Christen
653645c1cf
corrected solr query syntax
2012-08-22 00:48:03 +02:00
Michael Peter Christen
08ae142a3d
- enhanced caching after search queries to solr
...
- reduced caching after short memory
2012-08-22 00:31:14 +02:00
orbiter
716ea0cfe2
sorted the solr schema into mandatory and optional fields; reduced
...
number of used field to reduce solr index size
2012-08-21 23:52:56 +02:00
orbiter
9b8c8c0f47
fix from gaston in
...
http://forum.yacy-websuche.de/viewtopic.php?p=26909#p26909
2012-08-21 21:03:26 +02:00
orbiter
acb9f04e80
removed unused classes
2012-08-21 18:18:30 +02:00
Michael Peter Christen
0ad52ac4c3
gsa bugfix for date parser
2012-08-21 02:39:28 +02:00
Michael Peter Christen
3ce4c2f937
fixes for gsa result format
2012-08-21 01:57:46 +02:00
Michael Peter Christen
67d235fae9
added gzip encoding to solr2sor http interface, client side (server
...
already works)
2012-08-20 16:53:21 +02:00
Michael Peter Christen
a049761e0c
fixed double-check
2012-08-20 14:16:37 +02:00
Michael Peter Christen
f42a57cd7d
gsa format update
2012-08-20 12:50:51 +02:00
Michael Peter Christen
b3aad6cc35
bugfix for remote search when search is done to solr
2012-08-20 12:21:36 +02:00
Michael Peter Christen
ff3eaa21b0
added remote search to solr on YaCy peers!
...
- when doing a remote search, node peers are selected for solr queries
- the solr query is done concurrently to the standard YaCy rwi search
- the solr search result is feeded into the same data structure that
prepares the rwi search result
- the same remote seach that is done to several outside peers is done to
the local solr index
- the search process works now also without any 'old' RWI data using
solr
2012-08-20 12:16:11 +02:00
Michael Peter Christen
a06123aec6
more abstraction and less parameter overhead for remote search
2012-08-20 01:29:15 +02:00
Michael Peter Christen
f00733186b
code simplifications
2012-08-19 13:17:03 +02:00
Michael Peter Christen
755f5e76cf
removed strange assert statements and simplified code in metadata
...
transformation
2012-08-19 08:44:39 +02:00
Michael Peter Christen
db0d438709
fix for http://bugs.yacy.net/view.php?id=206
2012-08-19 08:43:56 +02:00
orbiter
404b0aab09
refactoring in remote search and stub for remote node peer selection
2012-08-18 23:59:25 +02:00
orbiter
d7ea45f698
- get nice text_t values from metadata conversions that are stored into
...
solr as fulltext search index.
- added slow migration from old metadata to solr index entries: each
entry from the old metadata is removed from that data structure and
written into solr.
2012-08-18 19:36:21 +02:00
orbiter
99ef57f103
reduced sleep times
2012-08-18 17:48:20 +02:00
orbiter
780f8974e7
added ramaining iteration methods for solr in fulltext class
2012-08-18 15:39:14 +02:00
orbiter
acd2dc3575
hack to removed StringBuilder overhead in query construction
2012-08-18 14:22:00 +02:00
orbiter
ee01c12e56
fixes for putDocument and putMetadata
2012-08-18 13:05:27 +02:00
orbiter
cc47a0876e
reverted bf55f69176
...
to have a fall-back option in case that memory problems as reported in
http://forum.yacy-websuche.de/viewtopic.php?p=26901#p26901
for full-solr installation are too strong and we have to work with an
'small memory footprint' peer system.
2012-08-18 10:28:40 +02:00
Michael Peter Christen
0904afe8fb
added concurrent iterator methods to the solr connectors
2012-08-17 18:22:56 +02:00
Michael Peter Christen
d54b80327a
refactoring
2012-08-17 17:28:27 +02:00
Michael Peter Christen
f9fc5cfaba
better check for bad urls in url transmission
2012-08-17 17:17:00 +02:00
Michael Peter Christen
d39463a85c
added deleteByQuery to solr connectors
2012-08-17 17:05:46 +02:00
Michael Peter Christen
0cab06c47c
refactoring
2012-08-17 15:52:33 +02:00
Michael Peter Christen
bf55f69176
removed write methods to old metadata file type; all metadata now goes
...
to solr
2012-08-17 15:46:26 +02:00
Michael Peter Christen
40c0856489
refactoring
2012-08-17 15:33:02 +02:00
Michael Peter Christen
06a78eecb7
code simplification
2012-08-17 14:43:32 +02:00
Michael Peter Christen
54bea21c02
bugfix for solr connector, possibly a cause for
...
http://forum.yacy-websuche.de/viewtopic.php?p=26893#p26893
2012-08-17 14:34:31 +02:00
Michael Peter Christen
9bece5ac5f
enhanced snippet fetch - removed a bug that caused documents to be
...
parsed even if a solr text was available
2012-08-17 14:22:07 +02:00
Michael Peter Christen
18f989dfb1
- refactoring (load -> getMetadata)
...
- added getDocument to retrieve Solr documents which shall replace
getMetadata
2012-08-17 01:34:38 +02:00
Michael Peter Christen
395b78a0d8
using the solr search index to concurrently search within solr and the
...
rwis during local search requests.
2012-08-17 01:21:56 +02:00
Michael Peter Christen
6197caf698
added clear-text search words in query params
2012-08-16 23:05:37 +02:00
Michael Peter Christen
23226676c6
FOR THE BRAVE.. this is a forced migration to solr which is now ready
...
for production as a replacement of the metadata-db.
This intermediate release 1.041 will switch on the previously optional
solr index and the old metadata-db will still work as it did before.
Solr+metadata are accessed in mixed mode, no migration is done yet.
If this causes not a catastrophe until the end of the weekend, we will
do a YaCy 1.1 main release containing this as default.
2012-08-16 18:17:47 +02:00
Michael Peter Christen
a1b2c9a67d
doctype2mime fix, influences metadata conversion between old metadata
...
and solr
2012-08-16 17:49:35 +02:00
Michael Peter Christen
a16206e38b
more attempts to clean the index (cleaning is faster then)
2012-08-16 17:24:25 +02:00
Michael Peter Christen
703f427303
fixed some peer-ping connection details
...
- larger time-out
- removed too old seedlist
- fixed a bug in connection test
2012-08-16 17:11:54 +02:00
Michael Peter Christen
597bb76e4f
get the peer location more quickly
2012-08-16 16:28:57 +02:00
