Michael Peter Christen
e5ef840f40
- renamed DoubleSolrConnector to MirrorSolrConnector and added a
...
hit/miss/document cache to the MirrorSolrConnector.
- more abstraction to SolrDocument in Connector interface
- bugfixes in Solr field reader
2012-08-13 13:32:32 +02:00
Michael Peter Christen
94a334f128
another fix to the Solr metadata reading process and to the shutdown
...
process
2012-08-13 11:13:53 +02:00
Michael Peter Christen
b51df6c7e8
- added coordinate storage in solr schema
...
- fixed shutdown process
- fixed some solr-to-metadata reading
- added a large number of metadata attributes in ViewFile.html
2012-08-13 10:40:04 +02:00
Michael Peter Christen
da851c6071
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
2012-08-11 01:21:18 +02:00
Michael Peter Christen
bd4f03bc85
removed unused class
2012-08-11 01:05:40 +02:00
orbiter
39f8eb60c3
tried to prevent calls to bad-hack getSize() method and reduced overhead
...
of that method a bit.
2012-08-10 18:10:25 +02:00
orbiter
9b88433f45
patch from hint in
...
http://forum.yacy-websuche.de/viewtopic.php?p=26858#p26858
from gaston
2012-08-10 15:44:37 +02:00
orbiter
e816b88b55
changed behaviour of metadata storage: in case that any solr is
...
attached, the metadata is not written to the metadata-db, even if it is
enabled but instead to solr. This prevents that metadata is written in
two store systems at the same time. It is also the next step to migrate
the current metadata-db to solr.
2012-08-10 15:39:10 +02:00
orbiter
2571e0d47a
removed unused classes
2012-08-10 14:47:44 +02:00
Michael Peter Christen
f9c0e6e950
- Implemented and integrated the URIMetadataNode object which is a
...
metadata representation from the solr index. This shall replace metadata
from the built-in database in the future.
- added the Solr-driven metadata into the search index of YaCy which
makes it now possible to run YaCy without the old metadata index. This
is a major stept forward to a full migration to Solr.
2012-08-10 13:26:51 +02:00
Michael Peter Christen
b2b480fff2
more abstraction of the YaCySchema -> Opensearch matching process
2012-08-10 09:48:15 +02:00
Michael Peter Christen
aa0ef98ffa
Merge branch 'master' of git://gitorious.org/~chalker/yacy/chalkers-yacy-rc1
2012-08-10 09:47:15 +02:00
Michael Peter Christen
73f6d69d03
more abstraction for solr query params parsing
2012-08-10 07:58:45 +02:00
Michael Peter Christen
24462e9baa
set the title every time, it is possible that it has changed
2012-08-10 07:51:57 +02:00
Michael Peter Christen
dcc72799c4
better abstraction for result writers using controlled vocabularies and
...
URIRefs
2012-08-10 07:45:43 +02:00
Michael Peter Christen
136fcb1ad9
refactoring
2012-08-10 06:47:13 +02:00
Michael Peter Christen
a12f693ec9
added two response writer for embedded solr interface:
...
a rss/opensearch writer and an enhanced solr xml writer.
The enhanced solr writer has less configuration overhead than the
original writer and should by slightly faster. The rss/opensearch writer
is at this time slightly incomplete compared with the already existing
rss search result form YaCy and also snippets are missing at this time.
To test the new interface, open for example:
http://localhost:8090/solr/select?wt=rss&q=olympia
The wt-code for the new result writers are=
wt=rss for opensearch
wt=exml for the enhanced solr xml writer.
Additionally, the SRU search parameters had been added to the solr
interface which can now also be used for a normal solr/xml search.
2012-08-09 18:06:48 +02:00
Сковорода Никита Андреевич
792ecf2444
Fix an error in Russian translation: "can not" => "can".
2012-08-08 11:35:45 +04:00
Michael Peter Christen
bca4a16603
replaced the multivalue generic string field name suffix _ss by _txt
...
because _ss is not part of the standard solr example schema.
2012-08-06 17:58:09 +02:00
orbiter
67edfd991c
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
2012-08-05 15:49:48 +02:00
orbiter
d9173ba7ed
added more solr fields to integrate values from URIMetadataRow. All
...
writings to the Metadata-DB are now also done to solr. This includes
metadata transfer during search and rwi transfer.
The new/added solr fields are:
## time when resource was loaded
load_date_dt
## date until resource shall be considered as fresh
fresh_date_dt
## id of the host, a 6-byte hash that is part of the document id
host_id_s
## ids of referrer to this document
referrer_id_ss
## the md5 of the raw source
md5_s
## the name of the publisher of the document
publisher_t
## the language used in the document; starts with primary language
language_ss
## an external ranking value
ranking_i
## the size of the raw source
size_i
## number of links to audio resources
audiolinkscount_i
## number of links to video resources
videolinkscount_i
## number of links to application resources
applinkscount_i
2012-08-05 15:49:27 +02:00
Michael Peter Christen
70b10e8316
added the JSON response writer to solr interface, add &wt=json to the
...
servlet GET properties to use this format
2012-08-01 00:14:56 +02:00
Michael Peter Christen
3276508d1b
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
2012-07-31 23:49:56 +02:00
Michael Peter Christen
3ce04cecf3
bad hack to prevent a bug appearing in solr
2012-07-31 23:49:07 +02:00
sixcooler
f32aa9a49c
prevent merge of blobs that can't be handled in memory
2012-07-31 23:23:16 +02:00
Michael Peter Christen
bbd242afb4
fix for a NPE
2012-07-30 14:51:01 +02:00
Michael Peter Christen
8d944f6517
nowrap from gaston in forum
...
http://forum.yacy-websuche.de/viewtopic.php?p=26815#p26815
2012-07-30 12:39:47 +02:00
Michael Peter Christen
24d9db1613
snippet retrieval loading processes may use a smaller minimum load time
...
value than crawling processes. This speeds up the search result
preparation dramatically.
2012-07-30 10:38:23 +02:00
Michael Peter Christen
ef488a15f7
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
2012-07-27 12:14:24 +02:00
Michael Peter Christen
1687737771
Abstraction of HandleMap and HandleSet
2012-07-27 12:13:53 +02:00
sixcooler
76b037a20a
check content domain fix:
...
search image/media should not show pages containing image/media
search text should show all/text but image/media
2012-07-27 04:11:52 +02:00
sixcooler
9cd409682f
close augmented stream if filled from cache to get its content
...
use augmented stream if proxyAugmentation is set only
2012-07-26 18:09:40 +02:00
Michael Peter Christen
e432bb9cd9
better calculation of possible saving in HeapReader index data structure
2012-07-26 10:05:06 +02:00
Michael Peter Christen
9549984c65
documentation/comments
2012-07-25 21:34:23 +02:00
Michael Peter Christen
beb6425f0c
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
2012-07-25 21:18:30 +02:00
sixcooler
83c93e9209
no translation of queue-links
2012-07-25 15:35:13 +02:00
Michael Peter Christen
3bcd9d622b
cleaned up classes and methods which are either superfluous at this time
...
or will be superfluous or subject of complete redesign after the
migration to solr. Removing these things now will make the transition to
solr more simple.
2012-07-25 14:31:54 +02:00
Michael Peter Christen
6f1ddb2519
Moved solr index-add method to the same method where the YaCy index is
...
written. Also done some code-cleanup.
2012-07-25 01:53:47 +02:00
Michael Peter Christen
315d83cfa0
cleanup
2012-07-24 22:16:56 +02:00
Michael Peter Christen
1f41d9c6f5
bugfix for a NPE
2012-07-24 17:29:32 +02:00
Michael Peter Christen
76202f068e
extended abstraction of local and remote solr index using one front-end
...
for index administration and querying.
2012-07-24 17:23:29 +02:00
Michael Peter Christen
d3f243e2e1
fixed node type calculation for principal peers
2012-07-23 23:40:50 +02:00
Michael Peter Christen
7ec7341f60
added user-authentication protection to solr search (same as implemented
...
for yacysearch)
2012-07-23 21:43:14 +02:00
Michael Peter Christen
e2a97ef8f6
better explain how to access the embedded solr
2012-07-23 21:31:12 +02:00
Michael Peter Christen
826967513b
changed options in IndexFederated_p to switch on/off parts of the index
...
individually. The settings are experimental and the values of the
settings will be overwritten when an index migration from urldb to solr
starts.
2012-07-23 16:28:39 +02:00
Michael Peter Christen
cba4ab862e
fix for http://bugs.yacy.net/view.php?id=202
2012-07-23 00:36:18 +02:00
Michael Peter Christen
b76836db7b
Merge branch 'master' of git://gitorious.org/~reger/yacy/bbyacy-rc1
2012-07-23 00:35:14 +02:00
reger
36c9875b6e
removed localized number formatting from num-results_totalcount response (this is only used in xml and json where localized format is not valid)
2012-07-23 00:00:40 +02:00
Michael Peter Christen
0640a6f7e6
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
2012-07-22 21:50:44 +02:00
orbiter
69e743d9e3
- more abstraction for the RWI index as preparation for solr integration
...
- added options in search index to switch parts of the index on or off
2012-07-22 13:18:45 +02:00