Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
8,718 Commits 1 Branch 20 Tags 836 MiB
aab0b680c3
Commit Graph

8718 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Michael Peter Christen
aab0b680c3 - added xslt support for solr result formats. 
try i.e.
http://localhost:8090/solr/select?q=*:*&start=0&rows=10&wt=xslt&tr=json.xsl
- added servlet-side mime-type configuration for streamed servlets. this
is used for the result formatters in solr result formats
 2012-08-14 11:12:50 +02:00
cominch
ad62609ec7 added a possibility to define a custom network definition URL for remote 
management
 2012-08-13 16:57:53 +02:00
cominch
fb0f430685 Merge remote-tracking branch 'original yacy/master' 2012-08-13 16:48:14 +02:00
Michael Peter Christen
9448d9a8a2 ups 2012-08-13 14:01:45 +02:00
Michael Peter Christen
e5ef840f40 - renamed DoubleSolrConnector to MirrorSolrConnector and added a 
hit/miss/document cache to the MirrorSolrConnector.
- more abstraction to SolrDocument in Connector interface
- bugfixes in Solr field reader
 2012-08-13 13:32:32 +02:00
Michael Peter Christen
94a334f128 another fix to the Solr metadata reading process and to the shutdown 
process
 2012-08-13 11:13:53 +02:00
Michael Peter Christen
b51df6c7e8 - added coordinate storage in solr schema 
- fixed shutdown process
- fixed some solr-to-metadata reading
- added a large number of metadata attributes in ViewFile.html
 2012-08-13 10:40:04 +02:00
Michael Peter Christen
da851c6071 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-08-11 01:21:18 +02:00
Michael Peter Christen
bd4f03bc85 removed unused class 2012-08-11 01:05:40 +02:00
orbiter
39f8eb60c3 tried to prevent calls to bad-hack getSize() method and reduced overhead 
of that method a bit.
 2012-08-10 18:10:25 +02:00
orbiter
9b88433f45 patch from hint in 
http://forum.yacy-websuche.de/viewtopic.php?p=26858#p26858
from gaston
 2012-08-10 15:44:37 +02:00
orbiter
e816b88b55 changed behaviour of metadata storage: in case that any solr is 
attached, the metadata is not written to the metadata-db, even if it is
enabled but instead to solr. This prevents that metadata is written in
two store systems at the same time. It is also the next step to migrate
the current metadata-db to solr.
 2012-08-10 15:39:10 +02:00
orbiter
2571e0d47a removed unused classes 2012-08-10 14:47:44 +02:00
Michael Peter Christen
f9c0e6e950 - Implemented and integrated the URIMetadataNode object which is a 
metadata representation from the solr index. This shall replace metadata
from the built-in database in the future.
- added the Solr-driven metadata into the search index of YaCy which
makes it now possible to run YaCy without the old metadata index. This
is a major stept forward to a full migration to Solr.
 2012-08-10 13:26:51 +02:00
Michael Peter Christen
b2b480fff2 more abstraction of the YaCySchema -> Opensearch matching process 2012-08-10 09:48:15 +02:00
Michael Peter Christen
aa0ef98ffa Merge branch 'master' of git://gitorious.org/~chalker/yacy/chalkers-yacy-rc1 2012-08-10 09:47:15 +02:00
Michael Peter Christen
73f6d69d03 more abstraction for solr query params parsing 2012-08-10 07:58:45 +02:00
Michael Peter Christen
24462e9baa set the title every time, it is possible that it has changed 2012-08-10 07:51:57 +02:00
Michael Peter Christen
dcc72799c4 better abstraction for result writers using controlled vocabularies and 
URIRefs
 2012-08-10 07:45:43 +02:00
Michael Peter Christen
136fcb1ad9 refactoring 2012-08-10 06:47:13 +02:00
Michael Peter Christen
a12f693ec9 added two response writer for embedded solr interface: 
a rss/opensearch writer and an enhanced solr xml writer.
The enhanced solr writer has less configuration overhead than the
original writer and should by slightly faster. The rss/opensearch writer
is at this time slightly incomplete compared with the already existing
rss search result form YaCy and also snippets are missing at this time.
To test the new interface, open for example:
http://localhost:8090/solr/select?wt=rss&q=olympia
The wt-code for the new result writers are=
wt=rss for opensearch
wt=exml for the enhanced solr xml writer.
Additionally, the SRU search parameters had been added to the solr
interface which can now also be used for a normal solr/xml search.
 2012-08-09 18:06:48 +02:00
Сковорода Никита Андреевич
792ecf2444 Fix an error in Russian translation: "can not" => "can". 2012-08-08 11:35:45 +04:00
Michael Peter Christen
bca4a16603 replaced the multivalue generic string field name suffix _ss by _txt 
because _ss is not part of the standard solr example schema.
 2012-08-06 17:58:09 +02:00
orbiter
67edfd991c Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-08-05 15:49:48 +02:00
orbiter
d9173ba7ed added more solr fields to integrate values from URIMetadataRow. All 
writings to the Metadata-DB are now also done to solr. This includes
metadata transfer during search and rwi transfer.

The new/added solr fields are:

## time when resource was loaded
load_date_dt

## date until resource shall be considered as fresh
fresh_date_dt

## id of the host, a 6-byte hash that is part of the document id
host_id_s

## ids of referrer to this document
referrer_id_ss

## the md5 of the raw source
md5_s

## the name of the publisher of the document
publisher_t

## the language used in the document; starts with primary language
language_ss

## an external ranking value
ranking_i

## the size of the raw source
size_i

## number of links to audio resources
audiolinkscount_i

## number of links to video resources
videolinkscount_i

## number of links to application resources
applinkscount_i
 2012-08-05 15:49:27 +02:00
Michael Peter Christen
70b10e8316 added the JSON response writer to solr interface, add &wt=json to the 
servlet GET properties to use this format
 2012-08-01 00:14:56 +02:00
Michael Peter Christen
3276508d1b Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-07-31 23:49:56 +02:00
Michael Peter Christen
3ce04cecf3 bad hack to prevent a bug appearing in solr 2012-07-31 23:49:07 +02:00
sixcooler
f32aa9a49c prevent merge of blobs that can't be handled in memory 2012-07-31 23:23:16 +02:00
Michael Peter Christen
bbd242afb4 fix for a NPE 2012-07-30 14:51:01 +02:00
Michael Peter Christen
8d944f6517 nowrap from gaston in forum 
http://forum.yacy-websuche.de/viewtopic.php?p=26815#p26815
 2012-07-30 12:39:47 +02:00
Michael Peter Christen
24d9db1613 snippet retrieval loading processes may use a smaller minimum load time 
value than crawling processes. This speeds up the search result
preparation dramatically.
 2012-07-30 10:38:23 +02:00
Michael Peter Christen
ef488a15f7 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-07-27 12:14:24 +02:00
Michael Peter Christen
1687737771 Abstraction of HandleMap and HandleSet 2012-07-27 12:13:53 +02:00
sixcooler
76b037a20a check content domain fix: 
search image/media should not show pages containing image/media
search text should show all/text but image/media
 2012-07-27 04:11:52 +02:00
sixcooler
9cd409682f close augmented stream if filled from cache to get its content 
use augmented stream if proxyAugmentation is set only
 2012-07-26 18:09:40 +02:00
Michael Peter Christen
e432bb9cd9 better calculation of possible saving in HeapReader index data structure 2012-07-26 10:05:06 +02:00
Michael Peter Christen
9549984c65 documentation/comments 2012-07-25 21:34:23 +02:00
Michael Peter Christen
beb6425f0c Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-07-25 21:18:30 +02:00
sixcooler
83c93e9209 no translation of queue-links 2012-07-25 15:35:13 +02:00
Michael Peter Christen
3bcd9d622b cleaned up classes and methods which are either superfluous at this time 
or will be superfluous or subject of complete redesign after the
migration to solr. Removing these things now will make the transition to
solr more simple.
 2012-07-25 14:31:54 +02:00
Michael Peter Christen
6f1ddb2519 Moved solr index-add method to the same method where the YaCy index is 
written. Also done some code-cleanup.
 2012-07-25 01:53:47 +02:00
Michael Peter Christen
315d83cfa0 cleanup 2012-07-24 22:16:56 +02:00
Michael Peter Christen
1f41d9c6f5 bugfix for a NPE 2012-07-24 17:29:32 +02:00
Michael Peter Christen
76202f068e extended abstraction of local and remote solr index using one front-end 
for index administration and querying.
 2012-07-24 17:23:29 +02:00
Michael Peter Christen
d3f243e2e1 fixed node type calculation for principal peers 2012-07-23 23:40:50 +02:00
Michael Peter Christen
7ec7341f60 added user-authentication protection to solr search (same as implemented 
for yacysearch)
 2012-07-23 21:43:14 +02:00
Michael Peter Christen
e2a97ef8f6 better explain how to access the embedded solr 2012-07-23 21:31:12 +02:00
Michael Peter Christen
826967513b changed options in IndexFederated_p to switch on/off parts of the index 
individually. The settings are experimental and the values of the
settings will be overwritten when an index migration from urldb to solr
starts.
 2012-07-23 16:28:39 +02:00
Michael Peter Christen
cba4ab862e fix for http://bugs.yacy.net/view.php?id=202 2012-07-23 00:36:18 +02:00