Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
10,181 Commits 1 Branch 20 Tags 836 MiB
20b48f894f
Commit Graph

6741 Commits

Author SHA1 Message Date
Michael Peter Christen
20b48f894f refactoring: moving all servlets to the same package (the solr servlet 
is currently actually a filter which should be changed somehow)
 2013-12-23 01:32:29 +01:00
Michael Peter Christen
84167adb49 removed unused anomichttpd code after migration to jetty 2013-12-23 01:23:40 +01:00
Michael Peter Christen
b461a27abb fixed the SolrServlet 2013-12-20 01:51:51 +01:00
Michael Peter Christen
7603e879dc Merge branch 'master' into HEAD 
Conflicts:
	.classpath
	source/net/yacy/cora/federate/solr/SolrServlet.java
 2013-12-20 01:19:06 +01:00
Michael Peter Christen
25250405f1 solr servlet preparation for join with jetty branch 2013-12-20 00:45:58 +01:00
Michael Peter Christen
2f16770681 migrated to solr 4.6.0 2013-12-19 21:51:05 +01:00
Michael Peter Christen
57f0f71ac6 added patch to allow binary response writer 2013-12-19 10:13:43 +01:00
orbiter
937273d4e3 added parsing of metadata to surrogate reading: 
a dublin core record inside of surrogate input files may now contain
tokens within the namespace 'md' (short for: metadata). The token names
must be valid withing the namespace of the solr field names. All
md-tokens inside of surrogate files then overwrite values within solr
documents before they are written to the solr index. This makes it
possible to assign collection names to each surrogate entry and also
ranking information can be added. Please see the example file.
 2013-12-17 14:02:27 +01:00
reger
18497f6475 remove unused init parameter from DefaultServlet 
- remove "RelativeResourceBase" parameter
 2013-12-15 23:39:19 +01:00
orbiter
4de3fefdb5 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-12-15 19:13:00 +01:00
orbiter
7e346e1d79 using stringbuilder in query construction 2013-12-15 19:12:49 +01:00
reger
c84c313fe1 Merge origin/master into jetty 2013-12-14 20:02:24 +01:00
Michael Peter Christen
2702d9e56b - added a SolrQueryResponse2SolrDocumentList method which is able to 
work around the unfolding process in Solr's BinaryResponseWriter.
This was a huge performance bottleneck in the embedded solr connector
and the problem is actually on Solr side, but we have now a workaround.
- This made it possible to abstract a high-performance index access
method which is implemented as method getDocumentListByParams. That
method is also implemented in the SolrServerConnector and provides a
very efficient access to a solr index if the index is embedded.
- a popular use of the document list retrieval is a result count which
can now also make use of the new method, via getDocumentCountByParams.
- enhanced the Error cache which now does not store error documents
within the ram cache if the document is also written to solr. When
documents are retrieved from the cache, they are partly read from the
ram cache and if not existent there, from the Solr index.
 2013-12-13 15:56:29 +01:00
Michael Peter Christen
74466d731a use pre-compiled patterns in ymark 2013-12-12 11:50:48 +01:00
Michael Peter Christen
34633044b4 made pattern computation static 2013-12-12 10:55:36 +01:00
Michael Peter Christen
ef7ddbc933 added date parser caches to prevent re-calculation of costly date 
parsing
 2013-12-12 10:55:12 +01:00
Michael Peter Christen
552ef9f18e fix for bad ErrorCache.exists test (bug from latest commit) 2013-12-12 10:38:32 +01:00
Michael Peter Christen
09412ea3a4 counting search requests in solr interface 2013-12-12 03:37:19 +01:00
Michael Peter Christen
303f5694ba avoid usage of existsByQuery. If a document can be loaded by the ID 
before testing other fields from the existsByQuery request, then a
document cache fills and queries after that one can be avoided.
 2013-12-12 03:36:30 +01:00
reger
b43bbd3cc4 join DefaultServlet and Jetty8 implementation 
- removing Jetty 8 specific dependencies
 2013-12-09 23:45:57 +01:00
reger
089c5007ee move conditionalHeader to DefaultServlet 
- by removing Jetty specific implementation detail
 2013-12-08 00:56:45 +01:00
Michael Peter Christen
79771c60c0 IPv6 fixes 2013-12-06 14:30:08 +01:00
reger
92d9c56f9f Merge origin/master into jetty 2013-12-05 22:53:29 +01:00
Michael Peter Christen
78eac85161 better calibration of caches and queue maximum sizes 2013-12-04 23:15:10 +01:00
Michael Peter Christen
c8af19bd37 removed unnecessary check which causes a NPE when searching with empty 
search string
 2013-12-04 17:58:36 +01:00
Michael Peter Christen
e3c2f09de9 - reduce computation in case that specific postprocessing fields are not 
selected
- de-select citation rank computation
 2013-12-04 17:48:12 +01:00
Michael Peter Christen
cfa08024c7 removed optimization bevore postprocessing because that may cause a 
time-out which will cause that postprocessing fails.
 2013-12-04 16:04:29 +01:00
Michael Peter Christen
6f3a923691 fixed urlmask which was not able to combine several constraints 2013-12-04 13:48:01 +01:00
Michael Peter Christen
9a27bf6e82 removed filter computation in Protocol class for remote searches because 
that is already done in the QueryParams class
 2013-12-04 13:09:15 +01:00
Michael Peter Christen
f1b5db2c45 - performance graph does not shop peer ping in memory monitor any more 
- after a forced GC, the PerformanceMemory view switches to automatic
update by default
 2013-12-04 12:59:30 +01:00
Michael Peter Christen
a125904a1c fixed a NPE in surrogat processing 2013-12-04 01:56:38 +01:00
Michael Peter Christen
0db8e34625 enhanced webgraph processing 2013-12-04 01:54:45 +01:00
reger
ac067b5236 clean-up Jetty handler classes 2013-12-01 19:36:24 +01:00
reger
b75e92aac3 add read queryparameter in gsaservlet 2013-11-30 06:29:57 +01:00
reger
1e94719084 fix NPE on mime detection of unknown file extension 2013-11-29 23:23:47 +01:00
reger
effea4bca0 Merge origin/master into jetty 
Conflicts:
	source/net/yacy/cora/federate/solr/SolrServlet.java
 2013-11-29 22:39:52 +01:00
sixcooler
2c2ebb0d92 tried some hardening in order not letting any Solr-Searchers open 2013-11-29 02:40:12 +01:00
Michael Peter Christen
a16534cb0a tried to fix timeout and connection-lost problems when using an outside 
solr.
 2013-11-28 01:31:53 +01:00
Michael Peter Christen
c3dcbdc8d5 try to recover from an OOM during citation index reading and fail-over 
to second solr core in case of unrecoverable OOM.
 2013-11-28 01:10:25 +01:00
Michael Peter Christen
9932c441c8 fixed a problem with Date fields parsing Solr results if a remote Solr 
is attached.
 2013-11-28 00:54:53 +01:00
sixcooler
94db054aff memory-leak-fix: the DocListSearcher fires an query in its constructor 
and it is highly recommend to close every SolrRequest.
Every Request, which is not closed leaves a Searcher with its Chaches an
can not be garbage-collectet.
 2013-11-27 19:07:36 +01:00
reger
26bb1e37b7 implement core selection in SolrServlet 
- making initcore() obsolete
 2013-11-27 02:51:02 +01:00
Michael Peter Christen
ae55d69ef6 include/exclude size NPE fix (recently added) 2013-11-26 11:47:04 +01:00
Michael Peter Christen
2c39b65409 fixes for searches containing stopwords. The fix was done using a 
reconstruction of the search word set access method to protect that
words are deleted from the sets from the outside of the QueryGoal class.
 2013-11-26 02:24:47 +01:00
Michael Peter Christen
5592ea57f0 hack to remove compiler warnings about deprecated classes. It would be 
better to remove the deprecated usage but to do this the Solr core must
adopt the latest apache http core changes as well .. this is not our
fault.
 2013-11-25 23:30:35 +01:00
orbiter
037cd0a57c using the BinaryResponseWriter which is supported within the YaCy solr 
servlet since YaCy 1.63. This is much more performant for the client
than using the XMLResponseWriter because parsing of XML data is very CPU
intensive. Older YaCy peers are still requested using the
XMLResponseWriter but the majority of YaCy peers already respond with
the binary writer. This makes remote searches much faster and less CPU
intensive.
 2013-11-25 21:31:40 +01:00
orbiter
61409788eb less word hash computations (removing some overhead because of MD5 
calcs) using the clear word in a normalized form.
 2013-11-25 15:20:54 +01:00
reger
f23471c471 add check to prevent index entries containing url_file_ext_s with ";jsession=xyz" 
note: check could be implemented in MultiProtocolURL (but at this time didn't oversee possible implication)
 2013-11-25 00:14:53 +01:00
reger
5c4a3d1c01 Merge origin/master into jetty 2013-11-24 21:00:39 +01:00
reger
444a9ae674 remove unused options and attributes from DefaultServlet 
cleanup obsolete class files
 2013-11-24 20:11:39 +01:00