Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
8,921 Commits 1 Branch 20 Tags 836 MiB
be4c96f3b1
Commit Graph

8921 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Michael Peter Christen
2d5fdfeb65 added authorization-based maximum results limitation to solr and gsa 
search
 2012-08-20 17:10:48 +02:00
Michael Peter Christen
67d235fae9 added gzip encoding to solr2sor http interface, client side (server 
already works)
 2012-08-20 16:53:21 +02:00
Michael Peter Christen
a049761e0c fixed double-check 2012-08-20 14:16:37 +02:00
Michael Peter Christen
6fc5400f91 added a tooltip for search navigation to mention that search pages can 
be navigated using the TAB key
 2012-08-20 13:02:29 +02:00
Michael Peter Christen
f42a57cd7d gsa format update 2012-08-20 12:50:51 +02:00
Michael Peter Christen
b3aad6cc35 bugfix for remote search when search is done to solr 2012-08-20 12:21:36 +02:00
Michael Peter Christen
ff3eaa21b0 added remote search to solr on YaCy peers! 
- when doing a remote search, node peers are selected for solr queries
- the solr query is done concurrently to the standard YaCy rwi search
- the solr search result is feeded into the same data structure that
prepares the rwi search result
- the same remote seach that is done to several outside peers is done to
the local solr index
- the search process works now also without any 'old' RWI data using
solr
 2012-08-20 12:16:11 +02:00
Michael Peter Christen
a06123aec6 more abstraction and less parameter overhead for remote search 2012-08-20 01:29:15 +02:00
Michael Peter Christen
f00733186b code simplifications 2012-08-19 13:17:03 +02:00
Michael Peter Christen
755f5e76cf removed strange assert statements and simplified code in metadata 
transformation
 2012-08-19 08:44:39 +02:00
Michael Peter Christen
db0d438709 fix for http://bugs.yacy.net/view.php?id=206 2012-08-19 08:43:56 +02:00
orbiter
404b0aab09 refactoring in remote search and stub for remote node peer selection 2012-08-18 23:59:25 +02:00
orbiter
d7ea45f698 - get nice text_t values from metadata conversions that are stored into 
solr as fulltext search index.
- added slow migration from old metadata to solr index entries: each
entry from the old metadata is removed from that data structure and
written into solr.
 2012-08-18 19:36:21 +02:00
orbiter
99ef57f103 reduced sleep times 2012-08-18 17:48:20 +02:00
orbiter
780f8974e7 added ramaining iteration methods for solr in fulltext class 2012-08-18 15:39:14 +02:00
orbiter
acd2dc3575 hack to removed StringBuilder overhead in query construction 2012-08-18 14:22:00 +02:00
orbiter
db6863db77 reduced solr cache sizes to check if that solves memory problems a bit 2012-08-18 13:45:37 +02:00
orbiter
6f01542aaa explicit double-check in transferURL 2012-08-18 13:18:51 +02:00
orbiter
ee01c12e56 fixes for putDocument and putMetadata 2012-08-18 13:05:27 +02:00
orbiter
cc47a0876e reverted bf55f69176 
to have a fall-back option in case that memory problems as reported in
http://forum.yacy-websuche.de/viewtopic.php?p=26901#p26901
for full-solr installation are too strong and we have to work with an
'small memory footprint' peer system.
 2012-08-18 10:28:40 +02:00
Michael Peter Christen
0904afe8fb added concurrent iterator methods to the solr connectors 2012-08-17 18:22:56 +02:00
Michael Peter Christen
d54b80327a refactoring 2012-08-17 17:28:27 +02:00
Michael Peter Christen
f9fc5cfaba better check for bad urls in url transmission 2012-08-17 17:17:00 +02:00
Michael Peter Christen
d39463a85c added deleteByQuery to solr connectors 2012-08-17 17:05:46 +02:00
Michael Peter Christen
0cab06c47c refactoring 2012-08-17 15:52:33 +02:00
Michael Peter Christen
bf55f69176 removed write methods to old metadata file type; all metadata now goes 
to solr
 2012-08-17 15:46:26 +02:00
Michael Peter Christen
40c0856489 refactoring 2012-08-17 15:33:02 +02:00
Michael Peter Christen
2ccf1dba71 upgrade to solr 3.6.1 2012-08-17 15:11:21 +02:00
Michael Peter Christen
e651d3e320 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-08-17 14:45:18 +02:00
Michael Peter Christen
06a78eecb7 code simplification 2012-08-17 14:43:32 +02:00
Michael Peter Christen
54bea21c02 bugfix for solr connector, possibly a cause for 
http://forum.yacy-websuche.de/viewtopic.php?p=26893#p26893
 2012-08-17 14:34:31 +02:00
Michael Peter Christen
9bece5ac5f enhanced snippet fetch - removed a bug that caused documents to be 
parsed even if a solr text was available
 2012-08-17 14:22:07 +02:00
cominch
8a91f4fa42 local robots.txt: disallow external crawlers to follow the URL proxy 2012-08-17 11:47:39 +02:00
Michael Peter Christen
18f989dfb1 - refactoring (load -> getMetadata) 
- added getDocument to retrieve Solr documents which shall replace
getMetadata
 2012-08-17 01:34:38 +02:00
Michael Peter Christen
395b78a0d8 using the solr search index to concurrently search within solr and the 
rwis during local search requests.
 2012-08-17 01:21:56 +02:00
Michael Peter Christen
6197caf698 added clear-text search words in query params 2012-08-16 23:05:37 +02:00
Michael Peter Christen
efafa79db5 - added a content-encoding: gzip to streamed http server responses 
- finish and close streamed http responses immediately
- this applies only to the solr interface which should be much faster
now!
 2012-08-16 22:35:19 +02:00
Michael Peter Christen
23226676c6 FOR THE BRAVE.. this is a forced migration to solr which is now ready 
for production as a replacement of the metadata-db.
This intermediate release 1.041 will switch on the previously optional
solr index and the old metadata-db will still work as it did before.
Solr+metadata are accessed in mixed mode, no migration is done yet.
If this causes not a catastrophe until the end of the weekend, we will
do a YaCy 1.1 main release containing this as default.
 2012-08-16 18:17:47 +02:00
Michael Peter Christen
a1b2c9a67d doctype2mime fix, influences metadata conversion between old metadata 
and solr
 2012-08-16 17:49:35 +02:00
Michael Peter Christen
7c31be1c80 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-08-16 17:45:26 +02:00
cominch
6456a1656a changed local robots.txt to prevent external crawlers to submit random 
search queries
 2012-08-16 17:38:10 +02:00
Michael Peter Christen
a16206e38b more attempts to clean the index (cleaning is faster then) 2012-08-16 17:24:25 +02:00
Michael Peter Christen
703f427303 fixed some peer-ping connection details 
- larger time-out
- removed too old seedlist
- fixed a bug in connection test
 2012-08-16 17:11:54 +02:00
Michael Peter Christen
597bb76e4f get the peer location more quickly 2012-08-16 16:28:57 +02:00
orbiter
156d457aec fix for Index out of bounds exception in Network servlet 2012-08-16 07:47:52 +02:00
orbiter
da93addec3 addon to e74d66e28c 
(removed htmlparser.jar): for Mac App
 2012-08-16 07:28:38 +02:00
Lotus
ae9cd7a118 fix xss bug #204 2012-08-15 14:23:21 +02:00
Michael Peter Christen
1641835fef replaced yacy xml encoding by solr xml encoding 2012-08-14 13:29:11 +02:00
Michael Peter Christen
89fe13e73d enhanced GSA and RSS output format: corrected date, added some missing 
fields, added xml encoding for utf8
 2012-08-14 13:19:29 +02:00
Michael Peter Christen
ea49a8aa8c Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-08-14 12:40:44 +02:00