Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
9,573 Commits 1 Branch 20 Tags 836 MiB
a5e328d7c5
Commit Graph

9573 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
reger
2962f2b9e9 Merge branch 'master' of git://gitorious.org/yacy/rc1.git 2013-03-12 02:51:17 +01:00
orbiter
ab74d559fb Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-03-11 18:23:43 +01:00
Michael Peter Christen
4490133909 removed target_tag_s (superfluous) 2013-03-11 10:46:29 +01:00
orbiter
cd197bb555 fix for NPE if surrogates do not exist 2013-03-10 19:46:06 +01:00
reger
6ae30f9d0f replace the terminateOldSessions - return immediate time from fixed 3 sec to requested minage parameter 2013-03-10 05:22:18 +01:00
Michael Peter Christen
68e739a90b Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-03-10 02:29:38 +01:00
Michael Peter Christen
3d9ce9cd04 - added more selection criteria for network seed list 
- enhanced up script
 2013-03-10 02:26:24 +01:00
orbiter
168e8d9b4d added/fixed missing DOCTYPE line (submitted by Thomas) 2013-03-08 14:40:09 +01:00
Michael Peter Christen
252bb51f98 fix for wrong mime type in noload crawler 2013-03-07 15:31:00 +01:00
Michael Peter Christen
25300913fa fixes to search debugging after testing with the different search 
debugging options
 2013-03-05 21:28:22 +01:00
Michael Peter Christen
81380ae5c8 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-03-05 12:24:10 +01:00
Michael Peter Christen
c2fde018b5 concurrent snippet fetching from solr results which do not have snippets 2013-03-05 12:24:01 +01:00
orbiter
b1140e3d82 added debug switches for detailed search testing 2013-03-05 12:19:32 +01:00
orbiter
cdbfddf091 added filter queries for better image, audio and video results 2013-03-04 21:18:54 +01:00
Michael Peter Christen
587ef83eab added missing cleanup statements for short memory cases during search 2013-03-04 13:01:24 +01:00
orbiter
2562f052b9 do not put the fulltext field text_t into the search cache because it is 
not used there and uses a lot of memory
 2013-03-04 12:01:10 +01:00
Michael Peter Christen
2b6c79d347 in method exists() also use the new caching-stacks for 
documents/metadata
 2013-03-04 01:13:17 +01:00
Michael Peter Christen
ae734b3f8d enhanced the search result processing 
- no waiting time at the end
- switched on 'classic' snippet production and verification (again)
 2013-03-04 00:17:29 +01:00
Michael Peter Christen
2d472a39f4 DHT-transferred metadata and crawl receipts now also use the delayed 
search cache to prevent that too much IO load is on the peer during
search.
 2013-03-04 00:07:52 +01:00
Michael Peter Christen
0d7b4bc891 better protection against OOM during search flush and fixed missing 
result push
 2013-03-03 23:45:47 +01:00
Michael Peter Christen
221ed7d764 - enhanced concurrency during search without IO blocking 
- introduced a second queue to flush remote search results (now: old
metadata structure from DHT peers)
- fixed result counters
 2013-03-03 22:38:50 +01:00
Marc Nause
2714b59f38 *) For some reason this seems to fix a ClassCastException on my system 
(OpenJDK).
 2013-03-03 20:38:20 +01:00
Michael Peter Christen
3b1d9dc884 made index storage from DHT search result concurrently. This prevents 
blocking by high CPU usage during search. Also: removed query from Solr
for DHT search results; results are taken from the pending queue.
 2013-03-02 10:25:52 +01:00
orbiter
f13c0b2abd fix for search 2013-03-01 19:18:16 +01:00
orbiter
0f7ea7ad9f - enhanced solr.add procedure for mass adds 
- removed unused solr access classes
- made snippet generation for documents aus YaCy RWI/DHT concurrent (as
it was before the search process removation)
- reduced the number of remote results in settings file because the
processing of such mass documents add is too CPU-intensive (in Solr)
 2013-03-01 15:27:17 +01:00
orbiter
7ff10bdb1b fix of page navigation for formatted totalcount numbers 2013-03-01 00:48:28 +01:00
orbiter
08d28eed1a Übersetzung des Domain Navigators als Anbieter Navigator; ist als Nutzen 
besser erklärbar
 2013-02-28 23:55:46 +01:00
Michael Peter Christen
f327ffedb4 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-02-28 15:55:13 +01:00
orbiter
9c09fd7d0b better/less requests to local solr; the request is made in chunks which 
are exactly at only that size which is needed to present the current
search result page. This will also cause that next solr request are made
automatically during switching to next pages.
 2013-02-28 14:04:08 +01:00
Michael Peter Christen
840fa22135 disabled clickdepth computation during craling since that is repeated 
during clean-up phase.
 2013-02-28 02:25:39 +01:00
orbiter
a734fbc4a5 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-02-27 22:44:57 +01:00
orbiter
d74472f562 corrected result counter 2013-02-27 22:40:23 +01:00
orbiter
2555542f7a removed the dns prefetch because that was not soo useful 2013-02-27 20:58:34 +01:00
orbiter
aa3c26c62e added recrawl/reload to CrawlStartSite for a timeout of 3 days 2013-02-27 11:43:36 +01:00
orbiter
c1b7e61882 added option to create empty vocabularies 2013-02-27 08:24:37 +01:00
bubu
e0edad689d fix link to IndexSchema_p.html 2013-02-26 21:12:44 +01:00
Michael Peter Christen
d957739441 removed size request 2013-02-26 17:53:44 +01:00
Michael Peter Christen
c95a84103a complete redesign of search process: 
- removed 'worker' processes
- no internal time-out behaviour: methods either are successful or
return null
- waiting is only done on top-level
- removed snippet-production; this is replaced by solr snippets
- removed statistics based on solr size queries (they had been VERY
long); the statistics (like suggestions or tag cloud) are now again
based on the old but very fast RWI index. In portal or intranet mode the
RWI index is usually switched off; if you like to have statistics again
then you must switch on the rwis again in this mode.
- fixed many bugs regarding correct page counter
 2013-02-26 17:16:31 +01:00
Michael Peter Christen
35fa718b77 testing to use solr for portalsearch caused some bugfixing but no full 
success: try to comment out the solr search request in
yacy-portalsearch.js
 2013-02-25 14:31:50 +01:00
Michael Peter Christen
008288719c fix for schema export to consider also automatically generated 
coordinate fields
 2013-02-25 01:13:03 +01:00
Michael Peter Christen
089dee1770 - generalized SchemaConfiguration into super-class Configuration and 
adopted other classes which used the configuration-only access for that
class
- removed many warnings
- adjusted logging
 2013-02-25 00:09:41 +01:00
Michael Peter Christen
c16de49f64 fix for webgraph delete query 2013-02-24 18:17:58 +01:00
Michael Peter Christen
56d5946a59 - added flags in IndexFederated_p.html to switch on or off the webgraph 
index (new solr core webgraph) .. this is now off by default
- completely redesigned this servlet
- added description how to attach a remote solr
- adjusted naming of servlet and menues
- moved 'lazy initialization' attribut from IndexSchema to
IndexFederated (this is a general option) back again.
 2013-02-24 18:09:34 +01:00
Michael Peter Christen
461d46101d - Removed log4j from libraries. This can be removed because the package 
log4j-over-slf4j is there. From slf4j all loggings are routed to the jdk
logger. Now all loggings are consistently done to the jdk logger.
- added some lines to the logging properties to suppress many solr
logging statements. The number of the logging entries had already become
a performance issue, therefore removing these from the log should
increase performance.
 2013-02-23 16:45:05 +01:00
Michael Peter Christen
b349c8145b Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-02-23 15:55:21 +01:00
orbiter
253a7aee88 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-02-23 14:33:29 +01:00
orbiter
36f9b0fc16 updated wstx-asl to 3.2.9 2013-02-23 14:33:17 +01:00
Michael Peter Christen
14cceb6b17 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 
Conflicts:
	htroot/IndexFederated_p.html
	source/net/yacy/cora/federate/solr/YaCySchema.java
	source/net/yacy/peers/Protocol.java
	source/net/yacy/search/Switchboard.java
	source/net/yacy/search/index/Segment.java

also moved portalsearch-dev to yacy-portalsearch to be able to fix
problems with new attachment to solr of the search widget
 2013-02-23 08:48:33 +01:00
Michael Peter Christen
58e1e6fa2b fixes to schema 2013-02-23 08:14:10 +01:00
reger
f291d60c5f on remote Solr search take only locally enabled schema fields from remote solrdocument for the inputdocument added to local index 2013-02-22 22:17:45 +01:00