Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
8,591 Commits 1 Branch 20 Tags 836 MiB
613b45f604
Commit Graph

8591 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Michael Peter Christen
613b45f604 - better data structures in secondary search 
- fixed a big memory leak in secondary search
 2012-07-03 07:12:20 +02:00
Michael Peter Christen
de903a53a0 parser refactoring & hacks 2012-07-03 06:06:38 +02:00
Michael Peter Christen
8a82609360 - smaller caches to save memory 
- close cloneable iterators to free memory
 2012-07-02 15:40:40 +02:00
Michael Peter Christen
7249d9c9de bugfix for concurrent seed loader 2012-07-02 14:37:57 +02:00
Michael Peter Christen
c72d3b12cd concurrently initialize the seed list during p2p network bootstrap 2012-07-02 14:27:37 +02:00
Michael Peter Christen
1825f165b8 better integration of blacklist according to use case 2012-07-02 13:57:29 +02:00
Michael Peter Christen
c18fa9fa75 Merge branch 'master' of git://gitorious.org/~reger/yacy/bbyacy-rc1 2012-07-02 12:20:57 +02:00
Michael Peter Christen
ce8d4b87d9 fixes for new eclipse 'Juno' warning 'Resource leak'. 2012-07-02 10:27:46 +02:00
Michael Peter Christen
0c345d1559 giving threads name so its easier to see whats happening during 
debugging and within a thread dump
 2012-07-02 09:51:43 +02:00
reger
067728bccc add search result heuristic. adding a crawl job with depth-1 for every displayed search result (crawling every external linked page of displayed search result pages) 2012-07-01 00:12:20 +02:00
Michael Peter Christen
03280fb161 removed segments-concept and the Segments class: 
the segments had been there to create a tenant-infrastructure but were
never be used since that was all much too complex. There will be a
replacement using a solr navigation using a segment field in the search
index.
 2012-06-28 14:27:29 +02:00
Michael Peter Christen
508a81b86c added solr field 'refresh_s' which stores the refresh url contained in 
the meta-refresh html header field.
 2012-06-28 13:27:45 +02:00
Michael Peter Christen
f3167def64 do not fill the keywords with title content if keywords do not exist. 2012-06-27 13:07:02 +02:00
Michael Peter Christen
9116013c64 - allow lazy initialization of solr value (if using 'lazy', then no 
0-values and no empty strings are written). This may save a lot of
memory (in ram and on disc) if excessive 0-values or empty strings
appear)
- do not allow default boolean values for checkboxes because that does
not make sense: browsers may omit the checkbox attribute name if the box
is not checked. A default value 'true' would not comply with the
semantic of the browsers response.
- add a checkbox in IndexFederated_p for the lazy initialization of solr
fields.
 2012-06-27 12:17:58 +02:00
sixcooler
97f60010d8 fix crawl start from file 2012-06-26 16:11:39 +02:00
Michael Peter Christen
c03d306afa shorter autocommit time (now: 1 second) to prevent that user cannot see 
results in solr the first time they try it out. The value can now be
easily set to a higher number using the IndexFederated_p interface.
 2012-06-26 14:53:45 +02:00
Michael Peter Christen
0294a53459 - add canonical field only if requested by solr schema 
- remove canonical url from in/outbound urls if present
 2012-06-26 14:51:57 +02:00
Michael Peter Christen
3fd4a01286 added option to record urls that are forwarded to the solr index 2012-06-26 13:54:48 +02:00
Michael Peter Christen
d763e4d94b fixed bad referer computation in SSIs which causes a NPE during host 
computation. This error was there before the latest IPv6 hack but did
not cause a NPE. The IPv6 hack was not the cause for this bug, but it
discovered the misconfiguration of the 'referer' referrer.
 2012-06-26 11:18:29 +02:00
Michael Peter Christen
358b04885e more IPv6 hacks 2012-06-26 00:25:46 +02:00
Michael Peter Christen
96aeb127e3 generalized localhost naming. 
this is also a preparation for a better IPv6 implementation.
 2012-06-26 00:08:25 +02:00
Michael Peter Christen
77f795756c fixing redirects and status codes: storing of status code in 
ResponseHeader to make it available for late evaluations, like storage
in solr.
 2012-06-25 18:17:31 +02:00
Michael Peter Christen
8dd469b9dd added option to configure the autocommit delay time of solr on-the-fly 2012-06-25 14:59:46 +02:00
Michael Peter Christen
5d9bd4ddc2 Merge remote-tracking branch 'origin/master' 2012-06-25 11:37:32 +02:00
Michael Peter Christen
b9dfca4b0a - fixed IndexFederated Servlet / a embedded Solr can now be selected 
- added code stub for an embedded Solr but generation of Solr store is
still commented out (it works but is not yet ready for usage)
 2012-06-25 11:34:38 +02:00
reger
2931726386 adjusted NetBeans classpath for new and updated libraries in lib 2012-06-24 22:50:08 +02:00
Michael Peter Christen
cc1b6762bb root, not yacy 2012-06-24 10:58:09 +02:00
Michael Peter Christen
2589158f44 changed recommended line in /etc/crontab for high-availability 2012-06-24 10:57:18 +02:00
reger
4156d4e12b Merge branch 'master' of git://gitorious.org/yacy/rc1.git 2012-06-23 21:22:46 +02:00
Michael Peter Christen
7b53be141f upgraded to pdfbox 1.7.0 
changes in http://www.apache.org/dist/pdfbox/1.7.0/RELEASE-NOTES.txt
with many bugfixes, including performance related
 2012-06-22 16:49:58 +02:00
Michael Peter Christen
fad3b14813 added jetty libraries, needed for future use as web server and as 
application server for the solr search interface
 2012-06-22 15:31:17 +02:00
Michael Peter Christen
a38b0a2c46 extended embedded solr tests to ensure that it will be usable within a 
jetty instance
 2012-06-22 11:40:02 +02:00
Michael Peter Christen
b9d42fd9c8 using com.google.common.io.Files instead of homebrew methods 2012-06-22 11:39:17 +02:00
Michael Peter Christen
a5eb91fa60 refactoring 2012-06-22 00:49:32 +02:00
Michael Peter Christen
1be0025a9c - added test for EmbeddedSolrConnector 
- added needed libraries for this test
this includes most (all) files needed for an embedded solr
 2012-06-22 00:36:49 +02:00
Michael Peter Christen
dbdd697f4d moved RDFaParser.xsl configuration file to defaults 2012-06-21 16:09:12 +02:00
Michael Peter Christen
90b82ce994 using guava for host resolution (non-blocking for ips) and time-out 2012-06-21 16:04:48 +02:00
Michael Peter Christen
f094936b89 added new class libraries to mac app 2012-06-21 14:59:55 +02:00
Michael Peter Christen
e12bb254b4 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-06-21 14:55:50 +02:00
Michael Peter Christen
3f55dc7c1e - added solr core and libraries that solr needs (lucene is missing, will 
follow later)
- added embedded solr connector which can connect to solr
programmatically (without using a server in between)
 2012-06-21 14:55:38 +02:00
reger
2fccc4e883 Merge branch 'master' of git://gitorious.org/yacy/rc1.git 2012-06-21 01:01:12 +02:00
Michael Peter Christen
c337190a00 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-06-20 20:18:10 +02:00
cominch
c63c3a4495 Show additional interaction elements in footer section on each page, if 
activated in ConfigPortal.html.
This footer is also visible in augmented browsing proxy mode.
 2012-06-20 18:04:23 +02:00
Michael Peter Christen
786be7d175 better integration of RDFaParser 2012-06-20 16:39:04 +02:00
cominch
fa98657bb3 Augmented Browsing: changed the settings page 2012-06-20 09:10:39 +02:00
cominch
751eeade0d Merge remote-tracking branch 'original yacy/master' 2012-06-20 07:58:27 +02:00
cominch
84a11ec48c Corrected loading of default page settings on ConfigPortal.html 2012-06-20 07:55:28 +02:00
sixcooler
bea002dc15 correct table in new look of Crawler_p 2012-06-19 13:13:00 +02:00
Michael Peter Christen
15f4551d88 Release 1.03 2012-06-19 08:51:26 +02:00
Michael Peter Christen
8738336408 set Xms lower than Xmx 2012-06-19 08:45:49 +02:00