orbiter
c40ba51ca6
added new suggest method which replaces more-than-one suggestions:
...
instead of computing suggest permutations of the given words, the
completion of a phrase using the given words is searched in the fulltext
index.
2014-02-03 12:44:52 +01:00
orbiter
416481c33e
added a boost on appearance of combined words (in the same order the
...
user submitted that) when searching for more than one word
2014-01-30 10:51:08 +01:00
orbiter
0b88137def
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
2014-01-29 22:01:40 +01:00
reger
c589ee8c6e
URLproxy access check too tight
...
respect config ip pattern (was own ip)
2014-01-28 22:39:45 +01:00
Michael Peter Christen
ebfaf753b7
- faster initialization of index files
...
- removal of not used space if index files shrink (rare, but possible)
2014-01-28 12:39:58 +01:00
orbiter
ba5ab11cc4
less logging
2014-01-27 21:54:52 +01:00
Michael Peter Christen
322854a5f8
fix auth for forced ping
2014-01-27 15:56:02 +01:00
Michael Peter Christen
fbf4f77d80
fixed missing corona in network picture
2014-01-27 15:43:08 +01:00
Michael Peter Christen
4b7f2fcf38
updated bootstrap seedlist list
2014-01-27 13:55:06 +01:00
Michael Peter Christen
d2b8f2b477
enhancements for staticIP and ipv6 handling
2014-01-27 13:48:20 +01:00
reger
a71718a459
add config value for ssl/https port (default=8443)
...
adjust server routines to use config
2014-01-27 01:09:56 +01:00
reger
91d79c1ac4
disable wrong forward to https on port change
2014-01-26 21:50:42 +01:00
reger
a3e2cca8e9
improve isOlder check to not overwrite node index with metadata on equal load date
2014-01-26 01:00:52 +01:00
reger
193b8235c2
remove double jquery-1.3.1.js and adjust header links to jquery-1.3.2
2014-01-26 00:58:54 +01:00
reger
9b24dae2b7
add language navigation filter clause to rwi results
2014-01-25 22:59:23 +01:00
reger
f307d65dcf
prepare for a language navigator
...
works fine to restrict language for local solrSearches.
More work needs to be done to make rwi/remote searches respect the modifier.language restriction.
2014-01-24 03:11:25 +01:00
reger
cf553e5045
added hint to web.xml and for completeness the full set of hardcoded mappings
2014-01-23 23:56:45 +01:00
orbiter
768b1306b8
Added a write-enabled checkbox for remote solr servers.
...
It is now possible to assign every peer other YaCy peers as remote solr
server which are only used for read operations during search. This also
affects crawling: it will exclude urls from crawls which exist on remote
solr/remote YaCy peers.
2014-01-23 22:48:31 +01:00
orbiter
f7d6dd136f
changed solr paths according to new default paths
2014-01-23 19:21:07 +01:00
Michael Peter Christen
c84bcc878a
first try to add a generic solr servlet as luke request servlet
2014-01-23 19:01:31 +01:00
Michael Peter Christen
a8fdaace31
changed the web.xml as well to migrate the solr servlet
2014-01-23 18:41:45 +01:00
Michael Peter Christen
4cb7e2a2ca
refactoring: renamed the SolrServlet to SolrSelectServlet for better
...
naming of more Solr Servlets
2014-01-23 17:20:49 +01:00
Michael Peter Christen
dc06e407ce
added two virtual instances of solr for the both cores: collection1 and
...
webgraph. These cores are now accessible at
/solr/collection1/select instead /solr/select?core=collection1
and
/solr/webgraph/select instead /solr/select?core=webgraph
in addition to the old behavior to support compatibility to the old
peers. These new paths are fully solr standard-conform and will allow
the cross-linking between YaCy peers using their public solr API.
2014-01-23 17:14:13 +01:00
Michael Peter Christen
8b14e92ba4
added button in host browser to re-load 404/failed documents
2014-01-23 15:56:36 +01:00
reger
f47067b0ce
fix search navigator not showing activated nav
...
introduced with 97e84439fb
2014-01-23 01:52:51 +01:00
orbiter
771d8261c1
Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
2014-01-22 21:53:27 +01:00
orbiter
c351e47a84
fix for bad-formatted lonlat
2014-01-22 21:33:11 +01:00
reger
4c603b216e
optimize parse ServerSideInclude
2014-01-22 21:23:32 +01:00
orbiter
5ec0c969c9
fix for http://bugs.yacy.net/view.php?id=354
2014-01-22 20:59:53 +01:00
orbiter
0002abd583
fix for OOM during remote search and too high load protection
2014-01-22 20:54:03 +01:00
sixcooler
5a917e13c6
use less ram on dht-URL transfer by not using a URIMetadataNode[]
2014-01-22 17:52:07 +01:00
Michael Peter Christen
c87cdfca2e
do not set a load prerequisite that prevents the start of one-time-jobs
2014-01-22 17:18:53 +01:00
sixcooler
0512e46c6a
bump to httpclient-4.3.2
2014-01-22 01:31:22 +01:00
sixcooler
4d77ca52c9
workaround to let dht-out run on smal Systems like a Pi
2014-01-22 01:26:44 +01:00
reger
9a96a7d73f
put list quick navigator buttons belowon BlackList_p editor
...
replacing the dropdown -> go navigation
2014-01-21 21:35:48 +01:00
Michael Peter Christen
6ada0daae9
making latency_factor and maximum number of same hosts in loader queue
...
settings available in Crawler_p.html servlet for steering.
2014-01-21 19:28:00 +01:00
Michael Peter Christen
489c3fbc90
code simplifications / removed warnings
2014-01-21 17:53:39 +01:00
Michael Peter Christen
0168f80c28
new crawling factors can now be changed during runtime
2014-01-21 17:52:16 +01:00
Michael Peter Christen
be5e808236
- removed hardcoded load-test which is now handled in BusyQueues
...
steering, see /PerformanceQueues_p.html
- changed default values for crawler queue load limit (high, because
these jobs are started upon user request)
2014-01-21 17:48:45 +01:00
sixcooler
40a4030b55
configurable max-load values for YaCy-Threads:
...
try lower values on smal systems like a Pi
2014-01-21 17:04:22 +01:00
sixcooler
6d8c023a5e
lower client-connection for single-cpu-systems
2014-01-21 16:56:44 +01:00
Michael Peter Christen
77531850b5
reverted crawling strategy from latest commit.
2014-01-21 16:05:55 +01:00
Michael Peter Christen
c0da966dfa
enhanced crawler speed
2014-01-20 21:46:40 +01:00
Michael Peter Christen
79809342fa
added synchronization to exists() call bacause the concurrent call to
...
that method showed in thread dump close to deadlock situations. Its also
better to synchronize IO operations because they become faster then.
2014-01-20 21:09:03 +01:00
Michael Peter Christen
9a6912f2e6
if a http client thread is still running but we do not wait for it any
...
more, call an interrupt
2014-01-20 18:39:36 +01:00
Michael Peter Christen
0d235a565b
cleanup crawl loader jobs
2014-01-20 18:36:00 +01:00
Michael Peter Christen
1ea17bd9f3
- removed old metadata database and all migration code
...
- refactored all code which uses URIMetadataRow as standard for word
hash length and word hash ordering and moved that to the class 'Word',
becuase the class URIMetadataRow defined the old metadata data structure
and should be superfluous in the future
- removed unused methods from URIMetadataRow as preparation for further
removal of that class
2014-01-20 18:31:46 +01:00
reger
d3de309953
fix IOexception logging issue in DefaultServlet
...
reason not sure but .logException triggers another exception
2014-01-20 08:12:35 +01:00
reger
97e84439fb
adjusted ConfigHeuristic and changed QueryGoal.getOriginalQueryString to .getQueryString
...
- since specific heuristic Twitter & Blekko is not longer available or redundant with OpenSearchHeuristic,
adjusted ConfigHeuristic to use OpensearchHeuristic settings only.
For this the default OSD search target list is made available (copied) by default and the other configs are removed.
- the return of QueryGoal.getOriginalQueryString includes the queryModifier, which are held separately in a modifier object,
but in most (all) cases just the query term is expected, clarified and renamed it to QueryGoal.getQueryString which returns
just the search term (if needed a .getOrigianlQueryString could be implemented in Queryparameters, adding the modifiers)
- started to adjust internal html href references from absolute to relative (currently it is mixed).
For future development we should prefer relative href targets (less trouble with context aware servlets)
2014-01-20 00:58:17 +01:00
reger
d24a0ec32c
upd heuristic default list (heuristicopensearch.conf)
...
- Faroo Web taken out (requires api key) http://www.faroo.com/hp/api/api.html#description
- update Faroo News to new url
- Twitter taken out (change to Api 1.1 not supporting rss) https://dev.twitter.com/discussions/24239
2014-01-20 00:03:55 +01:00