Commit Graph

9316 Commits

Author SHA1 Message Date
Michael Peter Christen
98a4a4aa97 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-02-15 01:38:23 +01:00
Michael Peter Christen
b6de1f42dc Full redesign of solr connection architecture. This was done to support
multiple solr cores instead of just one. Therefore it is now necessary
to distuingish between solr server connections (called an 'Instance')
and a connection to a single solr core. One Instance may now have
multiple connector classes assigned to it, each connecting to a single
core.
To support multiple cores it is also necessary to distinguish between
the connection configuration and the configuration of the index schema.
We will have multiple schema configurations in the future, each for
every solr core. This caused that the IndexFederated servlet had to be
split into two parts, the new Servlet for the Schema editor is now in
the IndexSchema Servlet.
2013-02-15 01:38:10 +01:00
Marc Nause
efb6cf7d21 Merge branch 'master' of git@gitorious.org:yacy/rc1.git 2013-02-13 19:31:12 +01:00
Marc Nause
ce5b7afab2 *) removed Skype online indicator (was not working anymore)
*) updated ICQ URLs
2013-02-13 19:29:40 +01:00
Michael Peter Christen
4111606654 removed the commitWithin attribute because that is not the way how the
index is updated the right way for us. May also be be superfluous with
the solr 4.0 softcommit.
2013-02-13 02:29:47 +01:00
Michael Peter Christen
c20fa3640d fix to unbalanced tag and license for null objects 2013-02-13 01:23:05 +01:00
Michael Peter Christen
3a6097966d added jsonp option to yjson result writer 2013-02-13 01:11:57 +01:00
Michael Peter Christen
de58043205 Added image license generation for solr image search results when
results are generated within yjson result writer. This makes it possible
to view images in yacyinteractive from solr.
2013-02-13 00:33:53 +01:00
Michael Peter Christen
d3508fa8ff fixed json search, quotes, auto-facets, urls etc. for
yacyinteractive.html
2013-02-13 00:01:38 +01:00
Michael Peter Christen
1db23e9eac Moved methods from SolrServerConnector to AbstractSolrConnector with the
result that most of these methods become superfluous in other classes.
This is a generalization step towards multi-indexes in Solr.
2013-02-12 22:03:10 +01:00
Michael Peter Christen
02fa31b5bf better filesearch layout 2013-02-12 12:21:29 +01:00
Michael Peter Christen
e55ec3071d reduced number of facets in yacyinteractive (only filetype necessary) 2013-02-12 12:00:54 +01:00
Michael Peter Christen
16d90859b7 reverted put-semantics back to as-usual in serverObjects and introduced
an add-method to put in several objects for the same key
2013-02-12 11:52:33 +01:00
Michael Peter Christen
0d888ff69e Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-02-12 03:42:58 +01:00
Michael Peter Christen
c34af7fe94 extended JSON Response Writer and Opensearch Response Writer for the
Solr search interface in such way that it is possible to use this
interface for the yacyinteractive search. This search interface is now
much faster using the Solr search directly. For the Solr interface it
was necessary to create a translation from the YaCy search modifiers to
the Solr facet selection. This was added in such a way that it becomes
generic for the normal YaCy search and as a on-top evaluation for Solr
queries.
2013-02-12 03:42:46 +01:00
reger
c37d718f16 make sure yacy.running is deleted if not running (catch exception)
- to prevent following log if YaCy was previously not properly shutdown 

E ... STARTUP WARNING: the file C:\src\git\yacy-rc1\DATA\yacy.running exists, this usually means that a YaCy instance is still running
E ... STARTUP FATAL ERROR: java.util.concurrent.TimeoutException
java.util.concurrent.ExecutionException: java.util.concurrent.TimeoutException
	at net.yacy.cora.protocol.TimeoutRequest.call(TimeoutRequest.java:91)
	at net.yacy.cora.protocol.TimeoutRequest.ping(TimeoutRequest.java:112)
	at net.yacy.yacy.startup(yacy.java:200)
	at net.yacy.yacy.main(yacy.java:638)
Caused by: java.util.concurrent.TimeoutException

- adjust Netbeans path (to solr4.1.jars)
2013-02-11 22:53:19 +01:00
Michael Peter Christen
762b687e47 extended the serverObjects to be able to hold multipel values for a
single key. This is done using the solr class MultiMapSolrParams. That
class is needed in the OpensearchResultWriter to get multiple facet
requests.
2013-02-11 22:12:15 +01:00
Michael Peter Christen
d70d99fab5 added more metadata fields and facets to OpensearchResponseWriter.
This should make it possible to replace the original and enriched yacy
opensearch result with a solr output in opensearch format.
2013-02-11 22:10:14 +01:00
Michael Peter Christen
6a4878940b fix in html parser and bookmark generation 2013-02-11 13:28:08 +01:00
Michael Peter Christen
51e7ab4f70 moved bookmarks back to more prominent location (even if this does not
fit to the 'Search Interfaces' headline)
2013-02-09 06:57:20 +01:00
Michael Peter Christen
dee8b24d3c better error handling for bookmarks 2013-02-09 06:55:57 +01:00
Michael Peter Christen
e1da39245a when searching the network, do not search on robinson peers with the old
DHT search interface. Now use the solr interface.
2013-02-08 18:30:08 +01:00
Michael Peter Christen
6f6ddaf7e7 A robinson peer does not need to write RWI data if such peers are only
searched using the solr interface. Searching public rpbinsons will be
done with solr only in the future.
2013-02-08 17:58:54 +01:00
Michael Peter Christen
ab4f74c82c fix for xml blacklist import 2013-02-08 15:12:10 +01:00
Michael Peter Christen
7806680ab8 fixed a problem with re-feeding of already indexed documents whith
coordinates attached.
2013-02-08 12:45:54 +01:00
Michael Peter Christen
cb38e860cf After the observation that Windows user simply forget that they started
YaCy; YaCy is still running and the user additionally expect that
another doubleclick on the YaCy icon simply opens the search windows
(again) I decided to add a function that complies to the expectation to
the user: simply open the browser pop-up page again if the user starts
YaCy while YaCy is still running.
2013-02-07 23:39:00 +01:00
Marc Nause
27894d2c1a Merge branch 'master' of git@gitorious.org:yacy/rc1.git 2013-02-05 21:09:41 +01:00
Marc Nause
75f9568472 *) only install files from the RELEASE directory
*) minor changes
2013-02-05 21:02:32 +01:00
Michael Peter Christen
eb80405a16 added a disable function in RemoteCrawl_p servlet which prevents setting
of remote crawl if peer is not a senior or principal peer
2013-02-05 12:47:20 +01:00
Michael Peter Christen
1e3d8cc235 show a link for the host in the host browser; see 2013-02-04 21:24:57 +01:00
Michael Peter Christen
19c46e4acf catch more exceptions 2013-02-04 21:24:39 +01:00
Michael Peter Christen
7de502f43d Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-02-04 20:02:35 +01:00
Marc Nause
3bc5ee6e3d *) added protection against CSRF in update download page
(http://localhost:8090/ConfigUpdate_p.html?releaseinstall=../../test.txt&deleteRelease=Delete+Release
does not work anymore)
2013-02-04 19:57:28 +01:00
Michael Peter Christen
4f270d89e2 another NPE 2013-02-04 18:04:52 +01:00
Michael Peter Christen
921091c3a6 use thread-safe http connection manager for authenticated remote solr
connections
2013-02-04 17:48:04 +01:00
Michael Peter Christen
e8f7b85b98 fixes to internal RWI usage if RWI is switched off (NPE etc) 2013-02-04 17:11:02 +01:00
Michael Peter Christen
3834829b37 bugfixes and more logging for solr connector 2013-02-04 16:42:10 +01:00
Michael Peter Christen
bc00097cbf arrr... forgot the new library 2013-02-04 12:02:37 +01:00
Michael Peter Christen
09a2b09c48 guava update 2013-02-04 11:21:05 +01:00
Michael Peter Christen
80fe3d7860 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
Conflicts:
	source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java
2013-02-04 10:57:54 +01:00
Michael Peter Christen
4323621a76 update to Solr 4.1.0 2013-02-04 10:55:49 +01:00
sixcooler
639c114199 remove jetty from classpath - as it was moved last commit 2013-02-03 23:24:19 +01:00
reger
160ce568b3 move testing SolrServlet.main to test, making include of jetty*.jar in distribution and classpath obsolete
- move jetty*.jar to test library 
- move SolrServlet.main as is to test, add also a junit test simulating main 
  - add build.xml cleanup for EmbeddedSolrConnectorTest created test/DATA
- adjust some test compile errors
2013-02-03 22:32:38 +01:00
orbiter
07a20e8253 removed unused import 2013-02-02 10:52:39 +01:00
Michael Peter Christen
d1cb4cbc84 enhanced network scanner, is faster and more flexible now
- start more processes
- remove superfluous host name resolution
- better/more flexible subnet ip range calculation
- prefer ipv4 makes better usable ip pre-settings in servlet
- extended servlet by new subnet /20 - option
- redesign of scanner start process in servlet (generalization)
2013-02-02 09:51:43 +01:00
Michael Peter Christen
592adf7ccb fix for domain navigation 2013-02-02 07:21:18 +01:00
Michael Peter Christen
4ca1b76627 less search overhead when first result set is smaller than requested 2013-02-02 07:20:56 +01:00
Michael Peter Christen
f748b0aa7c NPE fix 2013-02-02 07:20:02 +01:00
Michael Peter Christen
7dfcc92b71 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-01-31 13:15:42 +01:00
Michael Peter Christen
0b6566a389 optimizations when starting large crawl requests with many start urls in
one request:
- allow larger match-fields in html interface
- delete all host hashes at once from zurl
- when deleting by host, do not count size of deleted entries since that
was the reason it took so long
2013-01-31 13:15:28 +01:00