Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
9,736 Commits 1 Branch 20 Tags 836 MiB
4c242f9af9
Commit Graph

6428 Commits

Author SHA1 Message Date
Michael Peter Christen
4c242f9af9 always use a default value for boolean options to have transparency for 
the outcome if the attribute is missing in servlets
 2013-07-25 12:17:29 +02:00
Michael Peter Christen
61e015268b fix in forced deletion: forced commit needed 2013-07-25 09:53:19 +02:00
Michael Peter Christen
83e2921b39 new test case for http://bugs.yacy.net/view.php?id=141 2013-07-25 09:31:48 +02:00
Michael Peter Christen
304aacb2cc fix for http://bugs.yacy.net/view.php?id=267 2013-07-25 09:26:24 +02:00
Michael Peter Christen
c3b2301b2f fix for http://bugs.yacy.net/view.php?id=268 2013-07-25 09:21:37 +02:00
reger
aa1a1f1d2c - small adjustment to make sure genericParser is tried last 
-- for some documents genericParser grabs document instead of specific available parser due to unordered pick of 1st to try parser
      (like .ps .rdf files and other)
- remove redundant file extension registration
 2013-07-23 20:24:13 +02:00
orbiter
3e901dcb06 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-07-23 19:33:07 +02:00
orbiter
f50b596e0b do not run dht ditribution if system load is over 2.5 2013-07-23 19:32:32 +02:00
orbiter
056b42f5aa - added information about segment count to status_p.xml 
- also moved this information from the old index structure, which is
still in use for the RWI/DHT index to that front-end
 2013-07-23 18:03:33 +02:00
orbiter
6fb2811e68 fixes for problems with remote solr and non-activated webgraph index 2013-07-23 16:46:44 +02:00
sixcooler
af740f3058 changed optimization to a segment-size of index-size/5.000.000 
+ one if not idle
+ one (and force) if postprocessing
 2013-07-23 14:21:12 +02:00
Michael Peter Christen
336f86394c replaced StringBuffer with StringBuilder 2013-07-23 12:21:27 +02:00
Michael Peter Christen
aeac2fb763 replaced more containsKey() -> get() usages by a simple get(), followed 
by a test for NULL. This should increase the application speed and
reduces the lookup time for the affected methods by 50%
 2013-07-23 12:16:51 +02:00
orbiter
5364c4dcc9 delayed first peer-ping to send the first ping out after the http got 
up; if the ping comes before the http is up, it cannot be recognized as
senior peer (if at all). See also: http://bugs.yacy.net/view.php?id=266
 2013-07-22 18:21:37 +02:00
orbiter
e24016e30a added the property federated.service.solr.indexing.timeout to yacy.init 
to provide a configurable time-out for solr; see also:
http://bugs.yacy.net/view.php?id=254
 2013-07-22 17:45:12 +02:00
orbiter
c124037f19 removed forced non-soft commits to prevent index fragmentation 2013-07-22 17:28:20 +02:00
Michael Peter Christen
31483c47e1 fixed problem with remote luke requests 2013-07-22 15:55:20 +02:00
Michael Peter Christen
c15aa758dc removed failreason_t removal patch because that causes too much 
confusion using an external solr. to clean up the index after a schema
change, use the index cleaner function from the online servlet
 2013-07-22 14:17:38 +02:00
reger
2b7a38640a extend content type detection on file extension for .tif .tiff .htm 2013-07-21 22:57:21 +02:00
Michael Peter Christen
ac1aad5064 added a getSegmentCount method and use it to disable optimize if wanted 
current segment count is below optimization level
 2013-07-18 14:31:42 +02:00
Michael Peter Christen
36035e0a0a - used reger's LukeRequest to generalize the index info in 
SolrServerConnector
- used the LukeRequest in SolrServerConnector to replace the index size
method by a getNumDocs request to a LukeRequest result
 2013-07-18 13:26:07 +02:00
Michael Peter Christen
39fceb5ccf fix for NPE & bug #264 2013-07-18 12:37:32 +02:00
Michael Peter Christen
735a66eff3 enhancements to crawler 2013-07-18 12:29:04 +02:00
Roland Haeder
be0ff6018f Removed trailing spaces + some more final 2013-07-17 18:44:24 +02:00
Roland Haeder
aaedc0405d Fixes and avoid of catching bad exceptions (some): 
- Rewrote usage of HashMap/Map to concurrent versions (to avoid a
CME=ConcurrentModificationException)
- Rewrote ConnectionInfo (as an example) to use a synchronized iterator
instead of synchronizing an
  already synced HashSet (see Collections call)
- This avoids catching CMEs again
- Commented out noisy ConcurrentLog.logException() call

Conflicts:
	source/net/yacy/repository/LoaderDispatcher.java
 2013-07-17 18:37:34 +02:00
Roland Haeder
841a28ae76 Added 'final' for all exception blocks as this helps the Java compiler 
to optimize memory usage

Conflicts:
	source/net/yacy/search/Switchboard.java
 2013-07-17 18:31:30 +02:00
Felix Ableitner
03044589dd Fixed (?i) appearing in entries, fixed multiple equal lines in file. 2013-07-17 16:42:10 +02:00
Michael Peter Christen
89c0aa0e74 added collection_sxt to error documents 2013-07-17 15:20:56 +02:00
Michael Peter Christen
0df5195cb0 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-07-17 12:42:06 +02:00
Michael Peter Christen
1fd006cc56 fixes using the embedded connector 2013-07-17 12:41:54 +02:00
orbiter
d0dc86cf3d logging of deadlocks (if any) during cleanup process 2013-07-17 12:38:58 +02:00
Michael Peter Christen
c6a6f159e8 fix for crawl stack domain counter 2013-07-16 18:18:55 +02:00
Michael Peter Christen
93d1bac140 do a more frequent optimization, reduces IO after optimization 2013-07-16 17:16:48 +02:00
orbiter
b71d13a014 added load and deadlock detector in Memory util 2013-07-16 10:49:20 +02:00
orbiter
290e24564b Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-07-14 17:41:32 +02:00
orbiter
5533fc8e01 fix for bug 260 2013-07-14 17:40:28 +02:00
Michael Peter Christen
b79471ee67 grr 2013-07-14 10:15:47 +02:00
Michael Peter Christen
a79f288ac1 automatically running optimize on solr if user/search is idle for some 
time
 2013-07-14 10:02:08 +02:00
orbiter
a9c8046c87 do a light optimization at the end of a crawl postprocessing 2013-07-13 19:09:46 +02:00
orbiter
a548354c71 replaced type of solr schema object sku of text_en_splitting_tight by 
string
 2013-07-13 18:54:09 +02:00
orbiter
2f1ec8d4a2 npe fix 2013-07-13 11:10:05 +02:00
Michael Peter Christen
bcc623a843 refactoring of load_delay: this is a matter of client identification 2013-07-12 16:24:56 +02:00
orbiter
0d0b3a30f5 activate api actions after postprocessing of crawls 2013-07-12 16:05:48 +02:00
orbiter
3978c5ca5d fix for http://bugs.yacy.net/view.php?id=255 2013-07-12 14:38:30 +02:00
orbiter
2be456e7fb added a postprocessing field into api/status_p.xml to show if the 
postprocessing task is running at that time (status: busy) or not
(status:idle)
 2013-07-12 14:29:22 +02:00
orbiter
dac88561ae minimum access time has a tight connection to ClientIdentification, 
therefore it is defined there.
 2013-07-11 17:04:24 +02:00
Michael Peter Christen
9a29ab469e another patch to prevent CLOSE_WAIT status on solr connections 2013-07-11 12:53:39 +02:00
Michael Peter Christen
5091d627bc fixed parsing of peer flags 2013-07-11 12:53:16 +02:00
Michael Peter Christen
87e9052081 added Connection:close to all http requests in our http client to 
prevent CLOSE_WAIT states (as seen in lsof)
 2013-07-11 11:54:11 +02:00
Michael Peter Christen
5c6946dd5f replaced usage of log4j by ConcurrentLog where possible 2013-07-09 14:42:39 +02:00