Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
9,745 Commits 1 Branch 20 Tags 836 MiB
b58ca8622d
Commit Graph

4411 Commits

Author SHA1 Message Date
Roland Haeder
b58ca8622d Some cleanups: 
- added SKINS_PATH_DEFAULT as same as LISTS_PATH_DEFAULT was added
- Added 'final' keyword to a string
 2013-07-27 10:13:57 +02:00
Roland Haeder
e2ee412160 Use SwitchboardConstants.LISTS_PATH_DEFAULT instead of 'DATA/LISTS' 
Conflicts:
	htroot/api/blacklists_p.java
 2013-07-27 10:12:58 +02:00
Roland Haeder
ae19401af0 Removed another duplicate occurance of Blacklist.BLACKLIST_FILENAME_FILTER 2013-07-27 09:59:09 +02:00
Roland Haeder
59225487ea Fix for blacklist export, also applied the filename filter here 2013-07-27 09:58:56 +02:00
Roland Haeder
952fc0e7bd Removed superfluous check for files ending '.black' as the previous commit already excluded all other files (e.g. .ser dumps), added logging in catch-all block 2013-07-27 09:58:38 +02:00
Roland Haeder
060fec1577 Reuse Blacklist.BLACKLIST_FILENAME_FILTER 2013-07-27 09:57:50 +02:00
Roland Haeder
29049c71f5 Possible fix for ticket http://bugs.yacy.net/view.php?id=270, the filter for only including *.black must be applied 2013-07-27 09:57:07 +02:00
orbiter
9c681cc00d added segment sizes, postprocessing status and cpu load to crawler 
monitor
 2013-07-23 19:10:11 +02:00
orbiter
86b514cf46 added load info to status_p.xml 2013-07-23 18:20:07 +02:00
orbiter
056b42f5aa - added information about segment count to status_p.xml 
- also moved this information from the old index structure, which is
still in use for the RWI/DHT index to that front-end
 2013-07-23 18:03:33 +02:00
orbiter
6fb2811e68 fixes for problems with remote solr and non-activated webgraph index 2013-07-23 16:46:44 +02:00
orbiter
e24016e30a added the property federated.service.solr.indexing.timeout to yacy.init 
to provide a configurable time-out for solr; see also:
http://bugs.yacy.net/view.php?id=254
 2013-07-22 17:45:12 +02:00
orbiter
232100301c removed double-ocurring value assignments 2013-07-17 19:09:25 +02:00
Roland Haeder
aaedc0405d Fixes and avoid of catching bad exceptions (some): 
- Rewrote usage of HashMap/Map to concurrent versions (to avoid a
CME=ConcurrentModificationException)
- Rewrote ConnectionInfo (as an example) to use a synchronized iterator
instead of synchronizing an
  already synced HashSet (see Collections call)
- This avoids catching CMEs again
- Commented out noisy ConcurrentLog.logException() call

Conflicts:
	source/net/yacy/repository/LoaderDispatcher.java
 2013-07-17 18:37:34 +02:00
Roland Haeder
841a28ae76 Added 'final' for all exception blocks as this helps the Java compiler 
to optimize memory usage

Conflicts:
	source/net/yacy/search/Switchboard.java
 2013-07-17 18:31:30 +02:00
Felix Ableitner
376f9cd9d0 Merge branch 'master' of git://gitorious.org/yacy/rc1 into blacklist_structure 2013-07-17 15:58:09 +02:00
Michael Peter Christen
89c0aa0e74 added collection_sxt to error documents 2013-07-17 15:20:56 +02:00
Michael Peter Christen
0df5195cb0 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-07-17 12:42:06 +02:00
Michael Peter Christen
1fd006cc56 fixes using the embedded connector 2013-07-17 12:41:54 +02:00
orbiter
aba7cc5de7 added cpu load information to status page 2013-07-17 12:38:12 +02:00
Roland Haeder
59b4fdd5ad Merge remote-tracking branch 'upstream/master' 2013-07-13 15:12:51 +02:00
orbiter
5493389576 stealth mode shall only be available for authorized users, because 
unauthorized users can otherwise be monitored by authorized users
 2013-07-13 14:49:36 +02:00
Roland Haeder
ebbb3bc5c1 Fixed CHMOD on many files + added missing loggers (e.g. jena) and made some noisy loggers quiet 2013-07-13 13:12:36 +02:00
Michael Peter Christen
bcc623a843 refactoring of load_delay: this is a matter of client identification 2013-07-12 16:24:56 +02:00
orbiter
2be456e7fb added a postprocessing field into api/status_p.xml to show if the 
postprocessing task is running at that time (status: busy) or not
(status:idle)
 2013-07-12 14:29:22 +02:00
orbiter
575f913154 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-07-12 14:17:13 +02:00
orbiter
c4efb612e2 added list of crawls to status_p.xml 2013-07-12 14:16:51 +02:00
Lotus
bb6caa346c Do not allow automatic update in case YaCy is installed to the Program 
Files folder on Windows. There are no permissions to write that folder
and update would fail.
 2013-07-11 21:50:06 +02:00
orbiter
dac88561ae minimum access time has a tight connection to ClientIdentification, 
therefore it is defined there.
 2013-07-11 17:04:24 +02:00
Felix Ableitner
a020697d64 Fixed problems with blacklist entry insertion. 2013-07-11 13:10:23 +02:00
sixcooler
bff8c753c6 re-insert this file - was deleted by mistake 
+ correct an other case-typo
 2013-07-10 18:32:12 +02:00
Michael Peter Christen
5878c1d599 - refactoring of log to ConcurrentLog: 
jdk-based logger tend to block
at java.util.logging.Logger.log(Logger.java:476) in concurrent
environments. This makes logging a main performance issue. To overcome
this problem, this is a add-on to jdk logging to put log entries on a
concurrent message queue and log the messages one by one using a
separate process.
- FTPClient uses the concurrent logging instead of the log4j logger
 2013-07-09 14:28:25 +02:00
orbiter
c79f687110 enhanced the network scanner: find more hosts automatically by removal 
of common subdomains before application of protocol-specific prefix
 2013-07-09 11:42:13 +02:00
orbiter
b4677d1cad fix for bug #252 
the naming of the servlet was wrong, the bug may not be present on
systems where upper/lowercase matching is lazy (windows)
 2013-07-09 10:50:47 +02:00
Michael Peter Christen
07261fe274 Merge remote-tracking branch 'nutomics/blacklist_structure' 2013-07-08 23:32:15 +02:00
Michael Peter Christen
dea71851d2 - better concurrency for network scanner 
- network scanner can now start from the list of all hosts in the search
index
 2013-07-08 16:29:30 +02:00
orbiter
9f0cc9b401 enhanced network scanner 
- textarea input field can now be used to paste in a large list of hosts
- /31er subnet is possible (only one host)
- auto-detect subdomains for ftp and www subdomains
 2013-07-08 13:17:09 +02:00
orbiter
f8c28efd66 fix for rssTerminal coloring 2013-07-04 21:46:46 +02:00
Felix Ableitner
44f8fcf62e Changed class structure of Blacklist. 2013-07-04 18:37:57 +02:00
Michael Peter Christen
3054a6d4b9 added a patch from Sebastian M.B., submitted by email for coloring of 
rss terminal
 2013-07-04 17:12:19 +02:00
Michael Peter Christen
78af998f8f Merge commit 'fd90fcc4e08f80acbfd1c9a7ec62ce04cd309594' 2013-07-04 16:56:54 +02:00
Michael Peter Christen
57ffdfad4c added a crawl option to obey html-meta-robots-noindex. This is on by 
default.
 2013-07-03 14:50:06 +02:00
Felix Ableitner
fd90fcc4e0 Fixes #196. 2013-07-02 20:45:41 +02:00
Michael Peter Christen
f1c5338210 prepartion for greedy crawl profiles and refactoring 2013-07-01 13:10:09 +02:00
Michael Peter Christen
e6f361f474 adding the canonical tag to crawl queues 2013-07-01 13:09:41 +02:00
Michael Peter Christen
203921006a redesign of citation index storage 2013-06-30 02:11:46 +02:00
Michael Peter Christen
e92b9275ce Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-06-28 15:33:29 +02:00
Michael Peter Christen
56cdcfa2fa fixed greedy learning mode - global is not a search attribute in 
searchitems
 2013-06-28 15:33:19 +02:00
Michael Peter Christen
32aa1d4569 removed unused option for queries 2013-06-28 15:32:36 +02:00
Michael Peter Christen
0c5bed7e2c added configuration option for greedy learning function to ConfigPortal 
servlet
 2013-06-28 15:31:36 +02:00