Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
10,608 Commits 1 Branch 20 Tags 836 MiB
b9056ef2db
Commit Graph

6988 Commits

Author SHA1 Message Date
reger
b9056ef2db remove unused private header entries (HeaderFramework) 
X_YACY_ORIGINAL_REQUEST_LINE
X_YACY_KEEP_ALIVE_REQUEST_COUNT
CONNECTION_PROP_REQUESTLINE
 2014-03-26 23:28:19 +01:00
sixcooler
6d16fa993d make transparent proxy handle https-connections: 
the implemented handle for connect did not work for me - so lets try the
connectHandler
 2014-03-26 20:01:15 +01:00
Michael Peter Christen
61ad194065 fix for source and target clickdepth in webgraph index 2014-03-26 16:00:05 +01:00
Marc Nause
809b4e1fd9 Team added support for URLs with unicode characters in host part to 
blacklist. Punycode is used to handle unicode characters.
 2014-03-25 22:14:54 +01:00
reger
b126b9ba17 add some InputFileStream close at end of reads 
to make sure file is released
 2014-03-24 02:32:17 +01:00
reger
ca7444dbdf limit filetype nav to known extension also on image/media search 
- on text search we limit filetype nav already to known extension, apply filter to image search
 2014-03-23 23:10:29 +01:00
reger
651d057e93 surrogate import translate dc:language 3-char codes 
OAI records often use 3-char language codes, start converting some 3-char lang's to the internal ISO639-1 2-char code
 2014-03-23 00:40:36 +01:00
orbiter
22618e3ba2 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2014-03-21 20:28:50 +01:00
orbiter
01989f6af9 restrict write buffer size to a limit 2014-03-21 20:28:34 +01:00
Michael Peter Christen
d1091e79f8 - added stealth button to navigation menu 
- more fixes to progress bar
 2014-03-21 18:01:26 +01:00
reger
c297de5145 remove check for unused virtual path /currentyacypeer/ 
-  del jqueryheader.template (not used)
 2014-03-21 03:02:19 +01:00
orbiter
3c8d6e1eee added adminAccount switch to ConfigAccounts_p servlet to switch on 
protection of all pages; some refactoring as well
 2014-03-20 22:11:49 +01:00
orbiter
7d24bcb98d added flag to require that all web pages, even such without a "_p" 
extension require authorization. (default off)
 2014-03-20 19:09:47 +01:00
Michael Peter Christen
7a6658abec removed synchronization in embedded solr connection (that was probably 
a mistake?)
 2014-03-19 16:21:03 +01:00
Michael Peter Christen
a7d4379ef9 fixed shutdown of solr cores in case that more than one local core is to 
be closed (this happens if webgraph is enabled and the index is dumped
using /IndexControlURLs_p.html
 2014-03-19 12:23:40 +01:00
Michael Peter Christen
453bfd0f17 removed unused variables and warnings 2014-03-19 09:29:01 +01:00
Michael Peter Christen
05655d98df Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2014-03-17 11:57:01 +01:00
reger
9f02d2c47b fix: remove link to triplestore in Vocabulary_p (triplestore does not longer exist) 
- should be investigated in more detail to look for additional implications
Remove "yacyaction" from proxyservlet as it was only needed for removed interaction routines.
 2014-03-16 22:11:19 +01:00
reger
81a846ec33 fix: set YaCy CONNECTION_PROP_HOST Header in ProxyServlet to host incl. port 2014-03-16 20:51:32 +01:00
reger
251be9ecfa remove unused ProxySettings ref. from loader 
clean unused whois test code
 2014-03-16 05:19:01 +01:00
reger
82dc815af9 cleanup: remove unrelated and unused code 2014-03-16 00:15:12 +01:00
Michael Peter Christen
85a427ec54 support for multiple sitemaps in robots.txt 2014-03-14 13:33:23 +01:00
reger
a373fb717d remove more unused from legacy server.http 
- triggerOnlineAction not used
- useTemplateCache not used
 2014-03-14 03:12:04 +01:00
reger
749d020aeb remove redundant url string manipulation in HTTPDProxyHandler 
(still used by ProxyServlet)
 2014-03-14 02:24:12 +01:00
reger
612294cf84 use servletPath in ProxyServlet instead of fixed name 
to allow servlet-mapping via web.xml
 2014-03-13 02:46:05 +01:00
reger
1d01672bd3 fix DCEntry.getIdentifier 
on successful url parameter
 2014-03-12 23:35:57 +01:00
Michael Peter Christen
b08375da33 fix for bad/missing values of size_i 2014-03-11 09:51:04 +01:00
reger
6306d28a6a OAI import get multivalued keywords (dc:subject) 2014-03-09 03:15:35 +01:00
reger
0a8c8102de allow YaCy to start w/o ssl if JKS init fails 2014-03-07 20:06:14 +01:00
sixcooler
0b2101c59c Speed up the ProxyHandler: 
simplified cache-storing and make it concurrent in order to free the
clientconnection asap
let other prozesses wait on proxy-access like it was bevore
 2014-03-07 17:47:09 +01:00
reger
516f8c2489 fix: to allow unix scripts (bin/*.sh) to allways submit http admin apicalls 
using auth via config hash (legacy requirement)
 2014-03-07 00:16:57 +01:00
Michael Peter Christen
ea3aa30593 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2014-03-06 03:33:33 +01:00
reger
dd5bf0b71b cleanup old reference to HTTPDemon.setAlternativeResolver 
optimize .yacyh check in AbstractRemoteHandler
 2014-03-06 03:08:04 +01:00
Michael Peter Christen
51800007c4 - added concurrency to postprocessing of webgraph document 
- bundeled separate webgraph postprocesing steps into one
 2014-03-06 01:43:48 +01:00
Michael Peter Christen
5f4a6892c1 enhanced RowSet re-sort limit for small sets 2014-03-05 23:28:19 +01:00
reger
351c2be68d fix: make sure adminAccount changes made via ConfigAccounts_p are effective immediately 
force to remove current credentials from knownuser cache
 2014-03-05 02:59:27 +01:00
reger
5c9dcc269d improve OAI-PMH import identifier recognition 
- find best fittng identifier (url) by checking all given dc:identifier in record (many entries proviede several identifiers)
  as identifier is currently a multivalued field use "getParams" in preference of splitting the 1st string by ";" 
- add resolve DOI:... identifier via http://dx.doi.org/
 2014-03-04 03:08:37 +01:00
Michael Peter Christen
0e7d249a69 fixed another shutdown problem (only occurs if webgraph core is enabled) 2014-03-04 01:36:38 +01:00
Michael Peter Christen
e485fbd0ce - let crawl loader jobs die after 10 seconds without new jobs 
- corrected shutdown order t prevent a deadlock during shutdown
 2014-03-04 00:33:13 +01:00
Michael Peter Christen
bcd9dd9e1d enhanced concurrent loading by using a fixed set of concurrent loader 
processes in favor of throwaway-processes. The control mechanism does
less often report a 'queue full' message to the busy loop which then
does not perform a long busy waiting; instead all requests are queued
and new loader processes are started if necessary up to a given limit
(as set before)
 2014-03-03 22:13:40 +01:00
orbiter
051328271c bugfix-bugfix 2014-03-02 21:13:38 +01:00
orbiter
eedcbcd906 bugfix to proxy handler: recognize the own yacyh-host 2014-03-02 12:10:19 +01:00
orbiter
d68e5ad0c4 NPE fix for Thread name (just commited yesterday, sorry) 2014-03-02 11:20:48 +01:00
reger
6878c90f99 fix: IPv6 INTRANET_PATTERNS for local ip (see http://bugs.yacy.net/view.php?id=378) 
requiring following ":" for fc and fd prefix and made pattern match case insesitive
- add some more ipv6 test cases to MultiProtocolURLTest.java
 2014-03-02 06:13:21 +01:00
reger
a2e5ea2026 status panel link to set max mem 
+url proxy same error text as in transparent
 2014-03-01 00:56:45 +01:00
Michael Peter Christen
6ed9c0164e attaching names to all Threads to get a better view in profiling tools 
like VisualVM
 2014-02-28 15:02:01 +01:00
Michael Peter Christen
fdaeac374a - enhanced postprocessing speed and memory footprint (by using HashMaps 
instead of TreeMaps)
- enhanced memory footprint of database indexes (by introduction of
optimize calls)
- optimize calls shrink the amount of used memory for index sets if they
are not changed afterwards any more
 2014-02-28 14:01:09 +01:00
reger
ba49ff81ed little more verbose proxy 403 error message 2014-02-28 03:14:07 +01:00
Michael Peter Christen
d325cb8912 fixes and enhancements for postprocessing 2014-02-28 02:51:14 +01:00
Michael Peter Christen
7c1b968378 another fix for the shutdown exceptions 2014-02-28 01:53:32 +01:00