Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
14,471 Commits 1 Branch 20 Tags 836 MiB
040cd8be6d
Commit Graph

8947 Commits

Author SHA1 Message Date
reger24
a7e93d9328 Add option to add host to default blacklist from search result 
- added authorized ikon/button to blacklist a host
- host is added to default blacklist
- insired by https://github.com/yacy/yacy_search_server/issues/213#issuecomment-412485190
 2022-02-09 19:42:04 +01:00
reger24
027e284ef9 Enhance notability of current blacklist by diff color in header 
in servlet Blacklist_p.html
bugfix for 18dddb74c9
 2022-02-06 09:43:59 +01:00
reger24
18dddb74c9 Harmonize loading/reading blacklist 
between init  and servlet to use the same procedures
-added BlacklistHelper.blacklistToSortedArray to simplify use in servlet
 2022-02-06 00:10:55 +01:00
reger24
f28d705cd0 update IndexBroser_p add to blacklist button 
add feedback to user on success
 2022-02-03 03:25:13 +01:00
Michael Peter Christen
52fe2ed8ba Merge branch 'master' of git@github.com:yacy/yacy_search_server.git 2022-02-01 04:21:55 +01:00
Michael Peter Christen
39e7bbac13 removed deprecation warning for new Double() 2022-02-01 04:20:55 +01:00
reger24
6a5f0b3684 Servlet IndexBroser_p add button "Add to blacklist" 
allows to add the displayed host to add to the default blacklist
 2022-01-30 21:01:23 +01:00
Lukas Fülling
111cf48642 add missing prop 2022-01-29 19:28:57 +01:00
reger24
f33e0ed7fd revert commit 17fd1a4616 
wrong file selected
 2022-01-29 12:18:07 +01:00
unknown
17fd1a4616 delete .idea not needed in distribution 
.idea is created locally by IntelliJ IDEA upon import as gradle project to store IDEA specific settings.
No need to include in distribution
 2022-01-29 10:45:37 +01:00
Daleth Darko
3ced06c731 Various javadoc fixes 2022-01-26 11:22:43 +01:00
reger24
6a1e259fd0 Fix NPE in Switchboard . getURL https://github.com/yacy/yacy_search_server/issues/441 2022-01-26 06:07:38 +01:00
reger24
eae16287e9 Added epub (ebook) format to existing zipParser 
*.epub files are zip files containing xhtml files with content and other artifact files,
which the zipParser can  already feed to index
- extension "epub"
- mime "epub+zip"
 2022-01-24 13:51:27 +01:00
reger24
3e34f7c596 Import Ant build.xml into Gradle and use old compile of servlets in Gradle 
to be able to use/reuse Ant targets where task has not been implemented in Gradle build.
- use the import to include the compile of htroot as first important task

  ! it is possible that first build fails an compile of GitRevTask.jar !
  ! solution/workaround -> use "ant all" once to compile GitRevTask.jar !

- adjusted build.xml a little
   - split compile-core into compile-core and compile-htroot to have a target for htroot comp. only
   - set build-path to reuse Gradles build directory
   - (fix javadoc failure)

- changed the filtered-copy of yacyBuildProperties.java to ! the build path :-(
  as current (copy,delete,exclude) is complicated and not migration worthy,
  used simple/straigt forward approach (using a yacyBuildProperties.java.template file as copy source)
 2022-01-18 20:00:55 +01:00
reger24
398b105781 Prevent that YaCy always starts with a exception message on none Apple systems 
Perform try to access com.apple.eio.FileManager  only on none Win systems
 2022-01-18 13:02:12 +01:00
Lukas Fülling
e8a00007f6 add setting for public facing port 2022-01-11 17:10:48 +01:00
Michael Peter Christen
d7b17d8935 fixed missing thread name revert after balancer waiting 2021-12-22 01:46:18 +01:00
Michael Peter Christen
bd3f2483a1 replaced url and date retrieval by only url retrieval 
This should prevent that the search index is used for freshnes of the
index entry.
 2021-12-20 16:23:05 +01:00
Michael Peter Christen
163ba26d90 replaced check for load time method 
instead of loading the solr document, an index only for the last loading
time was created. This prevents that solr has to fetch from its index
while the index is created. Excessive re-loading of documents while
indexing has shown to produce deadlocks, so this should now be
prevented.
 2021-12-20 03:47:56 +01:00
Michael Peter Christen
1ead7b85b5 remove compiler warning 
"warning: [try] explicit call to close() on an auto-closeable resource"
 2021-12-13 12:28:34 +01:00
Michael Peter Christen
59777010dc Merge branch 'master' of git@github.com:yacy/yacy_search_server.git 2021-11-18 00:49:56 +01:00
Michael Peter Christen
7898815c41 disabling concurrent logging 
(maybe temporary)
 2021-11-18 00:49:46 +01:00
sgaebel
4bf6954474 uses clientBuilder not HttpClients.custom() to have these inside the 
Pool too
 2021-10-31 23:06:33 +01:00
sgaebel
cdf901270c always use HTTPClient by 'try with resources' pattern to free up 
resources
 2021-10-31 23:06:23 +01:00
sgaebel
69adaa9f55 makes our HTTPClient closable 2021-10-31 23:06:02 +01:00
sgaebel
fc4275f901 handle all references for client, response, request to be able to close 
them
 2021-10-31 23:05:50 +01:00
sgaebel
e7d3a363f2 refactor to use finish() 2021-10-31 11:22:35 +01:00
sgaebel
4fc876f4a3 revert back to use EntityUtils.consumeQuietly - as it simply closes the 
underlying stream
 2021-10-31 11:22:28 +01:00
sgaebel
4f0392e93e refactor use of AuthSchemeProvider 2021-10-31 11:21:59 +01:00
sgaebel
b74f337859 removes double setting of UserAgent 2021-10-31 11:21:06 +01:00
sgaebel
965748fefb some refactoring using try with resources 2021-10-31 11:20:28 +01:00
Michael Peter Christen
552ab7051b fix for warc importer 2021-10-25 19:35:15 +02:00
Michael Peter Christen
3c86b7b780 attempt to make a Mac Release using gradle 
This is almost working with many workarounds:
- run rm lib/yacycore.jar
- run ./gradlew clean build bundleNative
- run ant clean all
- run again rm lib/yacycore.jar
- run ./fixMacBuild.sh

The build is then inside build/mac/YaCy.app

Right now this works so far but it does not have the correct release
number inside.

Target is to make this working for Windows releases and to embedd jre
entirely.
 2021-10-25 18:37:39 +02:00
Michael Peter Christen
999c819e3e Merge branch 'master' of https://github.com/yacy/yacy_search_server.git 2021-10-24 20:50:14 +02:00
Michael Peter Christen
fd770e90e2 spike to identify paths for YaCy within mac application bundles 2021-10-24 20:49:59 +02:00
Michael Peter Christen
d19872fd26 making sure that crawl queues are closed correctly to prevent data loss 2021-10-14 00:30:04 +02:00
sgaebel
90507c0fdc comments out printing query params to std.out 2021-10-04 18:03:06 +02:00
Michael Peter Christen
be0aebad84 fixes https://github.com/yacy/yacy_search_server/issues/424 2021-10-04 14:38:49 +02:00
Michael Peter Christen
63ad8ce6b2 removed ymarks 
had not been used since a long time
 2021-09-16 22:23:51 +02:00
Michael Peter Christen
ef5a71a592 enhanced crawl start response time 
for very very large crawl start lists
 2021-09-16 21:01:01 +02:00
Michael Peter Christen
4cadd557dc removed synchronization in table creation 
to avoid possible deadlocks when handling OnDemandOpenFileIndex
which happens quite often during wide crawling
 2021-09-15 19:34:49 +02:00
admin
9b7668fa58 reduced memory footprint during indexing/crawling 2021-08-24 12:24:52 +02:00
Michael Peter Christen
e6a87e0426 enhanced crawler 
a main problem when crawling is long waiting time cuased by crawl-delay
values from robots.txt entries. that attribute is not supported by
google and interpreted by yandex and bing in different ways. In large
crawls there is always one host which blocks the whole crawl with
extreme large values. YaCy now still obeys crawl-delay but limits them
to 10 seconds.
Additionally the blocking logic when loading new robots.txt was analyzed
and a deadlock was removed. Furthermore the construction of new queue
lists was redesigned and it was ensured that always a large list of
different hosts for host-balancing is provided for the loader.
 2021-08-17 15:23:21 +02:00
Michael Peter Christen
e9c5e78868 replaced new Number(Number) with Number.instanceOf 
to remove deprecation warnings for Java 9
 2021-08-08 00:39:03 +02:00
Michael Peter Christen
9e13d77de4 removed call to class.finalize() because of deprecation in java 9 
next: removal of finalize() implementation
after testing with assert false
 2021-08-07 18:57:49 +02:00
Michael Peter Christen
9ef4503672 fixed some newInstance() warnings 
.. by adding .getDeclaredConstructor()
 2021-08-07 18:46:53 +02:00
Michael Peter Christen
1d41380f0a better support for mac-specific tray functions in java 9 2021-07-12 17:27:59 +02:00
Michael Peter Christen
e81b770f79 enabled crawl starts with very large sets of start urls 
i.e. 10MB large url list with approx 0.5 million start points
 2021-06-30 10:45:58 +02:00
Michael Peter Christen
c623a3252e fix for jdk 14 bug 2021-04-23 09:11:03 +02:00
Michael Peter Christen
dbd211a1ad removed/replaced reflection in memory tool 2021-04-22 20:24:13 +02:00
Michael Peter Christen
1cdb21592b added hazelcast and some modifications to align legacy YaCy with 
YaCyGrid
 2021-04-15 20:39:22 +02:00
Michael Christen
42ea2a1c6f
Merge pull request #405 from jfhs/jfhs/support-all-html-entities 
Improve HTML entities support
 2021-03-31 01:44:54 +02:00
Michael Christen
b2af745dd6
Merge pull request #404 from lnceballosz/master 
NGI0 - Updating licensing aspects according REUSE
 2021-03-30 23:48:21 +02:00
jfhs
10bddc2c2d Decode HTML entities in all property values by default 2021-03-30 22:24:55 +02:00
jfhs
2135d259e3 Replace hardcoded html/xml entities with a file, support decoding all defined HTML entities 2021-03-30 22:24:54 +02:00
Michael Peter Christen
8f876a8c72 added concurrency to enhance indexing speed during json surrogate import 2021-03-30 12:07:36 +02:00
Michael Peter Christen
f8cbaeef93 Merge branch 'master' of git@github.com:yacy/yacy_search_server.git 2021-03-29 18:46:53 +02:00
Michael Peter Christen
a857e3d3d5 fix for json importer 2021-03-29 18:46:42 +02:00
sgaebel
1546232c94 adds ranking for multi document queries only 2021-03-20 17:48:35 +01:00
sgaebel
93b353d22d does not boost or add fields for zero-row-queries (exists()) 2021-03-20 17:48:26 +01:00
sgaebel
f16cd154f7 removes unused imports and variables 2021-03-20 15:14:09 +01:00
sgaebel
c69c462a15 replaces a expensive getLoadTimeURL() by exists() 
refactors urlExists to getHarvestProcess as that is what it does
 2021-03-20 15:01:31 +01:00
sgaebel
a5488ac8f5 uses edismax queries on query counts > 1 only 2021-03-20 01:06:09 +01:00
sgaebel
26223dc25a replaces getLoadTime() by exists() with a simpler query 
since solr-8.8.1 getLoadTime() causes a high cpu usage
 2021-03-20 01:06:02 +01:00
sgaebel
8e4d014c06 removes useless SolrRequestInfo.clearRequestInfo(), avoids spamming the 
log
 2021-03-18 22:33:39 +01:00
Lina Ceballos
a96752f5ab adding SPDX license and copyright headers 2021-03-11 12:17:11 +01:00
Michael Peter Christen
e18d0ef544 trying to set a higher priority to the process that is involved in index 
export
 2021-03-09 00:04:05 +01:00
Michael Peter Christen
8b4394a6c5 fixes for solr 8.8.1 migration 
- replace new guava 30 with older 25 because that is the correct
dependency for solr 8.8.1. The newer one did actually not work!
- index will be crated in a DATA/INDEX/freeworld/SEGMENTS/solr_8_8_1
subfolder. The older solr_6_6 index is not touched but also not
migrated. The index starts with fresh (empty) content.
- Older indexes must be migrated by hand (export/import) so far until a
better solution is found.
- Large schema adoptions for lucene 8.8.1
 2021-03-08 13:39:27 +01:00
Michael Peter Christen
ed9789214e fixed seed initialization problem 2021-03-06 13:35:46 +01:00
Al Sutton
8ade8b8775 Remove forced clear to match new behaviour in 2da71c2a40 2021-03-04 16:37:56 +00:00
Al Sutton
09695fc6d3 Update exceptions to match updated API 2021-03-04 16:34:02 +00:00
Al Sutton
69014a701e Update API Usage 2021-03-04 16:14:56 +00:00
Michael Peter Christen
3da7628117 use environment variables to overwrite configuration variables 
you can i.e. do:
export YACY_PORT=8092 && ./startYACY.sh
Just append "YACY_" to uppercase version of environment variables and
replace all "." with "_".
 2021-02-09 20:26:49 +01:00
Michael Peter Christen
13a2e6dc6e Merge branch 'master' of https://github.com/yacy/yacy_search_server.git 2021-01-25 11:49:32 +01:00
Michael Peter Christen
0ae8ccf657 Make it possible to set an empty password disabling the authentication 
protocol completely
If you set now an empty password, then the http server will not ask to
authentify. This is required for environment where we attach an outside
authentification service like keycloak or similar using authentication
in an ingress proxy.
This change is part of the approach to run YaCy inside of a kubernetes
cluster where we do not want individual authentication of peers and want
to apply a ingress authentication.
 2021-01-25 11:49:21 +01:00
Michael Peter Christen
96592a10cf added option to set yacy configuration values using environment 
variables
To use that feature, set an environment variable with prefix "yacy." and
suffix identical to the yacy configuration attribute name.
Additionaly we implemented a way to set a peer name using the setting
"network.unit.agent". This can therefore now be used to set a peer name
with the java call parameter
-Dyacy.network.unit.agent=anonymous
The purpose for this feature is the ability to set peer names in
mass-deployed kubernetes clusters to the same name to prevent that we
are flooding peer name statistics with auto-deployment-generated names.
 2021-01-24 22:50:37 +01:00
Michael Peter Christen
198826c362 added network scanner process to discover all YaCy peers in the intranet 
this will be used to wire YaCy peers in a kubernetes cluster
 2021-01-23 15:14:49 +01:00
Michael Peter Christen
d9602e8325 Implemented a new syntax in the template engine to simplify json APIs 
Added also an example for one of the existing APIs. The problem is the
comma separator between objects which must not be there for the last
entry in a sequence. The new syntax adds the separator symbol
automatically.
 2021-01-18 00:01:08 +01:00
Michael Peter Christen
5a7f12a9c1 allow network scans for non-standard http/https ports 2021-01-11 00:28:24 +01:00
sgaebel
b8d264f7ec fixes logging 2021-01-04 20:53:40 +01:00
Michael Peter Christen
4c920d05b5 removed superfluous lines 2020-12-29 20:19:58 +01:00
Michael Peter Christen
907f121d0c do not overwrite PW with random PW 2020-12-29 20:18:25 +01:00
Michael Peter Christen
3e6a1e0a49 fixed surrogate process counter 2020-12-28 18:26:22 +01:00
Michael Peter Christen
d3526c52af fixed a problem in warc importer: do not fail if single WARC entries are 
faulty
 2020-12-28 17:05:06 +01:00
Michael Peter Christen
3078b74e1d Merge branch 'master' of https://github.com/yacy/yacy_search_server.git 2020-12-22 00:46:56 +01:00
Michael Peter Christen
01cc32217f fixed apicall call method parameters 
and verification in transaction manager
which did not have and exception for localhost/basic authentication
 2020-12-22 00:46:47 +01:00
Michael Peter Christen
63f58e4785 enhanced strategy in host browser 
limit number of fresh hosts in round robin hashes
 2020-12-20 23:15:55 +01:00
Michael Peter Christen
9be36800a4 increased redirect depth by one 
this makes sense if one redirect replaces http with https and another
replaces www subdomain by without (and vice versa)
 2020-12-20 19:44:16 +01:00
Michael Peter Christen
d0abb0cedb enabling all crawl profiles in all network modes 
also: increased default internet crawl speed to
4 urls/s/host
 2020-12-19 01:00:51 +01:00
Michael Peter Christen
baad56d83d beautified default peer names 2020-12-14 02:08:49 +01:00
Michael Peter Christen
43a9f4f574 updated solr 6.6.6 -> 7.7.3 
dropped GSA support (GSA API is still in YaCy Grid)
The 6.6.6 solr index works without migration also with 7.7.3
 2020-12-12 02:06:43 +01:00
Michael Peter Christen
c0d9a3e9a7 turned HostBrowser into a admin-only page, now called IndexBrowser 
This was required because spiders and bots crawled through this page and
created load on the peer without use for the user or the YaCy network.
 2020-12-11 00:50:52 +01:00
Michael Peter Christen
d359d521a1 fixed warc importer 
The importer tried to import a gziped files as plain warc.
It will now check the file extension and use a unzip automatically
on-the-fly.
 2020-12-10 11:19:25 +01:00
Michael Peter Christen
e54ab39958 Going back to basic authentication for console/shell commands 
This does not affect security because:
- it is going to localhost only
- only users who have already access to the pw hash can do this
- no clear text pw is transmitted because that is not stored anywhere
The switch to basic is required because these commands are required
in the context of hosting on root servers and docker containers
where a password change must be done. But the password shell command
was not working without password which made the concept unusable.
This deficit made it virtually impossible for root server operators
to use YaCy because they had been unable to set up a proper password.
 2020-12-09 02:36:55 +01:00
Michael Peter Christen
6271e9122c javadoc fix 2020-12-09 02:22:47 +01:00
Michael Peter Christen
e0f4e3fd9a enhanced ability to debug the code 2020-12-09 02:22:30 +01:00
Michael Peter Christen
eea2d71851 prevent creation of auth schema factories every time a servlet is called 2020-12-06 01:49:34 +01:00
Michael Peter Christen
fcc9386ed3 enhanced the (already fast!) png exporter 2020-12-03 12:18:07 +01:00
Michael Peter Christen
4e9b425f98 missing fix for latest commit 2020-12-03 00:40:51 +01:00
Michael Peter Christen
3213d9db37 updated jetty from 9.4.17 to 9.4.35 
and fixed a bug in ServerSideIncludes that appeared only in that recent
version of jetty
 2020-12-03 00:21:15 +01:00
Michael Peter Christen
787fec0658 reduced complexity - removed concurrency in sort 2020-12-02 18:39:45 +01:00
Michael Peter Christen
cef5fde343 adding message to UI to make port change transparent 2020-12-02 18:05:38 +01:00
Michael Peter Christen
52228cb6be added a gc to cleanup process (once every 10 minutes) 2020-12-02 00:13:00 +01:00
Michael Peter Christen
22841ffbf1 creating a threaddump during every cleanup process 
to be able to find out what a peer did (not) last time before a crash
 2020-12-01 03:00:24 +01:00
Michael Peter Christen
36e616271b do better documentation on how to set a default password 2020-12-01 02:18:08 +01:00
Michael Peter Christen
df2bf9ef28 try to fix maven build error 2020-11-29 14:24:33 +01:00
Michael Peter Christen
264bab6700 trying to fight the UI unavaiability 
this path addresses a possible issue with too many open connections to
remote peers
 2020-11-29 14:15:34 +01:00
Michael Peter Christen
7947baeb49 removed all remaining deprecation warnings 2020-11-23 00:03:18 +01:00
Michael Peter Christen
c0f6d6e11d removed one deprecation warning for jetty library initializing ssl 
server port
 2020-11-22 23:27:58 +01:00
Michael Peter Christen
133440a7a6 some debug lines 2020-11-22 23:12:04 +01:00
sgaebel
3431f91db9 removes unused 'unused' tokens 2020-08-04 20:09:34 +02:00
sgaebel
fc03c4b4fe removes some warning and unused objects 2020-08-03 20:44:31 +02:00
sgaebel
4a495df63a removes some deprecation-warnings 2020-07-31 17:28:06 +02:00
sgaebel
dd9d4b1188 replace org.junit.Assert.assertThat by 
org.hamcrest.MatcherAssert.assertThat from hamcrest 2.2 to avoid
deprecation-warning
 2020-07-28 19:09:26 +02:00
sgaebel
df9ea0a42a removes some warnings: unused imports, params 2020-07-27 22:20:49 +02:00
sgaebel
9bc2297161 fixes deleting during recrawl 2020-07-22 22:15:00 +02:00
sgaebel
80785b785e adds deleting during recrawl 2020-07-09 19:32:16 +02:00
Michael Peter Christen
e0ad8ca9da replaced json library from JSON.org with libandroid-json-java 
This fixes https://github.com/yacy/yacy_search_server/issues/347
 2020-04-24 11:45:25 +02:00
Michael Peter Christen
ea8df27e95 modified org.json.* library to fit into the YaCy environment 
as drop-in replacement.
Also made some fixes and enhancements to the library.
 2020-04-24 11:42:06 +02:00
Michael Peter Christen
60dc1241a3 added org.json.* library 
from https://android.googlesource.com/platform/libcore/+/refs/heads/master/json/src/main/java/org/json
as a preparation step for
https://github.com/yacy/yacy_search_server/issues/347
 2020-04-24 10:28:43 +02:00
Michael Peter Christen
053e54a2c7 grand CORS for json files 2019-11-05 11:50:56 +01:00
Michael Christen
cfa27d2fd5 fixed links 2019-10-20 20:20:50 +02:00
Michael Christen
cb20aa7e54 removed donation message in search result column 2019-10-17 01:35:44 +02:00
Michael Christen
25227676ae removed some warnings 2019-09-28 02:07:08 +02:00
luccioman
6b45cd5799 New optional crawl filter on the URL a doc must match to crawl its links 
For finer control over which parsed documents can trigger an addition of
their links to the crawl stack, complementary to the existing crawl
depth parameter.
 2019-05-01 08:54:19 +02:00
luccioman
d16bc99835 Added "Show Metadata" links to the ViewFile.html links mode 
To conveniently follow parsed links in the file viewer
 2019-04-18 15:31:38 +02:00
luccioman
a5771b1f14 Made SNI extension user configurable without the need for server restart 
TLS Server Name Indication (SNI) extension activation can now be
configured with the new Settings_p.html?page=httpClient administration
page.
SNI extension is also now enabled by default, as in 2019 the
unrecognized_name(112) alert is more properly handled by major web
servers TLS implementations, following the RFC 6066 standard.

Related YaCy issues : #153 #189 and #272
JDK 1.7 bug :
https://bugs.java.com/bugdatabase/view_bug.do?bug_id=7127374
Apache httpd issue :
https://bz.apache.org/bugzilla/show_bug.cgi?id=56241
RFC 6066 : https://tools.ietf.org/html/rfc6066#section-3
 2019-04-14 15:41:13 +02:00
luccioman
e90405b6f0 Support parsing audio URLs without file extension 
Added also a Junit for the audio tag parser
 2019-04-09 11:40:21 +02:00
luccioman
a8316c79da Allow JS resorting of search results by unauthenticated users 
Acces rate limitations to this search mode by unauthenticated users are
set low by default to prevent unwanted server overload but can be
customized through the SearchAccessRate_p.html configuration page

Fixes #291
 2019-04-03 14:21:53 +02:00
luccioman
0ab2b49c31 Made /yacysearch access rate limitations user configurable 
With a new admin page at /SearchAccessRate_p.html in menu Network Access
> Local Search > Access Rate Limitations
 2019-04-02 17:42:50 +02:00
luccioman
5b7e41202a Added Solr GSA writer support for responses from remote instances 2019-03-27 18:23:41 +01:00
luccioman
4d8a948455 Properly close PDF snapshots loaded with pdfbox library 2019-03-22 09:50:30 +01:00
luccioman
74e6d6e984 Added Solr GrepHTML writer support for responses from remote instances 2019-03-20 18:24:16 +01:00
luccioman
5e6501974d Added Solr snapshots writer support for responses from remote instances 2019-03-19 11:25:44 +01:00
luccioman
384c37102c Improve accuracy of total results count on latest pages in Stealth mode 
Previously, when mixing results from local RWI and local Solr (Stealth
mode), total local Solr count could be ignored on last result pages,
when the page offset was higher than local Solr count but lower than
total RWI count.
 2019-03-04 10:05:47 +01:00
luccioman
5e9a08355a Improved logging for federated search 
- Do not use spaces in logger identifier name so the log level can be
configured in yacy.logging
- Hold the logger instance to avoid the logging system to look for it
from its name at each appended log message
 2019-02-02 09:59:24 +01:00
luccioman
9782a98a9c Added the possibility to customize facets sort type and direction 
Previously search navigators/facets elements were sorted only by counts.
Now from the ConfigSearchPage_p.html admin page, sort direction
(ascending/descending) and type (on counts or labels) can be customized
independently for each navigator.
 2019-01-24 18:43:06 +01:00
sgaebel
c2398fd890 remove warnings: 'Statement unnecessarily nested within else clause' 2019-01-10 20:02:57 +01:00
sgaebel
811d40a6c4 taking care of closing inputstreams, HTTPClient 2019-01-04 18:58:49 +01:00
sgaebel
8d2e7262d9 Recrawl: 
- set the chunksize to 100 to meet the max of the embedded solr
- re-enable sorting (the case where we switched it of should be away)
- enable recrawling on remote-solr
 2019-01-04 18:46:59 +01:00
sgaebel
8f58c1dcfa extend the SolrServlet to be usable as remote solr (incl. update) 
this feature needs to be enabled by uncomment the url-pattern
 2019-01-04 18:27:44 +01:00
luccioman
7223a2fdb1 Removed usage of now deprecated Jetty function 2018-12-22 14:42:22 +01:00
luccioman
440d9f2fa0 Exclude peers with empty or disabled RWI from remote RWI search 2018-12-20 14:53:01 +01:00
luccioman
08ea0b0397 Added a configurable timeout to wkhtmltopdf calls for pdf snapshots 
Necessary to prevent blocking the indexing workflow when some
wkhtmltopdf renderings fail without terminating
 2018-12-11 22:31:31 +01:00
luccioman
3fb449b3b6 Properly resolve relative URLs against document URL in html base tags 
Fixes issue #256
 2018-12-06 20:18:00 +01:00
luccioman
73a6e45524 Extended detection of external tools used for Snapshots generation 
This enable detecting wkhtmltopdf and Imagemagick convert executables
when they are at system Path in addition to common installation paths.
 2018-12-06 09:53:08 +01:00
luccioman
7dc1f60619 Fixed detection of absolute data folder path on MS Windows 2018-11-18 10:08:20 +01:00
luccioman
595e144797 Trace a message on incomplete proper server finish when killing process 2018-11-15 17:32:22 +01:00
luccioman
9daeea823b Fixed concurrency issue on cache used for circles rendering 
Without synchronization lock, concurrent rendering of images including
circles could lead to glitches as reported in issue #248
 2018-11-10 22:00:49 +01:00
Michael Peter Christen
c347e7d3f8 Merge branch 'master' of https://github.com/yacy/yacy_search_server.git 2018-11-08 14:42:52 +01:00