Commit Graph

4421 Commits

Author SHA1 Message Date
Michael Christen
c36bdbf78d refactoring 2022-10-03 09:37:16 +02:00
Michael Peter Christen
1e1107c97c clean-up and new servlet method caching 2022-10-02 23:39:00 +02:00
Michael Peter Christen
adbda4c71b moved all remaining servlet classes to new location 2022-10-02 23:22:12 +02:00
Michael Peter Christen
33889b4501 moved more servlets to new location 2022-10-02 22:57:58 +02:00
Michael Peter Christen
6d388bb7bf refactoring - moved htroot/yacy classes 2022-10-02 22:26:53 +02:00
Michael Peter Christen
48fcf3b3b5 alternative servlet method, tested with wiki
may become the future method to store servlets
2022-09-30 18:29:01 +02:00
Michael Peter Christen
d23dea2642 refactoring 2022-09-30 17:42:21 +02:00
Michael Peter Christen
23f1dc3741 addressing/fixing some concurrency issues from
https://github.com/yacy/yacy_search_server/issues/505
2022-09-30 08:01:13 +02:00
Michael Peter Christen
9c1bc533fa removed hazelcast because it is phoning home, see also:
https://github.com/yacy/yacy_search_server/issues/504
2022-09-28 17:30:37 +02:00
Michael Peter Christen
fc98ca7a9c removed ContentControl servlet and functinality
This was not used at all (as I know) and was blocking a smooth
integration of ivy in the context of an existing JSON parser.
2022-09-28 17:25:04 +02:00
Thomas Koch
3116713672 rm buildDate from build.xml and its usages
The https://reproducible-builds.org project invests a lot of work
to make builds reproducible. This is a security property. It allows
to compare the build of binaries from different builder machines.
If they are identical, it means that either the builds have not
been manipulated or an attacker managed to attack all builder
machines in exactly the same way.

One problem that the reproducible-builds project often sees is
that projects include the build time in their binaries. This
makes builds unreproducible for apparently no reason. The build
date should not be of interest since binaries built on different
dates but from the same source code should not be different.

Thus I decided to remove the build date instead of re-implementing
the functionality without the GitRev task. Anyways the reported
date was not the build date but the date of the last git commit
which is even less informative. The git commit ID would have
information value but should only be relevant for "nightly builds".
2022-07-10 11:32:38 +00:00
Thomas Koch
572558244a rm unused build properties PKGMANAGER, RESTARTCMD, DESTDIR
PKGMANAGER is always false, thus the java code wrapped in
if statements for this property is dead code and can also
be removed.

The Debian packaging removed in c4659f0fb0
did set the PKGMANAGER property to true. When we do distro
packages again, we can revisit this commit and redo it with
property files instead.

RESTARTCMD is only used inside those dead code.

DESTDIR is never used even in the build.xml
2022-07-10 10:14:51 +00:00
Michael Peter Christen
3d138d3fdd catch error when initializing hazelcast
should fix https://github.com/yacy/yacy_search_server/issues/468
2022-06-20 17:27:56 +02:00
Burkhard
a6a9828181
Merge pull request #440 from lfuelling/master
Add setting for public facing port
2022-02-11 08:09:17 +01:00
reger24
141e86964e Fix compile deprecation warning
warning: [removal] AccessControlException in java.security has been deprecated and marked for removal
2022-02-11 00:27:55 +01:00
reger24
a7e93d9328 Add option to add host to default blacklist from search result
- added authorized ikon/button to blacklist a host
- host is added to default blacklist
- insired by https://github.com/yacy/yacy_search_server/issues/213#issuecomment-412485190
2022-02-09 19:42:04 +01:00
reger24
027e284ef9 Enhance notability of current blacklist by diff color in header
in servlet Blacklist_p.html
bugfix for 18dddb74c9
2022-02-06 09:43:59 +01:00
reger24
18dddb74c9 Harmonize loading/reading blacklist
between init  and servlet to use the same procedures
-added BlacklistHelper.blacklistToSortedArray to simplify use in servlet
2022-02-06 00:10:55 +01:00
reger24
f28d705cd0 update IndexBroser_p add to blacklist button
add feedback to user on success
2022-02-03 03:25:13 +01:00
Michael Peter Christen
52fe2ed8ba Merge branch 'master' of git@github.com:yacy/yacy_search_server.git 2022-02-01 04:21:55 +01:00
Michael Peter Christen
39e7bbac13 removed deprecation warning for new Double() 2022-02-01 04:20:55 +01:00
reger24
6a5f0b3684 Servlet IndexBroser_p add button "Add to blacklist"
allows to add the displayed host to add to the default blacklist
2022-01-30 21:01:23 +01:00
Lukas Fülling
111cf48642 add missing prop 2022-01-29 19:28:57 +01:00
reger24
f33e0ed7fd revert commit 17fd1a4616
wrong file selected
2022-01-29 12:18:07 +01:00
unknown
17fd1a4616 delete .idea not needed in distribution
.idea is created locally by IntelliJ IDEA upon import as gradle project to store IDEA specific settings.
No need to include in distribution
2022-01-29 10:45:37 +01:00
Daleth Darko
3ced06c731 Various javadoc fixes 2022-01-26 11:22:43 +01:00
reger24
6a1e259fd0 Fix NPE in Switchboard . getURL https://github.com/yacy/yacy_search_server/issues/441 2022-01-26 06:07:38 +01:00
reger24
eae16287e9 Added epub (ebook) format to existing zipParser
*.epub files are zip files containing xhtml files with content and other artifact files,
which the zipParser can  already feed to index
- extension "epub"
- mime "epub+zip"
2022-01-24 13:51:27 +01:00
reger24
3e34f7c596 Import Ant build.xml into Gradle and use old compile of servlets in Gradle
to be able to use/reuse Ant targets where task has not been implemented in Gradle build.
- use the import to include the compile of htroot as first important task

  ! it is possible that first build fails an compile of GitRevTask.jar !
  ! solution/workaround -> use "ant all" once to compile GitRevTask.jar !

- adjusted build.xml a little
   - split compile-core into compile-core and compile-htroot to have a target for htroot comp. only
   - set build-path to reuse Gradles build directory
   - (fix javadoc failure)

- changed the filtered-copy of yacyBuildProperties.java to ! the build path :-(
  as current (copy,delete,exclude) is complicated and not migration worthy,
  used simple/straigt forward approach (using a yacyBuildProperties.java.template file as copy source)
2022-01-18 20:00:55 +01:00
reger24
398b105781 Prevent that YaCy always starts with a exception message on none Apple systems
Perform try to access com.apple.eio.FileManager  only on none Win systems
2022-01-18 13:02:12 +01:00
Lukas Fülling
e8a00007f6 add setting for public facing port 2022-01-11 17:10:48 +01:00
Michael Peter Christen
d7b17d8935 fixed missing thread name revert after balancer waiting 2021-12-22 01:46:18 +01:00
Michael Peter Christen
bd3f2483a1 replaced url and date retrieval by only url retrieval
This should prevent that the search index is used for freshnes of the
index entry.
2021-12-20 16:23:05 +01:00
Michael Peter Christen
163ba26d90 replaced check for load time method
instead of loading the solr document, an index only for the last loading
time was created. This prevents that solr has to fetch from its index
while the index is created. Excessive re-loading of documents while
indexing has shown to produce deadlocks, so this should now be
prevented.
2021-12-20 03:47:56 +01:00
Michael Peter Christen
1ead7b85b5 remove compiler warning
"warning: [try] explicit call to close() on an auto-closeable resource"
2021-12-13 12:28:34 +01:00
Michael Peter Christen
59777010dc Merge branch 'master' of git@github.com:yacy/yacy_search_server.git 2021-11-18 00:49:56 +01:00
Michael Peter Christen
7898815c41 disabling concurrent logging
(maybe temporary)
2021-11-18 00:49:46 +01:00
sgaebel
4bf6954474 uses clientBuilder not HttpClients.custom() to have these inside the
Pool too
2021-10-31 23:06:33 +01:00
sgaebel
cdf901270c always use HTTPClient by 'try with resources' pattern to free up
resources
2021-10-31 23:06:23 +01:00
sgaebel
69adaa9f55 makes our HTTPClient closable 2021-10-31 23:06:02 +01:00
sgaebel
fc4275f901 handle all references for client, response, request to be able to close
them
2021-10-31 23:05:50 +01:00
sgaebel
e7d3a363f2 refactor to use finish() 2021-10-31 11:22:35 +01:00
sgaebel
4fc876f4a3 revert back to use EntityUtils.consumeQuietly - as it simply closes the
underlying stream
2021-10-31 11:22:28 +01:00
sgaebel
4f0392e93e refactor use of AuthSchemeProvider 2021-10-31 11:21:59 +01:00
sgaebel
b74f337859 removes double setting of UserAgent 2021-10-31 11:21:06 +01:00
sgaebel
965748fefb some refactoring using try with resources 2021-10-31 11:20:28 +01:00
Michael Peter Christen
552ab7051b fix for warc importer 2021-10-25 19:35:15 +02:00
Michael Peter Christen
3c86b7b780 attempt to make a Mac Release using gradle
This is almost working with many workarounds:
- run rm lib/yacycore.jar
- run ./gradlew clean build bundleNative
- run ant clean all
- run again rm lib/yacycore.jar
- run ./fixMacBuild.sh

The build is then inside build/mac/YaCy.app

Right now this works so far but it does not have the correct release
number inside.

Target is to make this working for Windows releases and to embedd jre
entirely.
2021-10-25 18:37:39 +02:00
Michael Peter Christen
999c819e3e Merge branch 'master' of https://github.com/yacy/yacy_search_server.git 2021-10-24 20:50:14 +02:00
Michael Peter Christen
fd770e90e2 spike to identify paths for YaCy within mac application bundles 2021-10-24 20:49:59 +02:00