Commit Graph

14310 Commits

Author SHA1 Message Date
reger24
2e177661c9 Implement creation of legacy distribution archive in Gradle
- Gradle task to execute "packageDist"
	- this will create *.tar.gz and *.zip archive
	- with legacy file name convention
- archive is created in build/distributions

- for the libbuild J7Zip-modified.jar use fallback as file dependency in lib
  as Gradle/Groovy use internal just the classes from subprojects with result
  that in above no jar is included in lib but a bunch of *.class files
- added also the option to publish J7Zip-modified.jar to mavenLocal() repo.
2022-01-22 06:27:14 +01:00
reger24
4df29edd03 Implement the Ant javac task to compile htroot direct in Gradle
- import of the build.xml not longer needed and removed
2022-01-21 07:42:47 +01:00
reger24
eb69d535a3 Set all dependencies in Gradle build script
- with currently used version
- updated support/subproject J7Zip-modified (in libbuild) to be compilable via Gradle
	- added dependency reference as sub-project
- skip the before applied copy of jars to lib directory (no benefit)
2022-01-21 04:53:12 +01:00
reger24
a34d514b4f Fix naming mismatch of build.xml and jar in ./lib
version part differs (dot)
build.xml snippet
      <pathelement location="${lib}/bcmail-jdk15on-1.69.jar" />
      <pathelement location="${lib}/bcpkix-jdk15on-1.69.jar" />
      <pathelement location="${lib}/bcprov-jdk15on-1.69.jar" />
2022-01-20 10:41:53 +01:00
reger24
e39bb5fbab Modified GitRevNr Ant-Plugin to work in Gradle
- new support library GitComInf to generate a property file for use in Gradle
- adjusted build.gradle to make use of it to create the yacyBuildProperties.java
   - including additon of properties ext.filterTokens as store for Gradle
- added gradle-compile-htroot (w/o depends) to old build.xml as target until complete migration
  as it is imho important

imho: much to complicated to get a running number - should be supplied by repo or simplified
2022-01-20 09:59:34 +01:00
reger24
3e34f7c596 Import Ant build.xml into Gradle and use old compile of servlets in Gradle
to be able to use/reuse Ant targets where task has not been implemented in Gradle build.
- use the import to include the compile of htroot as first important task

  ! it is possible that first build fails an compile of GitRevTask.jar !
  ! solution/workaround -> use "ant all" once to compile GitRevTask.jar !

- adjusted build.xml a little
   - split compile-core into compile-core and compile-htroot to have a target for htroot comp. only
   - set build-path to reuse Gradles build directory
   - (fix javadoc failure)

- changed the filtered-copy of yacyBuildProperties.java to ! the build path :-(
  as current (copy,delete,exclude) is complicated and not migration worthy,
  used simple/straigt forward approach (using a yacyBuildProperties.java.template file as copy source)
2022-01-18 20:00:55 +01:00
reger24
426c47013c Prepare to use Gradle dependencies while mimic current config
with jars/dependencies in lib directory (w/o overwriting existing jars)
2022-01-18 16:02:19 +01:00
reger24
398b105781 Prevent that YaCy always starts with a exception message on none Apple systems
Perform try to access com.apple.eio.FileManager  only on none Win systems
2022-01-18 13:02:12 +01:00
reger24
102277e946 Merge branch 'master' of https://github.com/yacy/yacy_search_server 2022-01-18 12:03:46 +01:00
reger
6a8d351df3 Enable Gradle test and add manifest to yacycore.jar, set outputdir to lib
as part of migration from Ant to Gradle builds.
2022-01-18 11:55:21 +01:00
Burkhard
46cb485e3e
Update README.md
removed the obsolete "Build with Maven" paragraph
2022-01-18 11:45:43 +01:00
reger24
d411566c45 Obsolete since replacement of Maven by Gradle build script 2022-01-16 05:36:24 +01:00
Lukas Fülling
e8a00007f6 add setting for public facing port 2022-01-11 17:10:48 +01:00
Michael Christen
c94b9b8197
Merge pull request #439 from otteresk/master
Adding sort functionality to Crawler Queue tables
2022-01-10 01:08:56 +01:00
Andreas
590f39b403
Add Sorting functionality to Crawler Queue Table
Allow to sort for count and host
2022-01-09 16:06:14 +01:00
Andreas
41e87a44bc
Merge pull request #6 from yacy/master
Update fork #6
2022-01-09 15:59:03 +01:00
Michael Christen
7abfeb221b
Merge pull request #436 from ZeroCool940711/master
Improved the Image search page to have bigger thumbnails, use a more area for results and have a smaller sidebar.
2021-12-28 02:55:01 +01:00
lifeofguenter
870319e769
Fix typo + remove dead seeds 2021-12-27 14:12:17 +01:00
ZeroCool940711
7e765b8483 Improved the Image search page to have bigger thumbnails, use a bigger area for results and a smaller left sidebar. 2021-12-26 23:41:04 -07:00
Michael Peter Christen
6fe905bb82 feature https://github.com/yacy/yacy_search_server/issues/434 2021-12-26 23:33:31 +01:00
Michael Peter Christen
d7b17d8935 fixed missing thread name revert after balancer waiting 2021-12-22 01:46:18 +01:00
Michael Peter Christen
9c38b1254e proper deletion of loadtime index 2021-12-22 01:22:46 +01:00
Michael Peter Christen
bd3f2483a1 replaced url and date retrieval by only url retrieval
This should prevent that the search index is used for freshnes of the
index entry.
2021-12-20 16:23:05 +01:00
Michael Peter Christen
163ba26d90 replaced check for load time method
instead of loading the solr document, an index only for the last loading
time was created. This prevents that solr has to fetch from its index
while the index is created. Excessive re-loading of documents while
indexing has shown to produce deadlocks, so this should now be
prevented.
2021-12-20 03:47:56 +01:00
Michael Peter Christen
1ead7b85b5 remove compiler warning
"warning: [try] explicit call to close() on an auto-closeable resource"
2021-12-13 12:28:34 +01:00
Michael Peter Christen
3dc6613096 updating slf4j 1.7.25 -> 1.7.32 2021-12-13 12:26:49 +01:00
Michael Christen
cd0ff48e99
there is no (more) log4j in YaCy 2021-12-12 13:53:19 +01:00
Michael Peter Christen
59777010dc Merge branch 'master' of git@github.com:yacy/yacy_search_server.git 2021-11-18 00:49:56 +01:00
Michael Peter Christen
7898815c41 disabling concurrent logging
(maybe temporary)
2021-11-18 00:49:46 +01:00
sgaebel
4bf6954474 uses clientBuilder not HttpClients.custom() to have these inside the
Pool too
2021-10-31 23:06:33 +01:00
sgaebel
cdf901270c always use HTTPClient by 'try with resources' pattern to free up
resources
2021-10-31 23:06:23 +01:00
sgaebel
69adaa9f55 makes our HTTPClient closable 2021-10-31 23:06:02 +01:00
sgaebel
fc4275f901 handle all references for client, response, request to be able to close
them
2021-10-31 23:05:50 +01:00
sgaebel
1cdc55a425 lets SOLR merge bigger segments (up to 50GB)
+ some setting to reduce caches
2021-10-31 11:33:42 +01:00
sgaebel
e7d3a363f2 refactor to use finish() 2021-10-31 11:22:35 +01:00
sgaebel
4fc876f4a3 revert back to use EntityUtils.consumeQuietly - as it simply closes the
underlying stream
2021-10-31 11:22:28 +01:00
sgaebel
4f0392e93e refactor use of AuthSchemeProvider 2021-10-31 11:21:59 +01:00
sgaebel
b74f337859 removes double setting of UserAgent 2021-10-31 11:21:06 +01:00
sgaebel
965748fefb some refactoring using try with resources 2021-10-31 11:20:28 +01:00
Michael Christen
f4834e8e31
link fix 2021-10-29 11:10:23 +02:00
Michael Christen
7f5d3e3a12
fixed name 2021-10-29 11:07:34 +02:00
Michael Peter Christen
552ab7051b fix for warc importer 2021-10-25 19:35:15 +02:00
Michael Peter Christen
3c86b7b780 attempt to make a Mac Release using gradle
This is almost working with many workarounds:
- run rm lib/yacycore.jar
- run ./gradlew clean build bundleNative
- run ant clean all
- run again rm lib/yacycore.jar
- run ./fixMacBuild.sh

The build is then inside build/mac/YaCy.app

Right now this works so far but it does not have the correct release
number inside.

Target is to make this working for Windows releases and to embedd jre
entirely.
2021-10-25 18:37:39 +02:00
Michael Peter Christen
49cae8ca62 network bootstraping addresses update 2021-10-25 18:32:57 +02:00
Michael Peter Christen
8e4383c49e downgrading gradle to 6.9
to be able to support org.mini2Dx.parcl
2021-10-25 18:32:34 +02:00
Michael Peter Christen
999c819e3e Merge branch 'master' of https://github.com/yacy/yacy_search_server.git 2021-10-24 20:50:14 +02:00
Michael Peter Christen
fd770e90e2 spike to identify paths for YaCy within mac application bundles 2021-10-24 20:49:59 +02:00
Michael Peter Christen
d19872fd26 making sure that crawl queues are closed correctly to prevent data loss 2021-10-14 00:30:04 +02:00
sgaebel
90507c0fdc comments out printing query params to std.out 2021-10-04 18:03:06 +02:00
Michael Peter Christen
be0aebad84 fixes https://github.com/yacy/yacy_search_server/issues/424 2021-10-04 14:38:49 +02:00