Commit Graph

14270 Commits

Author SHA1 Message Date
Daleth Darko
e673c3b57f removed redundant line from gitignore 2022-01-25 07:40:07 +01:00
Daleth Darko
dfc7a23e60 Idea project files by Gradle not ANT
As per the comment by @reger24
2022-01-25 07:31:51 +01:00
reger24
c49e9039e8 Prevent build script failing on prime build with fresh git clone
reason of failure was task compileHtrootServlets executing during gradle init
and javac resulting in error as not dependencies available in build dir

- happened only on very first build with with a fresh cloned repository
2022-01-25 03:46:35 +01:00
Michael Christen
def68aeafa
Merge pull request #442 from WeNeedCoffee/idea-patch
Idea project files
2022-01-24 23:18:41 +01:00
Daleth Darko
f93c947c0d Idea project files 2022-01-24 18:27:35 +01:00
reger24
eae16287e9 Added epub (ebook) format to existing zipParser
*.epub files are zip files containing xhtml files with content and other artifact files,
which the zipParser can  already feed to index
- extension "epub"
- mime "epub+zip"
2022-01-24 13:51:27 +01:00
reger24
a751369246 Further cleaned out transient dependencies + optimizations
- updated some jar's (as Gradle build relies only on one jar in lib dir)
	- guava-31.0.1-jre.jar
	- jsoup-1.14.3.jar
	- pdfbox-2.0.25.jar
- removed the not overwrite loop in copyDependenciesForDistribution as target dir is buildDir (not lib)
2022-01-24 10:37:11 +01:00
reger24
06cd77f7bc Clean up some transient dependencies
- not needed for Gradle build
one reason to love Maven and Gradle handles it too.
2022-01-23 07:58:33 +01:00
reger24
fa2fd37a48 Added NSIS plugin to Gradle build create Win install executable
found this nice NSIS plugin which doesn't require local NSIS installation
allowing to implement similar function as in old Ant build
using the existing build.nsi

- task distWinInstaller
- resulting *.exe  in build/distributions
2022-01-23 01:48:02 +01:00
reger24
6dcb58a4b5 Configured Gradle javadoc, added distribution archive for portalsearch
- in case it's really used separately

with that most Ant build targets are available native in Gradle build (imho all targets we  really need - see list of legacy Ant targets)

- "readBuildProperties" = see no use
- "init" = not needed (ant internal prop initialization)
- "javadoc" = gradle-task javadoc
- "compile-core" = gradle-task build
- "compile-htroot" = gradle-task compileHtrootServlets
- "compile" = gradle-task build
- "all" = gradle-task build
- "copyMain4Dist" = gradle-task packageDist
- "compileTest" = gradle-task compileTestJava (std)
- "test" = gradle-task test (std)
- "dist" = gradle-task packageDist
- "portalsearch" = gradle-task packagePortalsearch (is that file still used?)
- "clean" = gradle-task clean (std)
- "run" = gradle-task run (std) or shell
- "run-single" = not needed (old stuff)
- "debug" = won't gradleize ide or run
- "stop" = just a shell script
- "create-doc" = gradle-task javadoc
- "distWinInstaller" = won't impl. external tool needed
- "distMacApp" = won't impl. external tool needed
- "deleteData" = hm, won't impl. use nomale system
2022-01-22 10:56:11 +01:00
reger24
2e177661c9 Implement creation of legacy distribution archive in Gradle
- Gradle task to execute "packageDist"
	- this will create *.tar.gz and *.zip archive
	- with legacy file name convention
- archive is created in build/distributions

- for the libbuild J7Zip-modified.jar use fallback as file dependency in lib
  as Gradle/Groovy use internal just the classes from subprojects with result
  that in above no jar is included in lib but a bunch of *.class files
- added also the option to publish J7Zip-modified.jar to mavenLocal() repo.
2022-01-22 06:27:14 +01:00
reger24
4df29edd03 Implement the Ant javac task to compile htroot direct in Gradle
- import of the build.xml not longer needed and removed
2022-01-21 07:42:47 +01:00
reger24
eb69d535a3 Set all dependencies in Gradle build script
- with currently used version
- updated support/subproject J7Zip-modified (in libbuild) to be compilable via Gradle
	- added dependency reference as sub-project
- skip the before applied copy of jars to lib directory (no benefit)
2022-01-21 04:53:12 +01:00
reger24
a34d514b4f Fix naming mismatch of build.xml and jar in ./lib
version part differs (dot)
build.xml snippet
      <pathelement location="${lib}/bcmail-jdk15on-1.69.jar" />
      <pathelement location="${lib}/bcpkix-jdk15on-1.69.jar" />
      <pathelement location="${lib}/bcprov-jdk15on-1.69.jar" />
2022-01-20 10:41:53 +01:00
reger24
e39bb5fbab Modified GitRevNr Ant-Plugin to work in Gradle
- new support library GitComInf to generate a property file for use in Gradle
- adjusted build.gradle to make use of it to create the yacyBuildProperties.java
   - including additon of properties ext.filterTokens as store for Gradle
- added gradle-compile-htroot (w/o depends) to old build.xml as target until complete migration
  as it is imho important

imho: much to complicated to get a running number - should be supplied by repo or simplified
2022-01-20 09:59:34 +01:00
reger24
3e34f7c596 Import Ant build.xml into Gradle and use old compile of servlets in Gradle
to be able to use/reuse Ant targets where task has not been implemented in Gradle build.
- use the import to include the compile of htroot as first important task

  ! it is possible that first build fails an compile of GitRevTask.jar !
  ! solution/workaround -> use "ant all" once to compile GitRevTask.jar !

- adjusted build.xml a little
   - split compile-core into compile-core and compile-htroot to have a target for htroot comp. only
   - set build-path to reuse Gradles build directory
   - (fix javadoc failure)

- changed the filtered-copy of yacyBuildProperties.java to ! the build path :-(
  as current (copy,delete,exclude) is complicated and not migration worthy,
  used simple/straigt forward approach (using a yacyBuildProperties.java.template file as copy source)
2022-01-18 20:00:55 +01:00
reger24
426c47013c Prepare to use Gradle dependencies while mimic current config
with jars/dependencies in lib directory (w/o overwriting existing jars)
2022-01-18 16:02:19 +01:00
reger24
398b105781 Prevent that YaCy always starts with a exception message on none Apple systems
Perform try to access com.apple.eio.FileManager  only on none Win systems
2022-01-18 13:02:12 +01:00
reger24
102277e946 Merge branch 'master' of https://github.com/yacy/yacy_search_server 2022-01-18 12:03:46 +01:00
reger
6a8d351df3 Enable Gradle test and add manifest to yacycore.jar, set outputdir to lib
as part of migration from Ant to Gradle builds.
2022-01-18 11:55:21 +01:00
Burkhard
46cb485e3e
Update README.md
removed the obsolete "Build with Maven" paragraph
2022-01-18 11:45:43 +01:00
reger24
d411566c45 Obsolete since replacement of Maven by Gradle build script 2022-01-16 05:36:24 +01:00
Lukas Fülling
e8a00007f6 add setting for public facing port 2022-01-11 17:10:48 +01:00
Michael Christen
c94b9b8197
Merge pull request #439 from otteresk/master
Adding sort functionality to Crawler Queue tables
2022-01-10 01:08:56 +01:00
Andreas
590f39b403
Add Sorting functionality to Crawler Queue Table
Allow to sort for count and host
2022-01-09 16:06:14 +01:00
Andreas
41e87a44bc
Merge pull request #6 from yacy/master
Update fork #6
2022-01-09 15:59:03 +01:00
Michael Christen
7abfeb221b
Merge pull request #436 from ZeroCool940711/master
Improved the Image search page to have bigger thumbnails, use a more area for results and have a smaller sidebar.
2021-12-28 02:55:01 +01:00
lifeofguenter
870319e769
Fix typo + remove dead seeds 2021-12-27 14:12:17 +01:00
ZeroCool940711
7e765b8483 Improved the Image search page to have bigger thumbnails, use a bigger area for results and a smaller left sidebar. 2021-12-26 23:41:04 -07:00
Michael Peter Christen
6fe905bb82 feature https://github.com/yacy/yacy_search_server/issues/434 2021-12-26 23:33:31 +01:00
Michael Peter Christen
d7b17d8935 fixed missing thread name revert after balancer waiting 2021-12-22 01:46:18 +01:00
Michael Peter Christen
9c38b1254e proper deletion of loadtime index 2021-12-22 01:22:46 +01:00
Michael Peter Christen
bd3f2483a1 replaced url and date retrieval by only url retrieval
This should prevent that the search index is used for freshnes of the
index entry.
2021-12-20 16:23:05 +01:00
Michael Peter Christen
163ba26d90 replaced check for load time method
instead of loading the solr document, an index only for the last loading
time was created. This prevents that solr has to fetch from its index
while the index is created. Excessive re-loading of documents while
indexing has shown to produce deadlocks, so this should now be
prevented.
2021-12-20 03:47:56 +01:00
Michael Peter Christen
1ead7b85b5 remove compiler warning
"warning: [try] explicit call to close() on an auto-closeable resource"
2021-12-13 12:28:34 +01:00
Michael Peter Christen
3dc6613096 updating slf4j 1.7.25 -> 1.7.32 2021-12-13 12:26:49 +01:00
Michael Christen
cd0ff48e99
there is no (more) log4j in YaCy 2021-12-12 13:53:19 +01:00
Michael Peter Christen
59777010dc Merge branch 'master' of git@github.com:yacy/yacy_search_server.git 2021-11-18 00:49:56 +01:00
Michael Peter Christen
7898815c41 disabling concurrent logging
(maybe temporary)
2021-11-18 00:49:46 +01:00
sgaebel
4bf6954474 uses clientBuilder not HttpClients.custom() to have these inside the
Pool too
2021-10-31 23:06:33 +01:00
sgaebel
cdf901270c always use HTTPClient by 'try with resources' pattern to free up
resources
2021-10-31 23:06:23 +01:00
sgaebel
69adaa9f55 makes our HTTPClient closable 2021-10-31 23:06:02 +01:00
sgaebel
fc4275f901 handle all references for client, response, request to be able to close
them
2021-10-31 23:05:50 +01:00
sgaebel
1cdc55a425 lets SOLR merge bigger segments (up to 50GB)
+ some setting to reduce caches
2021-10-31 11:33:42 +01:00
sgaebel
e7d3a363f2 refactor to use finish() 2021-10-31 11:22:35 +01:00
sgaebel
4fc876f4a3 revert back to use EntityUtils.consumeQuietly - as it simply closes the
underlying stream
2021-10-31 11:22:28 +01:00
sgaebel
4f0392e93e refactor use of AuthSchemeProvider 2021-10-31 11:21:59 +01:00
sgaebel
b74f337859 removes double setting of UserAgent 2021-10-31 11:21:06 +01:00
sgaebel
965748fefb some refactoring using try with resources 2021-10-31 11:20:28 +01:00
Michael Christen
f4834e8e31
link fix 2021-10-29 11:10:23 +02:00