Commit Graph

14376 Commits

Author SHA1 Message Date
reger24
3996fb0920 Config Gradle standard distZip/distTar to use legacy archive file name
distTar is configured to use tar+gzip compression and creates now a archive named like  yacy_v1.925_20220125_10199.tar.gz  or distZip like yacy_v1.925_20220125_10199.zip
The legacy migration from Ant  -> gradle task packageDist  creates the same archive names.

but NOW in directory build/distributions/legacyDistFiles
to be able to use both in parallel (until knowing which one works better)
2022-01-25 16:19:46 +01:00
reger24
68e5c41385 Fix for upgrade to Gradle 7.3.3
see comment https://github.com/yacy/yacy_search_server/issues/377#issuecomment-1021068633
2022-01-25 14:11:24 +01:00
reger24
475e1c7907 Fix build error for IntelliJ after fake sourceSet htroot has been add ( dfc7a23e60 )
commit dfc7a23e60  uncomments the fake additional Gradle sourceSet (to make htroot in IDE visible), resulting in build error in IntelliJ IDEA if one uses just the IDE build button (and the old Ant build.xml is not available ... what is to expect soon).
This button runs as to expect the default Gradle task
   > Task :htrootClasses    (not our task compileHtrootServlets)
result :
htroot\processing\domaingraph\applet\domaingraph.java:1: error: package processing.core does not exist
import processing.core.*; import traer.physics.*; import traer.animation.*; import processing.net.*; import java.applet.*; import java.awt.*; import java.awt.image.*; import java.awt.event.*; import java.io.*; import java.net.*; import java.text.*; import java.util.*; import java.util.zip.*; public class domaingraph extends PApplet {// Domain visualization graph for YaCy
2022-01-25 13:18:02 +01:00
Burkhard
459dfae4c0
add: To start developing YaCy in IntelliJ IDEA 2022-01-25 12:07:52 +01:00
Michael Christen
6c6cc28c13
Merge pull request #443 from WeNeedCoffee/gradle-idea-patch
Idea project files by Gradle not ANT
2022-01-25 11:50:49 +01:00
reger24
65034be1ee Fix IntelliJ IDEA prime build error
test case
- install IntelliJ IDEA
   - New -> Project from Version Control     .....  and clone from github
- IntelliJ   Infobox  "Gradle build script found" -> "Load Gradle Project"
- in UI  execute task build
result: build error in compileHtrootServlets - (which would require manual interaction to get rid of)

reason IntelliJ apparently creates build directory before run build.gradle or configured as gradle project

solution: use other check condition for compileHtrootServlets to run
2022-01-25 09:31:42 +01:00
Daleth Darko
e673c3b57f removed redundant line from gitignore 2022-01-25 07:40:07 +01:00
Daleth Darko
dfc7a23e60 Idea project files by Gradle not ANT
As per the comment by @reger24
2022-01-25 07:31:51 +01:00
reger24
c49e9039e8 Prevent build script failing on prime build with fresh git clone
reason of failure was task compileHtrootServlets executing during gradle init
and javac resulting in error as not dependencies available in build dir

- happened only on very first build with with a fresh cloned repository
2022-01-25 03:46:35 +01:00
Michael Christen
def68aeafa
Merge pull request #442 from WeNeedCoffee/idea-patch
Idea project files
2022-01-24 23:18:41 +01:00
Daleth Darko
f93c947c0d Idea project files 2022-01-24 18:27:35 +01:00
reger24
eae16287e9 Added epub (ebook) format to existing zipParser
*.epub files are zip files containing xhtml files with content and other artifact files,
which the zipParser can  already feed to index
- extension "epub"
- mime "epub+zip"
2022-01-24 13:51:27 +01:00
reger24
a751369246 Further cleaned out transient dependencies + optimizations
- updated some jar's (as Gradle build relies only on one jar in lib dir)
	- guava-31.0.1-jre.jar
	- jsoup-1.14.3.jar
	- pdfbox-2.0.25.jar
- removed the not overwrite loop in copyDependenciesForDistribution as target dir is buildDir (not lib)
2022-01-24 10:37:11 +01:00
reger24
06cd77f7bc Clean up some transient dependencies
- not needed for Gradle build
one reason to love Maven and Gradle handles it too.
2022-01-23 07:58:33 +01:00
reger24
fa2fd37a48 Added NSIS plugin to Gradle build create Win install executable
found this nice NSIS plugin which doesn't require local NSIS installation
allowing to implement similar function as in old Ant build
using the existing build.nsi

- task distWinInstaller
- resulting *.exe  in build/distributions
2022-01-23 01:48:02 +01:00
reger24
6dcb58a4b5 Configured Gradle javadoc, added distribution archive for portalsearch
- in case it's really used separately

with that most Ant build targets are available native in Gradle build (imho all targets we  really need - see list of legacy Ant targets)

- "readBuildProperties" = see no use
- "init" = not needed (ant internal prop initialization)
- "javadoc" = gradle-task javadoc
- "compile-core" = gradle-task build
- "compile-htroot" = gradle-task compileHtrootServlets
- "compile" = gradle-task build
- "all" = gradle-task build
- "copyMain4Dist" = gradle-task packageDist
- "compileTest" = gradle-task compileTestJava (std)
- "test" = gradle-task test (std)
- "dist" = gradle-task packageDist
- "portalsearch" = gradle-task packagePortalsearch (is that file still used?)
- "clean" = gradle-task clean (std)
- "run" = gradle-task run (std) or shell
- "run-single" = not needed (old stuff)
- "debug" = won't gradleize ide or run
- "stop" = just a shell script
- "create-doc" = gradle-task javadoc
- "distWinInstaller" = won't impl. external tool needed
- "distMacApp" = won't impl. external tool needed
- "deleteData" = hm, won't impl. use nomale system
2022-01-22 10:56:11 +01:00
reger24
2e177661c9 Implement creation of legacy distribution archive in Gradle
- Gradle task to execute "packageDist"
	- this will create *.tar.gz and *.zip archive
	- with legacy file name convention
- archive is created in build/distributions

- for the libbuild J7Zip-modified.jar use fallback as file dependency in lib
  as Gradle/Groovy use internal just the classes from subprojects with result
  that in above no jar is included in lib but a bunch of *.class files
- added also the option to publish J7Zip-modified.jar to mavenLocal() repo.
2022-01-22 06:27:14 +01:00
reger24
4df29edd03 Implement the Ant javac task to compile htroot direct in Gradle
- import of the build.xml not longer needed and removed
2022-01-21 07:42:47 +01:00
reger24
eb69d535a3 Set all dependencies in Gradle build script
- with currently used version
- updated support/subproject J7Zip-modified (in libbuild) to be compilable via Gradle
	- added dependency reference as sub-project
- skip the before applied copy of jars to lib directory (no benefit)
2022-01-21 04:53:12 +01:00
reger24
a34d514b4f Fix naming mismatch of build.xml and jar in ./lib
version part differs (dot)
build.xml snippet
      <pathelement location="${lib}/bcmail-jdk15on-1.69.jar" />
      <pathelement location="${lib}/bcpkix-jdk15on-1.69.jar" />
      <pathelement location="${lib}/bcprov-jdk15on-1.69.jar" />
2022-01-20 10:41:53 +01:00
reger24
e39bb5fbab Modified GitRevNr Ant-Plugin to work in Gradle
- new support library GitComInf to generate a property file for use in Gradle
- adjusted build.gradle to make use of it to create the yacyBuildProperties.java
   - including additon of properties ext.filterTokens as store for Gradle
- added gradle-compile-htroot (w/o depends) to old build.xml as target until complete migration
  as it is imho important

imho: much to complicated to get a running number - should be supplied by repo or simplified
2022-01-20 09:59:34 +01:00
reger24
3e34f7c596 Import Ant build.xml into Gradle and use old compile of servlets in Gradle
to be able to use/reuse Ant targets where task has not been implemented in Gradle build.
- use the import to include the compile of htroot as first important task

  ! it is possible that first build fails an compile of GitRevTask.jar !
  ! solution/workaround -> use "ant all" once to compile GitRevTask.jar !

- adjusted build.xml a little
   - split compile-core into compile-core and compile-htroot to have a target for htroot comp. only
   - set build-path to reuse Gradles build directory
   - (fix javadoc failure)

- changed the filtered-copy of yacyBuildProperties.java to ! the build path :-(
  as current (copy,delete,exclude) is complicated and not migration worthy,
  used simple/straigt forward approach (using a yacyBuildProperties.java.template file as copy source)
2022-01-18 20:00:55 +01:00
reger24
426c47013c Prepare to use Gradle dependencies while mimic current config
with jars/dependencies in lib directory (w/o overwriting existing jars)
2022-01-18 16:02:19 +01:00
reger24
398b105781 Prevent that YaCy always starts with a exception message on none Apple systems
Perform try to access com.apple.eio.FileManager  only on none Win systems
2022-01-18 13:02:12 +01:00
reger24
102277e946 Merge branch 'master' of https://github.com/yacy/yacy_search_server 2022-01-18 12:03:46 +01:00
reger
6a8d351df3 Enable Gradle test and add manifest to yacycore.jar, set outputdir to lib
as part of migration from Ant to Gradle builds.
2022-01-18 11:55:21 +01:00
Burkhard
46cb485e3e
Update README.md
removed the obsolete "Build with Maven" paragraph
2022-01-18 11:45:43 +01:00
reger24
d411566c45 Obsolete since replacement of Maven by Gradle build script 2022-01-16 05:36:24 +01:00
Lukas Fülling
e8a00007f6 add setting for public facing port 2022-01-11 17:10:48 +01:00
Michael Christen
c94b9b8197
Merge pull request #439 from otteresk/master
Adding sort functionality to Crawler Queue tables
2022-01-10 01:08:56 +01:00
Andreas
590f39b403
Add Sorting functionality to Crawler Queue Table
Allow to sort for count and host
2022-01-09 16:06:14 +01:00
Andreas
41e87a44bc
Merge pull request #6 from yacy/master
Update fork #6
2022-01-09 15:59:03 +01:00
Michael Christen
7abfeb221b
Merge pull request #436 from ZeroCool940711/master
Improved the Image search page to have bigger thumbnails, use a more area for results and have a smaller sidebar.
2021-12-28 02:55:01 +01:00
lifeofguenter
870319e769
Fix typo + remove dead seeds 2021-12-27 14:12:17 +01:00
ZeroCool940711
7e765b8483 Improved the Image search page to have bigger thumbnails, use a bigger area for results and a smaller left sidebar. 2021-12-26 23:41:04 -07:00
Michael Peter Christen
6fe905bb82 feature https://github.com/yacy/yacy_search_server/issues/434 2021-12-26 23:33:31 +01:00
Michael Peter Christen
d7b17d8935 fixed missing thread name revert after balancer waiting 2021-12-22 01:46:18 +01:00
Michael Peter Christen
9c38b1254e proper deletion of loadtime index 2021-12-22 01:22:46 +01:00
Michael Peter Christen
bd3f2483a1 replaced url and date retrieval by only url retrieval
This should prevent that the search index is used for freshnes of the
index entry.
2021-12-20 16:23:05 +01:00
Michael Peter Christen
163ba26d90 replaced check for load time method
instead of loading the solr document, an index only for the last loading
time was created. This prevents that solr has to fetch from its index
while the index is created. Excessive re-loading of documents while
indexing has shown to produce deadlocks, so this should now be
prevented.
2021-12-20 03:47:56 +01:00
Michael Peter Christen
1ead7b85b5 remove compiler warning
"warning: [try] explicit call to close() on an auto-closeable resource"
2021-12-13 12:28:34 +01:00
Michael Peter Christen
3dc6613096 updating slf4j 1.7.25 -> 1.7.32 2021-12-13 12:26:49 +01:00
Michael Christen
cd0ff48e99
there is no (more) log4j in YaCy 2021-12-12 13:53:19 +01:00
Michael Peter Christen
59777010dc Merge branch 'master' of git@github.com:yacy/yacy_search_server.git 2021-11-18 00:49:56 +01:00
Michael Peter Christen
7898815c41 disabling concurrent logging
(maybe temporary)
2021-11-18 00:49:46 +01:00
sgaebel
4bf6954474 uses clientBuilder not HttpClients.custom() to have these inside the
Pool too
2021-10-31 23:06:33 +01:00
sgaebel
cdf901270c always use HTTPClient by 'try with resources' pattern to free up
resources
2021-10-31 23:06:23 +01:00
sgaebel
69adaa9f55 makes our HTTPClient closable 2021-10-31 23:06:02 +01:00
sgaebel
fc4275f901 handle all references for client, response, request to be able to close
them
2021-10-31 23:05:50 +01:00
sgaebel
1cdc55a425 lets SOLR merge bigger segments (up to 50GB)
+ some setting to reduce caches
2021-10-31 11:33:42 +01:00