Commit Graph

239 Commits

Author SHA1 Message Date
orbiter
be1f324fca performance setting for remote indexing configuration and latest changes for 0.39
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@424 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-22 13:56:19 +00:00
rramthun
9a7c4080d5 Fix for libx compression.
See http://www.yacy-forum.de/viewtopic.php?p=4922#4922

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@383 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-06 17:46:52 +00:00
theli
2d1ccae406 *) bugfix for svn revision number detection
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@379 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-05 09:31:23 +00:00
theli
b90f8e128a *) bugfix for svn revision number detection
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@378 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-05 09:24:14 +00:00
theli
3cd9a1f4bd *) correcting problems with wrong svn revision number
- if the .svn directory exists, the entries file will be
  parsed to determine the revision number of the corrent
  working copy
- then the revision number in the build.properties file
  will be overwritten with this value
- if the .svn directory does not exists the value from
  the build.properties file will be used

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@376 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-05 09:00:35 +00:00
orbiter
68dc2b0c6b added kelondroArray, the basis for upcoming kelondroHash and some bug fixes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@311 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-21 01:17:25 +00:00
orbiter
a25b5b4986 fixed possible memory leak in htmlScraper: be aware that now links can get lost; further work necessary
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@288 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-16 18:31:28 +00:00
theli
9a98988c3c *) Bugfix for SSL/NIO Bug
See: http://www.yacy-forum.de/viewtopic.php?t=516
   - removing NIO from server/serverCore.java because of massive problems
     with socket close issues
*) Adding support for remote port forwarding via sch
   @Orbiter: Please take a look into
   - hello.java
   - server/serverCore.java.publicIP()
   - yacy/yacyClient.java.publishMySeed(...)
*) Making startup loading of additional content parsers more failsafe


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@281 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-16 07:28:07 +00:00
orbiter
a5b40923b6 added word migration to assortments (start with 'java -classpath classes yacy -migratewords')
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@278 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-15 01:22:07 +00:00
theli
bd79c43b0b *) Correcting problems with run target
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@275 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-13 13:32:30 +00:00
orbiter
beb652847e added run-target for ant (needed for NetBeans4)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@274 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-13 12:01:58 +00:00
theli
6dd3ec0dc4 *) Adding debug="true" debuglevel="lines,vars,source" to ant build files
See: http://www.yacy-forum.de/viewtopic.php?p=4099


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@270 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-12 05:22:06 +00:00
theli
9e4bcc828d *) solving problems with wrong classpath
see: http://www.yacy-forum.de/viewtopic.php?p=4045

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@264 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-10 06:18:55 +00:00
orbiter
878ff0ae7b corrections
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@262 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 16:39:25 +00:00
orbiter
ee0758fe4d bugfixes/empty-dir-deletion/snippet-test-activation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@212 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-07 01:05:55 +00:00
orbiter
a12922536a latest changes for 0.38
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@209 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-03 09:22:50 +00:00
orbiter
594c591223 changes towards 0.38
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@208 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-03 02:43:35 +00:00
(no author)
c8b24569e3 *) solving problem with duplicated class files in htroot
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@207 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-02 06:58:20 +00:00
orbiter
d8fdc2526e added experimental snipplet-generation (to be disabled for 0.38)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@206 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-02 01:33:10 +00:00
allo
9c224ffe99 Execution Filerights for Unix Wrappers
added locales to the tar


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@181 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-27 08:59:01 +00:00
theli
84f9d8f7f0 *) migrating ant build files to generate a single extension tar per default
This is buildconfigureable by changing the extensionTarget Property in the build.properties file
*) Trying to solve "yacy.java template replacement / ant build failed" bug

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@169 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-24 09:02:04 +00:00
theli
8bd49ba535 *) setting root dir for all tar files properly
*) automatically adding SVN Revision number to tar file name
*) introducing build.properties file that can be used to set the build version number and date

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@164 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-23 15:47:08 +00:00
orbiter
e26ac60c3e modified assortment data structures
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@148 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-22 13:27:54 +00:00
theli
02bc33170a *) adding missing mkdir
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@129 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-17 09:34:02 +00:00
theli
361f05978d Multiple updates regarding the yacy seedUpload facility,
optional content parsers, thread pool configuration ...

Please help me testing if everything works correct.

*) Migration of yacy seedUpload functionality
See: http://www.yacy-forum.de/viewtopic.php?t=256
- new uploaders can now be easily introduced because of a new modulare uploader system
- default uploaders are: none, file, ftp
- adding optional uploader for scp
- each uploader provides its own configuration file that will be 
  included into the settings page using the new template include feature
- Each uploader can define its libx dependencies. If not all needed libs are
  available, the uploader is deactivated automatically.

*) Migration of optional parsers
See: http://www.yacy-forum.de/viewtopic.php?t=198
- Parsers can now also define there libx dependencies
- adding parser for bzip compressed content
- adding parser for gzip compressed content
- adding parser for zip files
- adding parser for tar files
- adding parser to detect the mime-type of a file
  this is needed by the bzip/gzip Parser.java
- adding parser for rtf files
- removing extra configuration file yacy.parser
  the list of enabled parsers is now stored in the main config file

*) Adding configuration option in the performance dialog to configure
See: http://www.yacy-forum.de/viewtopic.php?t=267
- maxActive / maxIdle / minIdle values for httpd-session-threadpool
- maxActive / maxIdle / minIdle values for crawler-threadpool

*) Changing Crawling Filter behaviour
See: http://www.yacy-forum.de/viewtopic.php?p=2631

*) Replacing some hardcoded strings with the proper constants of the httpHeader class

*) Adding new libs to libx directory. This libs are
- needed by new content parsers
- needed by new optional seed uploader
- needed by SOAP API (which will be committed later)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@126 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-17 08:25:04 +00:00
orbiter
287d2e6f10 further enhanced caching (new cache flush methods)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@111 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-12 23:33:48 +00:00
orbiter
0cfe94bb66 fixed last commit + added missing files
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@106 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-11 15:54:26 +00:00
allo
4b9982e2b2 ...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@93 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-08 14:55:49 +00:00
theli
1dad015b0b *) Migration of Ant build files
- now the ant build file has the same functionality as the makerelease build file
- from now on the ant build files can be used instead of the makerelease build script

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@84 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-06 08:43:11 +00:00
theli
351c86d5d9 *) Migration of optional Content Parser integration
- each additional parser must be in a subpackage 
  of plasma.parser
- each parser must have its own ant build file (which will 
  be called automatically from the main build file)
- Calling the main build file results in building a separate 
  zip file for each optional parser. This zip file includes:
  + sources of the Parser.java
  + compiled classes of the Parser.java
  + needed additional libs (libx)
- To install an additional parser the user simply needs to
  extract the zip file listed above into his/her yacy directory.
- The configuration (enabling/disabling) of a parser can be done
  via the webinterface (currently the settings dialoge) and is
  done "on-the-fly". The installation can not be done "on-the-fly"
  at the moment because of classpath issues.
- The classpath of the linux startup/stop scripts is generated 
  automatically now (including all libraries from lib and libx).

*) Bugfix: File Extension was not calculated correctly by the crawler
   e.g.: file extension was accidentally: .php?param=value
   Corrected.

*) Adding additional parser for parsing of rss/atom feeds
- added needed libs to do this.

TODO:
- automatic building classpath for windows startup scripts


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@78 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-03 09:47:56 +00:00
allo
1a4ad5a0ac updated the versionnummber ...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@77 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-02 19:16:45 +00:00
allo
4856f04797 set version
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@74 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-04-30 11:08:59 +00:00
allo
9cc4171d6d The Version is set in build.xml, too (@mc: update both places on new version)
The Target "all" Builds YaCy(like make) and the Target "dist" zips the addons(like make install)
(todo: tar.gz creation in the target)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@69 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-04-28 18:32:50 +00:00
theli
fdf206239a *) adding separate targets to build zip files for each optional content parser.
This archive file can then be extrcted to the yacy root to add the new yacy feature.

TODO: but how to solve the java classpath problem for the yacy startup scripts?

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@68 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-04-28 09:59:44 +00:00
theli
58b1a0ba40 *) adding an new package for extra content parsers
*) adding content parser for
- pdf (using the pdf-box library)
- doc (using the textmining.org library)
*) adding a Interface for content parsers
*) adding a configuration file which can be used to configure which parser is used for which mimeType
*) Sempahore class was moved and renamed to serverSemaphore
*) Changing yacy shutdown behaviour
Buzy waiting loop for shutdown was removed and replaced with a blocking call (using the semaphore class mentioned above) to the new switchboard.waitForShutdown method.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@46 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-04-24 21:24:53 +00:00
(no author)
f8d82cee47 *) Merging didn't work correctly. Fixed.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@27 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-04-19 07:00:13 +00:00
allo
c13411c198 Buildfile which inserts the Date.
The Version is set in the source; so it will be correct if you check old versions out.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@21 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-04-15 13:50:42 +00:00
(no author)
070758dbfa Adding missing compile statements for
- ${htroot}/htdocsdefault
- ${htroot}/yacy

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@19 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-04-14 08:05:51 +00:00
allo
97dad7eaf4 A Ant Buildfile for YaCY. It doesn't replace the REPL_VERSION and REPL_DATE Strings, yet.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@11 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-04-10 15:00:58 +00:00