Commit Graph

20 Commits

Author SHA1 Message Date
orbiter
650ca3955a added flush-thread for index cache and added language-name mapping in Language_p
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@203 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-31 17:39:14 +00:00
orbiter
3d8a2ff937 enhanced parallelization of local/global/remote crawling
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@197 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-29 11:56:40 +00:00
orbiter
a05d738ea4 enhanced caching, removed bug causing outOfMemory
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@195 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-29 00:34:01 +00:00
orbiter
21110dcd5e fixed bugs with open files and caching
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@175 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-25 13:48:48 +00:00
orbiter
5f90daa265 implemented localization environment
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@171 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-24 14:23:04 +00:00
orbiter
fdd606c8c8 fixed bugs
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@168 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-23 23:50:35 +00:00
orbiter
0c35171c85 assortment fine-tuning
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@163 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-23 15:07:16 +00:00
orbiter
76dc892017 refined assortment
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@159 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-23 13:15:28 +00:00
orbiter
4b01ff7548 activated assortments, removed write-queues
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@151 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-22 23:59:52 +00:00
orbiter
e26ac60c3e modified assortment data structures
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@148 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-22 13:27:54 +00:00
orbiter
79be6f003d enhanced Assortment class
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@141 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-19 16:41:08 +00:00
orbiter
5c6147a54c introduced assortment structure (generalization of singletons)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@139 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-18 21:52:17 +00:00
theli
361f05978d Multiple updates regarding the yacy seedUpload facility,
optional content parsers, thread pool configuration ...

Please help me testing if everything works correct.

*) Migration of yacy seedUpload functionality
See: http://www.yacy-forum.de/viewtopic.php?t=256
- new uploaders can now be easily introduced because of a new modulare uploader system
- default uploaders are: none, file, ftp
- adding optional uploader for scp
- each uploader provides its own configuration file that will be 
  included into the settings page using the new template include feature
- Each uploader can define its libx dependencies. If not all needed libs are
  available, the uploader is deactivated automatically.

*) Migration of optional parsers
See: http://www.yacy-forum.de/viewtopic.php?t=198
- Parsers can now also define there libx dependencies
- adding parser for bzip compressed content
- adding parser for gzip compressed content
- adding parser for zip files
- adding parser for tar files
- adding parser to detect the mime-type of a file
  this is needed by the bzip/gzip Parser.java
- adding parser for rtf files
- removing extra configuration file yacy.parser
  the list of enabled parsers is now stored in the main config file

*) Adding configuration option in the performance dialog to configure
See: http://www.yacy-forum.de/viewtopic.php?t=267
- maxActive / maxIdle / minIdle values for httpd-session-threadpool
- maxActive / maxIdle / minIdle values for crawler-threadpool

*) Changing Crawling Filter behaviour
See: http://www.yacy-forum.de/viewtopic.php?p=2631

*) Replacing some hardcoded strings with the proper constants of the httpHeader class

*) Adding new libs to libx directory. This libs are
- needed by new content parsers
- needed by new optional seed uploader
- needed by SOAP API (which will be committed later)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@126 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-17 08:25:04 +00:00
orbiter
287d2e6f10 further enhanced caching (new cache flush methods)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@111 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-12 23:33:48 +00:00
orbiter
376b917c91 fixed shut-down by stopYACY.sh
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@108 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-12 10:05:17 +00:00
orbiter
ea478f3975 enhanced indexing-caching
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@107 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-11 23:42:40 +00:00
orbiter
10a4a2741d fixed missing close
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@94 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-08 20:39:11 +00:00
orbiter
db1da3345d introduced singleton-database
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@92 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-08 14:42:09 +00:00
orbiter
a9b22647dc fixed bug in indexDump.stack - generation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@88 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-08 00:11:59 +00:00
orbiter
1d7fed87dc redesign of index caching - removed indexCache.db
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@86 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-07 21:11:18 +00:00