Commit Graph

146 Commits

Author SHA1 Message Date
theli
49a23bb89c *) adding reference to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@257 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:57:42 +00:00
theli
02d43a7bd4 *) adding additional methods that should be implemented by all
serverHandler classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@256 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:54:03 +00:00
theli
652a19cf13 *) adding a method to read and zip a file (needed by the filehandler)
for doing gzip encoding

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@255 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:52:59 +00:00
theli
af9cd67334 *) migration to Java NIO
- to avoid buzy waiting and
   - make socket blocking interruptable
*) changing reference to logger
*) introduce commandObjMethodCache to improve performance
*) doing a stream shutdown before closing the connection
   to aviod problems when using persistent connections
*) calling method UNKNOWN of the server-handler class when 
   receiving an unknown command
*) calling method EMPTY of the server-handler class when
   receiving an empty command

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@254 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:51:56 +00:00
theli
1eff96f471 *) removing buzy waiting
*) changing reference to logger

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@253 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:47:50 +00:00
theli
c1a4e0dc28 *) changing reference to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@252 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:44:55 +00:00
theli
d0083f845f *) changing reference to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@251 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:39:09 +00:00
theli
1b5ae054f8 *) changing reference to logger
*) parser will not be returned into pool if the parser was deactivated
   via gui

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@250 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:38:00 +00:00
theli
68f30811fa *) changing reference to logger
*) bugfix in function getCachePath

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@249 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:36:39 +00:00
theli
fbbea813c5 *) changing references to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@248 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:34:20 +00:00
theli
451ca6b818 *) changing reference to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@247 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:25:39 +00:00
theli
4d7540e679 *) adding new constants
*) adding a list of default error messages which belong to each
   http status code

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@246 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:23:03 +00:00
theli
cb17ff4aa2 *) adding support of proxy access logging (much similar to squids
access.log) file
*) splitting doGet function in separate functions for fulfilling
   requests from cache and from web to make error handling easier
*) using connection property and httpHeader constants instead of
   hardcoded strings whenever possible
*) sending back a proxy error message as body of every respond
   containing a http error code
*) correcting problems of messages received from other proxies
   containing 204, 304 status codes.
*) using chunked transfer encoding if the server has not set the
   content length (e.g. because of gzip content encoding) but 
   the client has established a persistent connection to yacy.
   This is only possible for http/1.1 clients. For http/1.0 clients
   the connection will simply be closed on the end of the message.   
*) removing unneeded functions (e.g. respondError) because of newly
   introduced functions of httpd.java
*) removing hop by hop headers (according to rfc)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@245 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:22:05 +00:00
theli
f157181086 *) starting implementation of Content-MD5 header
which should help to detect transfer errors on yacy to yacy
   communication
   - not finished yet
*) removing unneeded functions (e.g. respondHeader) because newly
   introduced functions in class httpd.java
*) httpdFileHandler.java now always sends back a proxy error message
   as body of a response with an error code
*) adding support of gzip content encoding 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@244 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:12:07 +00:00
theli
d2042ec8fa *) adding constants for all needed connection properties
*) adding hashset for file-extensions that should not be transfered
   using gzip content encoding
*) doing bugfixes on old keep-alive implementation
*) doing some additional http header validation according to rfc
*) doing all persistent connection detection in separate function now
*) doing server authentication in separate function now
*) doing proxy authentication in separate function now
*) simplifying GET, POST, HEAD functions because of new introduced 
   functions listed above
*) adding new function to handle empty request lines (which could
   occure after post requests send via a persistent connection; 
   this depends on the used browser)
*) adding new function to handle unknown request methods by sending
   a correct error message back to the client
*) setting correct content-length when sending back error messages
   to the client
*) adding new functions that must be used by all http-Handler classes
   to send 
   - a proxy error message
   - a http header
   back to the client
*) adding new function: shallTransportZipped
   moved here from httpc

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@243 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:07:02 +00:00
theli
06b0db2cac *) adding toString method to
- httpc
   - response
*) simplifying gzip encoding
*) remembering http version of contacted server
   (neede for later support of keep alive by httpc)
*) moving function shallTransportZipped to httpd.java
   because this function is used multiple times

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@242 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:56:41 +00:00
theli
7b0b72dd23 *) adding new streams for
- implementation of outgoing chunked transfer encoding (httpChunkedOutputStream.java)
- byte counting for proxy access logging / global traffic count
  (httpdByteCount(In|Out)putStream.java

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@241 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:47:56 +00:00
theli
318dfc4636 *) activating new logging framework
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@240 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:46:43 +00:00
theli
aae3e3b7ab *) reusing buffer instead of creating a new one
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@239 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:45:58 +00:00
theli
829b65c1c8 *) adding additional classes needed for new logging
- ConsoleOutErrHandler.java used to log warnings/errors to stderr 
  and all other messages to stdout
- GuiHandler.java
  used to keep logging messages in memory that can then be viewed
  via the http gui
- serverSimpleLogFormatter.java
  needed to format logging messages for FileHandler, ConsoleOutErrHandler
  and GuiHandler
- serverMiniLogFormatter.java
  needed for proxy access logging

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@233 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:25:08 +00:00
low012
8c2789b22a to catch is an irregular verb
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@225 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 00:07:25 +00:00
orbiter
4574fa4ce7 bugfixes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@224 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-08 15:28:29 +00:00
theli
83b41ef2f7 *) Adding timeouts for shutdown
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@223 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-08 13:44:25 +00:00
theli
ef6851798b *) changing thread priority while parsing a pdf file to avoid 100% CPU usage.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@222 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-08 13:23:35 +00:00
orbiter
33f9315e58 implemented multithreading of indexing
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@221 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-08 13:19:05 +00:00
orbiter
ca3b4ccaf4 added snippet-routines (not yet finished)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@218 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-08 00:52:24 +00:00
theli
167ddd5612 *) Displaying "SVN xxx" instead of "@REPL_VERSION@" if yacy is started directly from SVN
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@213 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-07 08:31:49 +00:00
orbiter
ee0758fe4d bugfixes/empty-dir-deletion/snippet-test-activation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@212 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-07 01:05:55 +00:00
orbiter
594c591223 changes towards 0.38
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@208 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-03 02:43:35 +00:00
orbiter
d8fdc2526e added experimental snipplet-generation (to be disabled for 0.38)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@206 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-02 01:33:10 +00:00
orbiter
3771b10b89 implemented automated migration indexCache 0.37 -> indexAssortmentCluster
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@205 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-01 14:24:25 +00:00
orbiter
e89ded9e41 bugfixes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@204 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-31 22:12:43 +00:00
orbiter
650ca3955a added flush-thread for index cache and added language-name mapping in Language_p
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@203 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-31 17:39:14 +00:00
mkossin
3dd3431c58 Fixes a Problem with mySeed.txt. If this file is accessible but has no content it will be recreated: http://www.yacy-forum.de/viewtopic.php?p=3657#3657
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@202 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-31 13:58:30 +00:00
orbiter
3d8a2ff937 enhanced parallelization of local/global/remote crawling
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@197 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-29 11:56:40 +00:00
orbiter
a05d738ea4 enhanced caching, removed bug causing outOfMemory
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@195 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-29 00:34:01 +00:00
allo
3b03d99797 Bugfix: Wrong Path of DATA/LOCALE Dir
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@182 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-27 14:42:54 +00:00
allo
8796987def Copy the shipped Locales on Startup to DATA/LOCALES
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@180 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-27 08:36:07 +00:00
allo
f57d280afd serverLog instead of System.out.println
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@179 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-26 19:45:19 +00:00
allo
a8b2dccc7a New Translation System.
Use Wordlists like these:
de.lng
Network\ Overview=Netzwerk-Übersicht


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@176 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-26 10:26:32 +00:00
orbiter
21110dcd5e fixed bugs with open files and caching
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@175 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-25 13:48:48 +00:00
orbiter
f8f8dd05db fixed "Too many open files" - bug
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@174 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-25 11:35:01 +00:00
theli
74eb21f62e *) adding image tag into rss template
*) adding a xslt stylesheet so that the rss document can be viewed in a normal webbrowser
*) adding pubDate tag to each search item

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@173 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-25 08:47:34 +00:00
orbiter
5f90daa265 implemented localization environment
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@171 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-24 14:23:04 +00:00
theli
84f9d8f7f0 *) migrating ant build files to generate a single extension tar per default
This is buildconfigureable by changing the extensionTarget Property in the build.properties file
*) Trying to solve "yacy.java template replacement / ant build failed" bug

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@169 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-24 09:02:04 +00:00
orbiter
fdd606c8c8 fixed bugs
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@168 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-23 23:50:35 +00:00
theli
8bd49ba535 *) setting root dir for all tar files properly
*) automatically adding SVN Revision number to tar file name
*) introducing build.properties file that can be used to set the build version number and date

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@164 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-23 15:47:08 +00:00
orbiter
0c35171c85 assortment fine-tuning
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@163 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-23 15:07:16 +00:00
orbiter
76dc892017 refined assortment
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@159 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-23 13:15:28 +00:00
theli
0e1d9e9722 *) shrinking httpc linebuffer when httpc is returned to pool. This is done to free memory
*) Making Seed-Upload configuration more verbose.
*) Some Changes in SOAP Search API (not finished yet).

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@158 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-23 10:10:51 +00:00