*) serverLog.java logging functions now also accept exceptions als
additional parameters.
The Stacktrace of this ecceptions will then be appended to the
logging message and can e.g. be viewed on the gui logging page
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@265 6c8d7289-2bf4-0310-a012-ef5d649a1542
access.log) file
*) splitting doGet function in separate functions for fulfilling
requests from cache and from web to make error handling easier
*) using connection property and httpHeader constants instead of
hardcoded strings whenever possible
*) sending back a proxy error message as body of every respond
containing a http error code
*) correcting problems of messages received from other proxies
containing 204, 304 status codes.
*) using chunked transfer encoding if the server has not set the
content length (e.g. because of gzip content encoding) but
the client has established a persistent connection to yacy.
This is only possible for http/1.1 clients. For http/1.0 clients
the connection will simply be closed on the end of the message.
*) removing unneeded functions (e.g. respondError) because of newly
introduced functions of httpd.java
*) removing hop by hop headers (according to rfc)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@245 6c8d7289-2bf4-0310-a012-ef5d649a1542
which should help to detect transfer errors on yacy to yacy
communication
- not finished yet
*) removing unneeded functions (e.g. respondHeader) because newly
introduced functions in class httpd.java
*) httpdFileHandler.java now always sends back a proxy error message
as body of a response with an error code
*) adding support of gzip content encoding
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@244 6c8d7289-2bf4-0310-a012-ef5d649a1542
*) adding hashset for file-extensions that should not be transfered
using gzip content encoding
*) doing bugfixes on old keep-alive implementation
*) doing some additional http header validation according to rfc
*) doing all persistent connection detection in separate function now
*) doing server authentication in separate function now
*) doing proxy authentication in separate function now
*) simplifying GET, POST, HEAD functions because of new introduced
functions listed above
*) adding new function to handle empty request lines (which could
occure after post requests send via a persistent connection;
this depends on the used browser)
*) adding new function to handle unknown request methods by sending
a correct error message back to the client
*) setting correct content-length when sending back error messages
to the client
*) adding new functions that must be used by all http-Handler classes
to send
- a proxy error message
- a http header
back to the client
*) adding new function: shallTransportZipped
moved here from httpc
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@243 6c8d7289-2bf4-0310-a012-ef5d649a1542
- httpc
- response
*) simplifying gzip encoding
*) remembering http version of contacted server
(neede for later support of keep alive by httpc)
*) moving function shallTransportZipped to httpd.java
because this function is used multiple times
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@242 6c8d7289-2bf4-0310-a012-ef5d649a1542
*) Making Seed-Upload configuration more verbose.
*) Some Changes in SOAP Search API (not finished yet).
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@158 6c8d7289-2bf4-0310-a012-ef5d649a1542
optional content parsers, thread pool configuration ...
Please help me testing if everything works correct.
*) Migration of yacy seedUpload functionality
See: http://www.yacy-forum.de/viewtopic.php?t=256
- new uploaders can now be easily introduced because of a new modulare uploader system
- default uploaders are: none, file, ftp
- adding optional uploader for scp
- each uploader provides its own configuration file that will be
included into the settings page using the new template include feature
- Each uploader can define its libx dependencies. If not all needed libs are
available, the uploader is deactivated automatically.
*) Migration of optional parsers
See: http://www.yacy-forum.de/viewtopic.php?t=198
- Parsers can now also define there libx dependencies
- adding parser for bzip compressed content
- adding parser for gzip compressed content
- adding parser for zip files
- adding parser for tar files
- adding parser to detect the mime-type of a file
this is needed by the bzip/gzip Parser.java
- adding parser for rtf files
- removing extra configuration file yacy.parser
the list of enabled parsers is now stored in the main config file
*) Adding configuration option in the performance dialog to configure
See: http://www.yacy-forum.de/viewtopic.php?t=267
- maxActive / maxIdle / minIdle values for httpd-session-threadpool
- maxActive / maxIdle / minIdle values for crawler-threadpool
*) Changing Crawling Filter behaviour
See: http://www.yacy-forum.de/viewtopic.php?p=2631
*) Replacing some hardcoded strings with the proper constants of the httpHeader class
*) Adding new libs to libx directory. This libs are
- needed by new content parsers
- needed by new optional seed uploader
- needed by SOAP API (which will be committed later)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@126 6c8d7289-2bf4-0310-a012-ef5d649a1542
so that a thread dump is more verbose
*) Moving code for transparent proxy support to a separate function
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@98 6c8d7289-2bf4-0310-a012-ef5d649a1542
- introduction of a threadpool for crawling
- introduction of a job queue to avoid buzy waiting for a free crawler slot
*) New classes added
- queue for receiving of crawler jobs
- semaphore class to do reader/writer synchronization (mutual exclusion)
- message object to hold all needed data about a crawler job
*) Trying to solve session-thread shutdown problem
- session thread stopped variable is now set from outside before interrupting the
session thread.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@39 6c8d7289-2bf4-0310-a012-ef5d649a1542
can be used instead of a ByteArrayOutputStream
*) Using a serverByteBuffer for lineBuffering in class httpc
instead of a ByteArrayOutputStream
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@35 6c8d7289-2bf4-0310-a012-ef5d649a1542
- httpc: wrong error-message on 404
- httpc: error message was accidentally shown when object
was released from pool
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@31 6c8d7289-2bf4-0310-a012-ef5d649a1542
- many classes set to final
- implementation of a session-thread pool
- reusage of the server handler class (normally the httpd object)
within the session thread
- implementation of a httpc object pool
- introduction of a linebuffer in httpd which can be reused
- reusing the properties table in the httpc
- added to apache libs (commons-collections, commons-pool) which
are needed for the object/thread pool implementation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@26 6c8d7289-2bf4-0310-a012-ef5d649a1542
