Commit Graph

377 Commits

Author SHA1 Message Date
theli
cb17ff4aa2 *) adding support of proxy access logging (much similar to squids
access.log) file
*) splitting doGet function in separate functions for fulfilling
   requests from cache and from web to make error handling easier
*) using connection property and httpHeader constants instead of
   hardcoded strings whenever possible
*) sending back a proxy error message as body of every respond
   containing a http error code
*) correcting problems of messages received from other proxies
   containing 204, 304 status codes.
*) using chunked transfer encoding if the server has not set the
   content length (e.g. because of gzip content encoding) but 
   the client has established a persistent connection to yacy.
   This is only possible for http/1.1 clients. For http/1.0 clients
   the connection will simply be closed on the end of the message.   
*) removing unneeded functions (e.g. respondError) because of newly
   introduced functions of httpd.java
*) removing hop by hop headers (according to rfc)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@245 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:22:05 +00:00
theli
f157181086 *) starting implementation of Content-MD5 header
which should help to detect transfer errors on yacy to yacy
   communication
   - not finished yet
*) removing unneeded functions (e.g. respondHeader) because newly
   introduced functions in class httpd.java
*) httpdFileHandler.java now always sends back a proxy error message
   as body of a response with an error code
*) adding support of gzip content encoding 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@244 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:12:07 +00:00
theli
d2042ec8fa *) adding constants for all needed connection properties
*) adding hashset for file-extensions that should not be transfered
   using gzip content encoding
*) doing bugfixes on old keep-alive implementation
*) doing some additional http header validation according to rfc
*) doing all persistent connection detection in separate function now
*) doing server authentication in separate function now
*) doing proxy authentication in separate function now
*) simplifying GET, POST, HEAD functions because of new introduced 
   functions listed above
*) adding new function to handle empty request lines (which could
   occure after post requests send via a persistent connection; 
   this depends on the used browser)
*) adding new function to handle unknown request methods by sending
   a correct error message back to the client
*) setting correct content-length when sending back error messages
   to the client
*) adding new functions that must be used by all http-Handler classes
   to send 
   - a proxy error message
   - a http header
   back to the client
*) adding new function: shallTransportZipped
   moved here from httpc

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@243 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:07:02 +00:00
theli
06b0db2cac *) adding toString method to
- httpc
   - response
*) simplifying gzip encoding
*) remembering http version of contacted server
   (neede for later support of keep alive by httpc)
*) moving function shallTransportZipped to httpd.java
   because this function is used multiple times

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@242 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:56:41 +00:00
theli
7b0b72dd23 *) adding new streams for
- implementation of outgoing chunked transfer encoding (httpChunkedOutputStream.java)
- byte counting for proxy access logging / global traffic count
  (httpdByteCount(In|Out)putStream.java

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@241 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:47:56 +00:00
theli
318dfc4636 *) activating new logging framework
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@240 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:46:43 +00:00
theli
aae3e3b7ab *) reusing buffer instead of creating a new one
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@239 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:45:58 +00:00
theli
a566588e9b *) adding configuration section for new http keep-alive support
*) moving transparent proxy configuration into new config section


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@238 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:44:07 +00:00
theli
6bde37950f *) adding xsl stylesheet that can be used by browsers to format the rss search result in a user friendly format
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@237 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:36:44 +00:00
theli
e302c50390 *) Changes needed because of logging migration
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@236 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:34:44 +00:00
theli
f9a95b5cb8 *) Displaying more user friendly Memory Usage statistic
*) Displaying traffic consumed by yacy 
   - this is not finished yet
   - at the moment only outgoing proxy traffic is counted

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@235 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:33:15 +00:00
theli
f8de2556f8 *) Changes needed to allow a move verbose error message
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@234 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:29:35 +00:00
theli
829b65c1c8 *) adding additional classes needed for new logging
- ConsoleOutErrHandler.java used to log warnings/errors to stderr 
  and all other messages to stdout
- GuiHandler.java
  used to keep logging messages in memory that can then be viewed
  via the http gui
- serverSimpleLogFormatter.java
  needed to format logging messages for FileHandler, ConsoleOutErrHandler
  and GuiHandler
- serverMiniLogFormatter.java
  needed for proxy access logging

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@233 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:25:08 +00:00
theli
cef944eb33 *) Adding property for enabling/disabling connection keep-alive support
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@232 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:21:00 +00:00
theli
84438bfa6b *) adding configuration file for new logging implementation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@231 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:17:21 +00:00
low012
8c2789b22a to catch is an irregular verb
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@225 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 00:07:25 +00:00
orbiter
4574fa4ce7 bugfixes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@224 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-08 15:28:29 +00:00
theli
83b41ef2f7 *) Adding timeouts for shutdown
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@223 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-08 13:44:25 +00:00
theli
ef6851798b *) changing thread priority while parsing a pdf file to avoid 100% CPU usage.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@222 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-08 13:23:35 +00:00
orbiter
33f9315e58 implemented multithreading of indexing
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@221 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-08 13:19:05 +00:00
allo
7c318f8886 Language files, not skins...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@220 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-08 11:05:04 +00:00
theli
52cf732fad *) correcting "seed-ftp-upload/Nothing changed" bug:
See: http://www.yacy-forum.de/viewtopic.php?p=3986#3986

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@219 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-08 09:54:52 +00:00
orbiter
ca3b4ccaf4 added snippet-routines (not yet finished)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@218 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-08 00:52:24 +00:00
orbiter
9244b6ad6f more typo/expression corrections (mailed to me from Sebastian Seifert)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@217 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-07 12:54:42 +00:00
orbiter
a73e6de005 faq/doc - Fix
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@215 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-07 09:28:45 +00:00
theli
9af1bf4b38 *) displaying memory usage of yacy in Status.html
*) displaying more expressive uptime information on Status.html and Network.html

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@214 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-07 09:05:21 +00:00
theli
167ddd5612 *) Displaying "SVN xxx" instead of "@REPL_VERSION@" if yacy is started directly from SVN
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@213 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-07 08:31:49 +00:00
orbiter
ee0758fe4d bugfixes/empty-dir-deletion/snippet-test-activation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@212 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-07 01:05:55 +00:00
allo
2d85bdb341 Installer for YaCy 0.38
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@210 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-03 13:38:07 +00:00
orbiter
a12922536a latest changes for 0.38
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@209 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-03 09:22:50 +00:00
orbiter
594c591223 changes towards 0.38
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@208 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-03 02:43:35 +00:00
(no author)
c8b24569e3 *) solving problem with duplicated class files in htroot
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@207 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-02 06:58:20 +00:00
orbiter
d8fdc2526e added experimental snipplet-generation (to be disabled for 0.38)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@206 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-02 01:33:10 +00:00
orbiter
3771b10b89 implemented automated migration indexCache 0.37 -> indexAssortmentCluster
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@205 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-01 14:24:25 +00:00
orbiter
e89ded9e41 bugfixes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@204 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-31 22:12:43 +00:00
orbiter
650ca3955a added flush-thread for index cache and added language-name mapping in Language_p
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@203 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-31 17:39:14 +00:00
mkossin
3dd3431c58 Fixes a Problem with mySeed.txt. If this file is accessible but has no content it will be recreated: http://www.yacy-forum.de/viewtopic.php?p=3657#3657
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@202 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-31 13:58:30 +00:00
rramthun
ec77e6faa6 updated german language file
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@201 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-30 18:19:04 +00:00
allo
04b230f85d Startmenuentry for startYACY_noconsole.bat.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@199 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-29 20:12:56 +00:00
rramthun
4e63456dba some corrections/enhancements to the webinterface
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@198 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-29 18:23:58 +00:00
orbiter
3d8a2ff937 enhanced parallelization of local/global/remote crawling
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@197 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-29 11:56:40 +00:00
orbiter
a05d738ea4 enhanced caching, removed bug causing outOfMemory
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@195 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-29 00:34:01 +00:00
allo
8ab021b39f Your-Peer Stats added
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@185 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-28 11:07:08 +00:00
allo
c6331cf088 "Comma separated Values File" for the Network Overview Page.
http://www.yacy-forum.de/viewtopic.php?t=4


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@184 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-28 10:45:42 +00:00
rramthun
9678614db5 Some minor changes to test functionality. Works...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@183 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-27 16:31:09 +00:00
allo
3b03d99797 Bugfix: Wrong Path of DATA/LOCALE Dir
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@182 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-27 14:42:54 +00:00
allo
9c224ffe99 Execution Filerights for Unix Wrappers
added locales to the tar


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@181 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-27 08:59:01 +00:00
allo
8796987def Copy the shipped Locales on Startup to DATA/LOCALES
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@180 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-27 08:36:07 +00:00
allo
f57d280afd serverLog instead of System.out.println
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@179 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-26 19:45:19 +00:00
allo
7de98b1895 Deleted static locales because they conflict with the dynamic ones.
use Language_p.html and $rootdir/locales/*.lng to generate locales in htroot/locales


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@178 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-26 14:08:22 +00:00