Commit Graph

172 Commits

Author SHA1 Message Date
theli
bf66c6d6ec *) Bugfix for java.lang.NoClassDefFoundError: de/anomic/yacy/seedUpload/sshc:de/anomic/yacy/seedUpload/sshc
See: http://www.yacy-forum.de/viewtopic.php?p=4345

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@291 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-17 07:54:51 +00:00
orbiter
5d06ded005 enhanced html parser speed
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@290 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-17 01:26:51 +00:00
orbiter
5a490aa065 fixed html parser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@289 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-16 21:49:56 +00:00
orbiter
a25b5b4986 fixed possible memory leak in htmlScraper: be aware that now links can get lost; further work necessary
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@288 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-16 18:31:28 +00:00
orbiter
dbda6e1e85 corrections to port forwarding
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@286 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-16 11:40:36 +00:00
theli
822c9fa276 *) forgotten to add new classes for server port forwarding under version control
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@284 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-16 10:36:23 +00:00
theli
9e47ba5ad6 *) adding missing calls for function close() to avoid "too many open file" bug
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@282 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-16 08:34:52 +00:00
theli
9a98988c3c *) Bugfix for SSL/NIO Bug
See: http://www.yacy-forum.de/viewtopic.php?t=516
   - removing NIO from server/serverCore.java because of massive problems
     with socket close issues
*) Adding support for remote port forwarding via sch
   @Orbiter: Please take a look into
   - hello.java
   - server/serverCore.java.publicIP()
   - yacy/yacyClient.java.publishMySeed(...)
*) Making startup loading of additional content parsers more failsafe


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@281 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-16 07:28:07 +00:00
orbiter
a1ffc27041 preparations for image/movie/music indexing
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@280 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-16 00:31:13 +00:00
orbiter
f45dc29f35 maintenance
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@279 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-15 14:59:53 +00:00
orbiter
a5b40923b6 added word migration to assortments (start with 'java -classpath classes yacy -migratewords')
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@278 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-15 01:22:07 +00:00
theli
33e992cc8b *) unsing println instead of print
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@277 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-13 13:53:15 +00:00
theli
890e3f4d4a *) adding missing calls for function close() to avoid "too many open file" bug*) adding
*) bugfix in plasma/plasmaParser.java:
   - parsers with missing dependencies wehre not ignored correctly
*) passing a logger instance to the parsers modules which can be used 
   for logging purposes by the parsers (not done yet)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@276 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-13 13:49:17 +00:00
orbiter
beb652847e added run-target for ant (needed for NetBeans4)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@274 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-13 12:01:58 +00:00
theli
c7d294d8d4 *) Bugfix for:
- 302 redirection Problem on Amazon Server
   - Wrong References in proxymsg/error.html
   See: http://www.yacy-forum.de/viewtopic.php?t=515

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@271 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-13 07:50:35 +00:00
theli
6dd3ec0dc4 *) Adding debug="true" debuglevel="lines,vars,source" to ant build files
See: http://www.yacy-forum.de/viewtopic.php?p=4099


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@270 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-12 05:22:06 +00:00
orbiter
4f9c30ef49 using mime-type instead of file extension for doctype
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@269 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-10 12:34:18 +00:00
theli
8b89f433b1 *) Changing loglevel to debug
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@268 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-10 09:39:52 +00:00
theli
bbc46f037f *) avoid exception throwing on already catched exception
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@267 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-10 09:38:38 +00:00
theli
f0440911e8 *) removing unnecessary stacktrace printout
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@266 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-10 09:34:50 +00:00
theli
ee9e110366 *) removing old logging configuration properties from yacy.init
*) serverLog.java logging functions now also accept exceptions als
   additional parameters.
   The Stacktrace of this ecceptions will then be appended to the 
   logging message and can e.g. be viewed on the gui logging page

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@265 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-10 09:19:24 +00:00
orbiter
878ff0ae7b corrections
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@262 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 16:39:25 +00:00
theli
361ba50056 *) changing reference to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@261 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 11:15:36 +00:00
theli
7b4778b4b9 *) changing reference to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@260 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 11:02:54 +00:00
(no author)
dedadd1067 *) remove old serverLog class
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@259 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:59:48 +00:00
theli
e5ee4198af *) changing reference to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@258 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:58:54 +00:00
theli
49a23bb89c *) adding reference to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@257 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:57:42 +00:00
theli
02d43a7bd4 *) adding additional methods that should be implemented by all
serverHandler classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@256 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:54:03 +00:00
theli
652a19cf13 *) adding a method to read and zip a file (needed by the filehandler)
for doing gzip encoding

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@255 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:52:59 +00:00
theli
af9cd67334 *) migration to Java NIO
- to avoid buzy waiting and
   - make socket blocking interruptable
*) changing reference to logger
*) introduce commandObjMethodCache to improve performance
*) doing a stream shutdown before closing the connection
   to aviod problems when using persistent connections
*) calling method UNKNOWN of the server-handler class when 
   receiving an unknown command
*) calling method EMPTY of the server-handler class when
   receiving an empty command

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@254 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:51:56 +00:00
theli
1eff96f471 *) removing buzy waiting
*) changing reference to logger

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@253 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:47:50 +00:00
theli
c1a4e0dc28 *) changing reference to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@252 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:44:55 +00:00
theli
d0083f845f *) changing reference to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@251 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:39:09 +00:00
theli
1b5ae054f8 *) changing reference to logger
*) parser will not be returned into pool if the parser was deactivated
   via gui

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@250 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:38:00 +00:00
theli
68f30811fa *) changing reference to logger
*) bugfix in function getCachePath

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@249 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:36:39 +00:00
theli
fbbea813c5 *) changing references to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@248 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:34:20 +00:00
theli
451ca6b818 *) changing reference to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@247 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:25:39 +00:00
theli
4d7540e679 *) adding new constants
*) adding a list of default error messages which belong to each
   http status code

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@246 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:23:03 +00:00
theli
cb17ff4aa2 *) adding support of proxy access logging (much similar to squids
access.log) file
*) splitting doGet function in separate functions for fulfilling
   requests from cache and from web to make error handling easier
*) using connection property and httpHeader constants instead of
   hardcoded strings whenever possible
*) sending back a proxy error message as body of every respond
   containing a http error code
*) correcting problems of messages received from other proxies
   containing 204, 304 status codes.
*) using chunked transfer encoding if the server has not set the
   content length (e.g. because of gzip content encoding) but 
   the client has established a persistent connection to yacy.
   This is only possible for http/1.1 clients. For http/1.0 clients
   the connection will simply be closed on the end of the message.   
*) removing unneeded functions (e.g. respondError) because of newly
   introduced functions of httpd.java
*) removing hop by hop headers (according to rfc)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@245 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:22:05 +00:00
theli
f157181086 *) starting implementation of Content-MD5 header
which should help to detect transfer errors on yacy to yacy
   communication
   - not finished yet
*) removing unneeded functions (e.g. respondHeader) because newly
   introduced functions in class httpd.java
*) httpdFileHandler.java now always sends back a proxy error message
   as body of a response with an error code
*) adding support of gzip content encoding 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@244 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:12:07 +00:00
theli
d2042ec8fa *) adding constants for all needed connection properties
*) adding hashset for file-extensions that should not be transfered
   using gzip content encoding
*) doing bugfixes on old keep-alive implementation
*) doing some additional http header validation according to rfc
*) doing all persistent connection detection in separate function now
*) doing server authentication in separate function now
*) doing proxy authentication in separate function now
*) simplifying GET, POST, HEAD functions because of new introduced 
   functions listed above
*) adding new function to handle empty request lines (which could
   occure after post requests send via a persistent connection; 
   this depends on the used browser)
*) adding new function to handle unknown request methods by sending
   a correct error message back to the client
*) setting correct content-length when sending back error messages
   to the client
*) adding new functions that must be used by all http-Handler classes
   to send 
   - a proxy error message
   - a http header
   back to the client
*) adding new function: shallTransportZipped
   moved here from httpc

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@243 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:07:02 +00:00
theli
06b0db2cac *) adding toString method to
- httpc
   - response
*) simplifying gzip encoding
*) remembering http version of contacted server
   (neede for later support of keep alive by httpc)
*) moving function shallTransportZipped to httpd.java
   because this function is used multiple times

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@242 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:56:41 +00:00
theli
7b0b72dd23 *) adding new streams for
- implementation of outgoing chunked transfer encoding (httpChunkedOutputStream.java)
- byte counting for proxy access logging / global traffic count
  (httpdByteCount(In|Out)putStream.java

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@241 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:47:56 +00:00
theli
318dfc4636 *) activating new logging framework
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@240 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:46:43 +00:00
theli
aae3e3b7ab *) reusing buffer instead of creating a new one
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@239 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:45:58 +00:00
theli
829b65c1c8 *) adding additional classes needed for new logging
- ConsoleOutErrHandler.java used to log warnings/errors to stderr 
  and all other messages to stdout
- GuiHandler.java
  used to keep logging messages in memory that can then be viewed
  via the http gui
- serverSimpleLogFormatter.java
  needed to format logging messages for FileHandler, ConsoleOutErrHandler
  and GuiHandler
- serverMiniLogFormatter.java
  needed for proxy access logging

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@233 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:25:08 +00:00
low012
8c2789b22a to catch is an irregular verb
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@225 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 00:07:25 +00:00
orbiter
4574fa4ce7 bugfixes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@224 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-08 15:28:29 +00:00
theli
83b41ef2f7 *) Adding timeouts for shutdown
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@223 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-08 13:44:25 +00:00
theli
ef6851798b *) changing thread priority while parsing a pdf file to avoid 100% CPU usage.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@222 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-08 13:23:35 +00:00