Commit Graph

181 Commits

Author SHA1 Message Date
orbiter
e3c92818db avoiding OutOfMemoryError routines
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@302 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-19 13:37:17 +00:00
orbiter
3e8ee5a46d enhanced caching in kelondroRecords and added better synchronization/finalizer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@301 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-19 05:27:42 +00:00
orbiter
3be98f194d tried to find the socket bug
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@300 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-18 01:48:11 +00:00
theli
dff937a9a3 *) Adding some logging statements to detect problems with seed upload functionality
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@297 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-17 08:38:22 +00:00
theli
98230a67f0 *) correcting version number for remote port forwarding feature check
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@296 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-17 08:16:43 +00:00
theli
9f5d239051 *) correcting version number for remote port forwarding feature check
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@295 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-17 08:06:23 +00:00
theli
0405de7635 *) Avoiding NullpointerException while testing exception test
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@294 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-17 08:04:53 +00:00
theli
db3ed75728 *) closing stream correctly
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@293 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-17 07:58:02 +00:00
theli
aea355c03c *) adding test for connection status of port forwarding feature
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@292 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-17 07:57:00 +00:00
theli
bf66c6d6ec *) Bugfix for java.lang.NoClassDefFoundError: de/anomic/yacy/seedUpload/sshc:de/anomic/yacy/seedUpload/sshc
See: http://www.yacy-forum.de/viewtopic.php?p=4345

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@291 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-17 07:54:51 +00:00
orbiter
5d06ded005 enhanced html parser speed
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@290 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-17 01:26:51 +00:00
orbiter
5a490aa065 fixed html parser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@289 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-16 21:49:56 +00:00
orbiter
a25b5b4986 fixed possible memory leak in htmlScraper: be aware that now links can get lost; further work necessary
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@288 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-16 18:31:28 +00:00
orbiter
dbda6e1e85 corrections to port forwarding
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@286 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-16 11:40:36 +00:00
theli
822c9fa276 *) forgotten to add new classes for server port forwarding under version control
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@284 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-16 10:36:23 +00:00
theli
9e47ba5ad6 *) adding missing calls for function close() to avoid "too many open file" bug
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@282 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-16 08:34:52 +00:00
theli
9a98988c3c *) Bugfix for SSL/NIO Bug
See: http://www.yacy-forum.de/viewtopic.php?t=516
   - removing NIO from server/serverCore.java because of massive problems
     with socket close issues
*) Adding support for remote port forwarding via sch
   @Orbiter: Please take a look into
   - hello.java
   - server/serverCore.java.publicIP()
   - yacy/yacyClient.java.publishMySeed(...)
*) Making startup loading of additional content parsers more failsafe


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@281 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-16 07:28:07 +00:00
orbiter
a1ffc27041 preparations for image/movie/music indexing
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@280 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-16 00:31:13 +00:00
orbiter
f45dc29f35 maintenance
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@279 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-15 14:59:53 +00:00
orbiter
a5b40923b6 added word migration to assortments (start with 'java -classpath classes yacy -migratewords')
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@278 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-15 01:22:07 +00:00
theli
33e992cc8b *) unsing println instead of print
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@277 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-13 13:53:15 +00:00
theli
890e3f4d4a *) adding missing calls for function close() to avoid "too many open file" bug*) adding
*) bugfix in plasma/plasmaParser.java:
   - parsers with missing dependencies wehre not ignored correctly
*) passing a logger instance to the parsers modules which can be used 
   for logging purposes by the parsers (not done yet)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@276 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-13 13:49:17 +00:00
orbiter
beb652847e added run-target for ant (needed for NetBeans4)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@274 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-13 12:01:58 +00:00
theli
c7d294d8d4 *) Bugfix for:
- 302 redirection Problem on Amazon Server
   - Wrong References in proxymsg/error.html
   See: http://www.yacy-forum.de/viewtopic.php?t=515

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@271 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-13 07:50:35 +00:00
theli
6dd3ec0dc4 *) Adding debug="true" debuglevel="lines,vars,source" to ant build files
See: http://www.yacy-forum.de/viewtopic.php?p=4099


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@270 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-12 05:22:06 +00:00
orbiter
4f9c30ef49 using mime-type instead of file extension for doctype
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@269 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-10 12:34:18 +00:00
theli
8b89f433b1 *) Changing loglevel to debug
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@268 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-10 09:39:52 +00:00
theli
bbc46f037f *) avoid exception throwing on already catched exception
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@267 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-10 09:38:38 +00:00
theli
f0440911e8 *) removing unnecessary stacktrace printout
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@266 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-10 09:34:50 +00:00
theli
ee9e110366 *) removing old logging configuration properties from yacy.init
*) serverLog.java logging functions now also accept exceptions als
   additional parameters.
   The Stacktrace of this ecceptions will then be appended to the 
   logging message and can e.g. be viewed on the gui logging page

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@265 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-10 09:19:24 +00:00
orbiter
878ff0ae7b corrections
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@262 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 16:39:25 +00:00
theli
361ba50056 *) changing reference to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@261 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 11:15:36 +00:00
theli
7b4778b4b9 *) changing reference to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@260 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 11:02:54 +00:00
(no author)
dedadd1067 *) remove old serverLog class
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@259 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:59:48 +00:00
theli
e5ee4198af *) changing reference to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@258 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:58:54 +00:00
theli
49a23bb89c *) adding reference to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@257 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:57:42 +00:00
theli
02d43a7bd4 *) adding additional methods that should be implemented by all
serverHandler classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@256 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:54:03 +00:00
theli
652a19cf13 *) adding a method to read and zip a file (needed by the filehandler)
for doing gzip encoding

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@255 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:52:59 +00:00
theli
af9cd67334 *) migration to Java NIO
- to avoid buzy waiting and
   - make socket blocking interruptable
*) changing reference to logger
*) introduce commandObjMethodCache to improve performance
*) doing a stream shutdown before closing the connection
   to aviod problems when using persistent connections
*) calling method UNKNOWN of the server-handler class when 
   receiving an unknown command
*) calling method EMPTY of the server-handler class when
   receiving an empty command

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@254 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:51:56 +00:00
theli
1eff96f471 *) removing buzy waiting
*) changing reference to logger

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@253 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:47:50 +00:00
theli
c1a4e0dc28 *) changing reference to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@252 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:44:55 +00:00
theli
d0083f845f *) changing reference to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@251 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:39:09 +00:00
theli
1b5ae054f8 *) changing reference to logger
*) parser will not be returned into pool if the parser was deactivated
   via gui

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@250 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:38:00 +00:00
theli
68f30811fa *) changing reference to logger
*) bugfix in function getCachePath

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@249 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:36:39 +00:00
theli
fbbea813c5 *) changing references to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@248 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:34:20 +00:00
theli
451ca6b818 *) changing reference to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@247 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:25:39 +00:00
theli
4d7540e679 *) adding new constants
*) adding a list of default error messages which belong to each
   http status code

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@246 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:23:03 +00:00
theli
cb17ff4aa2 *) adding support of proxy access logging (much similar to squids
access.log) file
*) splitting doGet function in separate functions for fulfilling
   requests from cache and from web to make error handling easier
*) using connection property and httpHeader constants instead of
   hardcoded strings whenever possible
*) sending back a proxy error message as body of every respond
   containing a http error code
*) correcting problems of messages received from other proxies
   containing 204, 304 status codes.
*) using chunked transfer encoding if the server has not set the
   content length (e.g. because of gzip content encoding) but 
   the client has established a persistent connection to yacy.
   This is only possible for http/1.1 clients. For http/1.0 clients
   the connection will simply be closed on the end of the message.   
*) removing unneeded functions (e.g. respondError) because of newly
   introduced functions of httpd.java
*) removing hop by hop headers (according to rfc)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@245 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:22:05 +00:00
theli
f157181086 *) starting implementation of Content-MD5 header
which should help to detect transfer errors on yacy to yacy
   communication
   - not finished yet
*) removing unneeded functions (e.g. respondHeader) because newly
   introduced functions in class httpd.java
*) httpdFileHandler.java now always sends back a proxy error message
   as body of a response with an error code
*) adding support of gzip content encoding 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@244 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:12:07 +00:00
theli
d2042ec8fa *) adding constants for all needed connection properties
*) adding hashset for file-extensions that should not be transfered
   using gzip content encoding
*) doing bugfixes on old keep-alive implementation
*) doing some additional http header validation according to rfc
*) doing all persistent connection detection in separate function now
*) doing server authentication in separate function now
*) doing proxy authentication in separate function now
*) simplifying GET, POST, HEAD functions because of new introduced 
   functions listed above
*) adding new function to handle empty request lines (which could
   occure after post requests send via a persistent connection; 
   this depends on the used browser)
*) adding new function to handle unknown request methods by sending
   a correct error message back to the client
*) setting correct content-length when sending back error messages
   to the client
*) adding new functions that must be used by all http-Handler classes
   to send 
   - a proxy error message
   - a http header
   back to the client
*) adding new function: shallTransportZipped
   moved here from httpc

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@243 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:07:02 +00:00