Commit Graph

71 Commits

Author SHA1 Message Date
theli
b32e7c516c git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@507 6c8d7289-2bf4-0310-a012-ef5d649a1542 2005-08-09 09:07:19 +00:00
theli
e6aced0162 *) Setting higher priority for session threads
See: http://www.yacy-forum.de/viewtopic.php?p=6120#6120

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@490 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-08-03 17:48:11 +00:00
orbiter
2d8557cb10 minor changes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@487 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-08-03 02:02:39 +00:00
orbiter
fb6f238d70 fix for expires-problem
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@485 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-08-03 00:28:12 +00:00
orbiter
d34eb23e4e fixed news; added news appearance on Network and IndexCreate page; added intention string to global crawl
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@466 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-08-01 01:12:02 +00:00
orbiter
af67c633d5 doc-changes and more strict brute-force handling
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@431 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-25 09:56:54 +00:00
orbiter
86d778f7bc default-button in profile menue
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@419 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-20 17:08:58 +00:00
orbiter
b73557ed2d better assortment monitoring and enhanced profile menue
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@416 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-20 13:03:41 +00:00
rramthun
0f11399d16 Some corrections...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@409 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-19 16:07:13 +00:00
orbiter
1d2155675b changed assortment memory cache flush
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@403 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-17 21:22:18 +00:00
orbiter
19dbed7cc8 code clean-up
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@401 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-12 15:09:35 +00:00
orbiter
40036ba69c fixed dht transmission; added url-blacklist blocking also for remote search
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@398 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-12 00:07:09 +00:00
orbiter
311e627363 blocking of blacklisted urls in indexReceive and small changes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@397 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-11 15:36:10 +00:00
theli
55d10b864c *) further improvements in shutdown behaviour
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@392 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-07 22:03:52 +00:00
theli
96386d6fc6 *) further improvements in shutdown behaviour
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@391 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-07 21:44:31 +00:00
theli
0b95c9c434 *) Bugfix for Thread.getID() usagage + PeerPing-Shutdown Deadlock
See:
   - http://www.yacy-forum.de/viewtopic.php?p=4937
   - http://www.yacy-forum.de/viewtopic.php?p=4939

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@390 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-07 21:00:11 +00:00
theli
13eeaa08f3 *) httpc.java:
- Now it's possible to interrupt pending httpc-actions on server shutdown  
   - this is possible because of a newly introduced registration mechanism for
     open sockets
*) yacyCore.java
   - blocking peerPing threads can now be interrupted on server shutdown
*) serverCore.java
   - restructuring shutdown code 
*) error.html
   - port number is now set correctly if port forwarding was enabled


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@389 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-07 13:58:54 +00:00
orbiter
858cd94299 replaced indexing ram-queue by file-based stack-queue
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@381 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-06 14:48:41 +00:00
theli
57c30f1d78 *) bugfix for usage of httpc without gzip content encoding
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@369 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-04 11:25:25 +00:00
theli
0e2c33ee55 *) Network.html/Network.java:
- Adding function to manually force peer ping to remote yacy peer
  See:Network.html?page=4
- for debugging purpose only!

*) serverAbstractThread.java:
- Adding posibility to notify a server thread via a synchronization object
- this is needed e.g. by the port forwarding feature to send a notification
  to the peerPing thread to redo peer-ping with the new ip/port Settings_p.html

*) Port Forwarding Feature (it should work now)
- adding a serverThread which is responsible to detect broken port forwarding 
  connections and to do reconnect if needed
- serverCore.java: moving port forwarding initialization into a separate function
- adding positility to configure the ssh port 
- moving configuration section on the gui into a separate fieldset
- hello.java: only trying to do a second connect to the clientIp address during
  peer handshake if either remote port forwarding is not enabled locally or
  the clientIP is not equal to any local ip

*) httpdFileHandler.java:
- printout a more verbose errormessage

*) httpc.java
- allowing to deactivate content encoding from outside


 

*) plasmaCrawlWorker.java
- the crawler worker now tries to refetch the content of a website without
  gzip content encoding if a gzip error occured



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@368 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-04 11:09:48 +00:00
orbiter
712fe9ef18 bugfixed utf-8 decoding and parser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@346 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-29 22:55:37 +00:00
orbiter
abba8fe61b fixed utf-8 decoding in htmlFilterAbstractScraper and removed httpd timing
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@323 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-26 22:40:50 +00:00
theli
d53b2393e5 *) autoconfig.java: ip address was not reported correctly when port-forwardin is on
*) hello.java: reportedip my be empty at peer startup
*) httpc.java: adding method to determine if the connection was already closed or is broken
*) httpdProxyHandler.java: trying to do a better errorhandling
*) server/serverCore.java
- setting myseed ip-address and port correctly if port-forwarding is on
- doing a more failsafe close and adding some debugging output
*) yacyClient.java: adding some logging statements to allow a better detection of 
   "degraded to senior"-bug
*) yacyCore.java: restructuring publishMySeed
   (@Orbiter: pleas take a look)
- to avoid buzy waiting
- to allow a gracefull shutdown on server shutdown
- new seed count was not calculated correctly in the previous version
*) yacySeedDB.java: host ip and port was not initialized correctly if port-forwarding
   was activated

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@318 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-23 11:00:26 +00:00
orbiter
1e7f062350 many bugfixes, memory leak fixes, performance enhancements; new kelondroHashtable; activated snippets
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@313 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-23 02:07:45 +00:00
theli
bf78e0a6a4 *) Bugfix of Socket closed Exception
See: http://www.yacy-forum.de/viewtopic.php?p=4437

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@304 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-20 13:33:23 +00:00
orbiter
85075269a6 extended fail-safe memory-managament. prevents too much allocation, too often GC and should help for the 100%CPU-bug
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@303 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-20 00:46:23 +00:00
theli
aea355c03c *) adding test for connection status of port forwarding feature
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@292 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-17 07:57:00 +00:00
orbiter
5d06ded005 enhanced html parser speed
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@290 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-17 01:26:51 +00:00
orbiter
a25b5b4986 fixed possible memory leak in htmlScraper: be aware that now links can get lost; further work necessary
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@288 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-16 18:31:28 +00:00
theli
822c9fa276 *) forgotten to add new classes for server port forwarding under version control
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@284 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-16 10:36:23 +00:00
theli
9e47ba5ad6 *) adding missing calls for function close() to avoid "too many open file" bug
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@282 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-16 08:34:52 +00:00
theli
9a98988c3c *) Bugfix for SSL/NIO Bug
See: http://www.yacy-forum.de/viewtopic.php?t=516
   - removing NIO from server/serverCore.java because of massive problems
     with socket close issues
*) Adding support for remote port forwarding via sch
   @Orbiter: Please take a look into
   - hello.java
   - server/serverCore.java.publicIP()
   - yacy/yacyClient.java.publishMySeed(...)
*) Making startup loading of additional content parsers more failsafe


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@281 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-16 07:28:07 +00:00
orbiter
a5b40923b6 added word migration to assortments (start with 'java -classpath classes yacy -migratewords')
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@278 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-15 01:22:07 +00:00
theli
33e992cc8b *) unsing println instead of print
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@277 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-13 13:53:15 +00:00
theli
890e3f4d4a *) adding missing calls for function close() to avoid "too many open file" bug*) adding
*) bugfix in plasma/plasmaParser.java:
   - parsers with missing dependencies wehre not ignored correctly
*) passing a logger instance to the parsers modules which can be used 
   for logging purposes by the parsers (not done yet)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@276 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-13 13:49:17 +00:00
orbiter
beb652847e added run-target for ant (needed for NetBeans4)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@274 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-13 12:01:58 +00:00
theli
8b89f433b1 *) Changing loglevel to debug
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@268 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-10 09:39:52 +00:00
theli
ee9e110366 *) removing old logging configuration properties from yacy.init
*) serverLog.java logging functions now also accept exceptions als
   additional parameters.
   The Stacktrace of this ecceptions will then be appended to the 
   logging message and can e.g. be viewed on the gui logging page

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@265 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-10 09:19:24 +00:00
orbiter
878ff0ae7b corrections
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@262 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 16:39:25 +00:00
(no author)
dedadd1067 *) remove old serverLog class
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@259 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:59:48 +00:00
theli
49a23bb89c *) adding reference to logger
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@257 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:57:42 +00:00
theli
02d43a7bd4 *) adding additional methods that should be implemented by all
serverHandler classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@256 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:54:03 +00:00
theli
652a19cf13 *) adding a method to read and zip a file (needed by the filehandler)
for doing gzip encoding

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@255 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:52:59 +00:00
theli
af9cd67334 *) migration to Java NIO
- to avoid buzy waiting and
   - make socket blocking interruptable
*) changing reference to logger
*) introduce commandObjMethodCache to improve performance
*) doing a stream shutdown before closing the connection
   to aviod problems when using persistent connections
*) calling method UNKNOWN of the server-handler class when 
   receiving an unknown command
*) calling method EMPTY of the server-handler class when
   receiving an empty command

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@254 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:51:56 +00:00
theli
1eff96f471 *) removing buzy waiting
*) changing reference to logger

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@253 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 10:47:50 +00:00
theli
829b65c1c8 *) adding additional classes needed for new logging
- ConsoleOutErrHandler.java used to log warnings/errors to stderr 
  and all other messages to stdout
- GuiHandler.java
  used to keep logging messages in memory that can then be viewed
  via the http gui
- serverSimpleLogFormatter.java
  needed to format logging messages for FileHandler, ConsoleOutErrHandler
  and GuiHandler
- serverMiniLogFormatter.java
  needed for proxy access logging

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@233 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:25:08 +00:00
low012
8c2789b22a to catch is an irregular verb
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@225 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 00:07:25 +00:00
orbiter
4574fa4ce7 bugfixes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@224 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-08 15:28:29 +00:00
theli
83b41ef2f7 *) Adding timeouts for shutdown
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@223 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-08 13:44:25 +00:00
orbiter
ca3b4ccaf4 added snippet-routines (not yet finished)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@218 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-08 00:52:24 +00:00