Commit Graph

47 Commits

Author SHA1 Message Date
theli
9444852896 *) Correcting problems if the port number was set to -1, e.g. because of the usage of function
URL.getPort()

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@673 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-09-07 09:54:11 +00:00
theli
6c722706b7 *) Moving yacyDebugMode intialization to switchboard
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@660 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-09-05 10:34:34 +00:00
theli
a7256e8f4e *) Adding X-Forwarded-For Header
See: http://www.yacy-forum.de/viewtopic.php?t=1118&highlight=xforwardedfor
*) httpc.java: Bugfix for incorrect http response statuscode parsing 
   In some situations the statustext whas chopped
*) Adding a lot of fileheaders containing YaCy copyright and license
*) httpd.java: Adding additional debugging http header that should help du detect
   the "binary data in browser window" bug.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@653 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-09-05 08:01:54 +00:00
theli
a20814291f *) Bugfix for "Race condition zwischen httpc und switchboard"
See: http://www.yacy-forum.de/viewtopic.php?p=9036

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@644 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-09-03 13:40:32 +00:00
theli
286853fd39 *) Bugfix for "YACY hängt sich beim Beenden auf" Bug
See: http://www.yacy-forum.de/viewtopic.php?p=8997

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@643 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-09-03 07:40:51 +00:00
allo
022c1ab179 performance fix for yacyDebugMode and useYacyReferer.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@638 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-09-02 08:21:33 +00:00
allo
286442fbc5 do not Use YaCy-Sites as Referer, if useYacyReferer = false
http://www.yacy-forum.de/viewtopic.php?p=8896#8896


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@637 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-09-02 06:26:38 +00:00
theli
4fd5b95b1f *) Renaming Logger function names to reflect the proper Java Logging API Loglevels
- please use logFine instead of logDebug
   - please use logSevere instead of logFailure and logError
   See: http://www.yacy-forum.de/viewtopic.php?p=8726#8726

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@615 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-08-30 21:32:59 +00:00
theli
6adf8a4bde *) Renaming Logger function names to reflect the proper Java Logging API Loglevels
- please use logFine instead of logDebug
   - please use logFailure instead of logError
   See: http://www.yacy-forum.de/viewtopic.php?p=8726#8726

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@614 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-08-30 21:10:39 +00:00
allo
60074b4301 more DebugMode(60 Secs Timeout)
needed for Yacy with tor.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@573 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-08-22 14:02:34 +00:00
theli
7d8af6b41a *) Bugfix for heise newsletter Problem
See: http://www.yacy-forum.de/viewtopic.php?p=7836#7836

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@559 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-08-19 08:23:12 +00:00
theli
4335bfe822 *) Using timeout also to establish a connection
See: http://www.yacy-forum.de/viewtopic.php?t=979&highlight=

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@558 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-08-19 06:47:34 +00:00
orbiter
ba0a486328 moved printStackTrace() to logging
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@539 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-08-14 23:35:18 +00:00
jerri
fa154e6ce5 Added some more javadoc into httpc.java. Moved the inner class response to the
end of the class definition, as this eases the reading of the outer class.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@514 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-08-10 21:50:17 +00:00
theli
b32e7c516c git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@507 6c8d7289-2bf4-0310-a012-ef5d649a1542 2005-08-09 09:07:19 +00:00
jerri
09193023fe Began with some documentation for the httpc-class. The code of the httpc-class
looks very disordered? Inner classes and methods mixed together. Maybe the code
should be cleaned up a little bit?


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@503 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-08-07 16:27:38 +00:00
jerri
7792e5ae9b Added an build-target to the ant-configuration to create the
yacy-javadoc-documentation in doc/api. Just do ant create-doc and point your
favourite browser to doc/api/index.html. As most of the classes are not
documented right now this just gives a great overview of all classes.
Hopefully this helps stimulating the creation of
javadoc-insource-documentation.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@502 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-08-07 15:16:33 +00:00
orbiter
2d8557cb10 minor changes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@487 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-08-03 02:02:39 +00:00
rramthun
eacff63eda Typos...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@482 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-08-02 16:09:19 +00:00
orbiter
40da910f41 bugfixes and automatic news-cleanup
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@481 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-08-02 16:03:35 +00:00
orbiter
2f0d7ea8d3 removed htcache stati (superfluous now)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@396 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-09 00:33:34 +00:00
theli
0b95c9c434 *) Bugfix for Thread.getID() usagage + PeerPing-Shutdown Deadlock
See:
   - http://www.yacy-forum.de/viewtopic.php?p=4937
   - http://www.yacy-forum.de/viewtopic.php?p=4939

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@390 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-07 21:00:11 +00:00
theli
13eeaa08f3 *) httpc.java:
- Now it's possible to interrupt pending httpc-actions on server shutdown  
   - this is possible because of a newly introduced registration mechanism for
     open sockets
*) yacyCore.java
   - blocking peerPing threads can now be interrupted on server shutdown
*) serverCore.java
   - restructuring shutdown code 
*) error.html
   - port number is now set correctly if port forwarding was enabled


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@389 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-07 13:58:54 +00:00
orbiter
858cd94299 replaced indexing ram-queue by file-based stack-queue
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@381 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-06 14:48:41 +00:00
theli
57c30f1d78 *) bugfix for usage of httpc without gzip content encoding
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@369 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-04 11:25:25 +00:00
theli
0e2c33ee55 *) Network.html/Network.java:
- Adding function to manually force peer ping to remote yacy peer
  See:Network.html?page=4
- for debugging purpose only!

*) serverAbstractThread.java:
- Adding posibility to notify a server thread via a synchronization object
- this is needed e.g. by the port forwarding feature to send a notification
  to the peerPing thread to redo peer-ping with the new ip/port Settings_p.html

*) Port Forwarding Feature (it should work now)
- adding a serverThread which is responsible to detect broken port forwarding 
  connections and to do reconnect if needed
- serverCore.java: moving port forwarding initialization into a separate function
- adding positility to configure the ssh port 
- moving configuration section on the gui into a separate fieldset
- hello.java: only trying to do a second connect to the clientIp address during
  peer handshake if either remote port forwarding is not enabled locally or
  the clientIP is not equal to any local ip

*) httpdFileHandler.java:
- printout a more verbose errormessage

*) httpc.java
- allowing to deactivate content encoding from outside


 

*) plasmaCrawlWorker.java
- the crawler worker now tries to refetch the content of a website without
  gzip content encoding if a gzip error occured



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@368 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-07-04 11:09:48 +00:00
theli
d53b2393e5 *) autoconfig.java: ip address was not reported correctly when port-forwardin is on
*) hello.java: reportedip my be empty at peer startup
*) httpc.java: adding method to determine if the connection was already closed or is broken
*) httpdProxyHandler.java: trying to do a better errorhandling
*) server/serverCore.java
- setting myseed ip-address and port correctly if port-forwarding is on
- doing a more failsafe close and adding some debugging output
*) yacyClient.java: adding some logging statements to allow a better detection of 
   "degraded to senior"-bug
*) yacyCore.java: restructuring publishMySeed
   (@Orbiter: pleas take a look)
- to avoid buzy waiting
- to allow a gracefull shutdown on server shutdown
- new seed count was not calculated correctly in the previous version
*) yacySeedDB.java: host ip and port was not initialized correctly if port-forwarding
   was activated

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@318 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-23 11:00:26 +00:00
orbiter
3be98f194d tried to find the socket bug
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@300 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-18 01:48:11 +00:00
orbiter
5d06ded005 enhanced html parser speed
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@290 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-17 01:26:51 +00:00
theli
06b0db2cac *) adding toString method to
- httpc
   - response
*) simplifying gzip encoding
*) remembering http version of contacted server
   (neede for later support of keep alive by httpc)
*) moving function shallTransportZipped to httpd.java
   because this function is used multiple times

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@242 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-06-09 09:56:41 +00:00
theli
0e1d9e9722 *) shrinking httpc linebuffer when httpc is returned to pool. This is done to free memory
*) Making Seed-Upload configuration more verbose.
*) Some Changes in SOAP Search API (not finished yet).

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@158 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-23 10:10:51 +00:00
theli
4dd387aae9 *) moving constants (see last commit) to proper httpHeader class
*) migrating fileHandler + proxyHandler to use constants instead of hardcoded values

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@114 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-13 09:14:12 +00:00
theli
aea3479a9c *) introducing of additional constants
to improve maintainability of the sourcecode

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@113 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-13 08:30:30 +00:00
theli
cbdc499ba6 *) adding many missing (File)?(Input|Output)Stream.close() calls to avoid "Too many open files bug".
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@90 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-08 07:24:33 +00:00
theli
2aa5fe8f50 *) Import statements reorganized
Now it's easier to determine which class really uses which other class

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@82 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-05-05 05:32:19 +00:00
orbiter
c7c6aaf06e many bug-fixes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@73 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-04-30 01:22:46 +00:00
theli
3756e6d20f *) "Httpc object was not returned to object pool." bug fixed.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@62 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-04-26 10:38:35 +00:00
orbiter
8b31f9e202 enhanced shut-down behaviour & added experimental nio-wrapper for kelondroRA (not active yet)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@44 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-04-23 13:00:56 +00:00
orbiter
00f223cfc1 fixed post-parsing (a case when the bluelist is empty)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@41 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-04-21 17:13:43 +00:00
theli
c9c0a1f11c *) Trying to speedup local crawling
- introduction of a threadpool for crawling
- introduction of a job queue to avoid buzy waiting for a free crawler slot

*) New classes added
- queue for receiving of crawler jobs
- semaphore class to do reader/writer synchronization (mutual exclusion)
- message object to hold all needed data about a crawler job

*) Trying to solve session-thread shutdown problem
- session thread stopped variable is now set from outside before interrupting the
  session thread.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@39 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-04-21 10:31:40 +00:00
(no author)
942914ffd2 *) Adding additional functions to serverByteBuffer so that it
can be used instead of a ByteArrayOutputStream
*) Using a serverByteBuffer for lineBuffering in class httpc
   instead of a ByteArrayOutputStream

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@35 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-04-20 07:39:40 +00:00
(no author)
4a76ccc6d6 *) Some minor bugfixes
- httpc: wrong error-message on 404
- httpc: error message was accidentally shown when object 
  was released from pool


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@31 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-04-19 10:42:48 +00:00
(no author)
f39812da91 *) Some performance improvements
- many classes set to final
- implementation of a session-thread pool
- reusage of the server handler class (normally the httpd object)
  within the session thread
- implementation of a httpc object pool
- introduction of a linebuffer in httpd which can be reused
- reusing the properties table in the httpc
- added to apache libs (commons-collections, commons-pool) which 
  are needed for the object/thread pool implementation

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@26 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-04-19 06:55:57 +00:00
orbiter
c0807abd33 new crawl/proxy/cache design + fixes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@18 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-04-13 23:00:20 +00:00
orbiter
96516fc9d8 fixed bugs (search+kelondroException, dns)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@16 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-04-13 11:16:31 +00:00
orbiter
072052f150 fixed bugs (dns, seedDB)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@13 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-04-11 22:44:40 +00:00
orbiter
248077d3f0 initial load with yacy 0.36
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-04-07 19:19:42 +00:00