Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
950 Commits 1 Branch 20 Tags 836 MiB
8d827cdb30
Commit Graph

627 Commits

Author SHA1 Message Date
theli
c8a35a0130 *) Adding new connection tracking page (currently only for incoming connections) 
*) Displaying statistic for incoming connections on status page
*) Bugfix for Loop-Access Bug when trying to access the yacy page while yacy is configured as proxy
   See: http://www.yacy-forum.de/viewtopic.php?p=6826
*) Bugfix for Referer Bug
   See: http://www.yacy-forum.de/viewtopic.php?p=11098#11098
*) Adding reverse Name lookup for yacy-domain names (used by the connection tracking page)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@916 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-12 08:17:43 +00:00
allo
141417821e limitless Accounts (set timelimit = 0) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@914 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-11 20:56:51 +00:00
orbiter
b80b2fbdcc crawling peers now produce waves in network graphic 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@912 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-11 13:06:07 +00:00
orbiter
b45ffecd39 log to fix http://www.yacy-forum.de/viewtopic.php?p=11111#11111 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@911 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-11 07:46:14 +00:00
orbiter
10d3627c90 changed word cache flush scheduling and removed possible locks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@910 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-11 07:06:33 +00:00
orbiter
dced5c761e added 'corona' for crawling peers in network graphic 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@909 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-10 16:21:09 +00:00
orbiter
4cf0d1eb6d less memory occupation in ImagePainter 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@908 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-10 15:02:53 +00:00
allo
6bd9348bf1 small change 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@907 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-10 14:25:18 +00:00
orbiter
aae5af9686 imagePainter may now paint arbitrary large circles; small update to network graphics 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@906 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-10 13:48:25 +00:00
allo
be8e95d86a debug printlns removed 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@905 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-10 13:38:13 +00:00
allo
f1ff33177d reset Timelimits on Daychange 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@904 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-10 13:06:03 +00:00
theli
54a97a7355 *) IfesL: Suppressing "Broken pipe" stacktrace in log file 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@903 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-10 11:31:46 +00:00
theli
a9e25c26e1 *) adding new sendRespondError method to httpd which accepts a template include file 
for individual error messages

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@902 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-10 10:33:09 +00:00
allo
5605cc8018 TimeLimits 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@901 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-10 10:21:25 +00:00
orbiter
6a72f06c40 resizable network picture + greater on click 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@900 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-10 10:08:28 +00:00
orbiter
839db8869c added high/low priority for index adding 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@899 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-10 09:28:28 +00:00
theli
3966b38360 *) Bugfix for userDB time limit calculation 
See: http://www.yacy-forum.de/viewtopic.php?p=10949

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@898 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-10 09:17:19 +00:00
theli
1688be8590 *) plasmaSwitchboard.java 
adding more verbose logging output for db initialization
*) httpdFileHandler.java
   adding cache for servlet response methods


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@897 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-10 09:13:17 +00:00
theli
e3a586d7bd *) Using serverByteBuffer instead of ByteArrayOutputStream 
to speedup httpdFileHandler

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@896 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-10 07:15:57 +00:00
orbiter
e9eb5e4b56 refactoring of index-entity join methods 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@895 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-10 00:45:18 +00:00
orbiter
258fd9eb8e adding missing file for websearch refactoring 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@894 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-10 00:33:25 +00:00
orbiter
77ae30063d refactoring of websearch process 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@893 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-10 00:32:15 +00:00
orbiter
579b22d8ff small update to network drawing 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@892 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-09 23:11:17 +00:00
orbiter
2b5829c3da small fix 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@891 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-09 19:29:25 +00:00
orbiter
e85989510a update to network image; added disconneced peers by disconnection time and changed colors 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@890 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-09 17:20:40 +00:00
orbiter
4c7918f5b5 added shotdown to crawl stacker (moved from 882) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@889 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-09 16:40:44 +00:00
orbiter
2851658c2a re-integrated Martins last change to crawl stacker from svn 882 that I had deleted accidently 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@888 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-09 16:11:41 +00:00
orbiter
c83594528c integrated crawl stacker into thread control 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@887 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-09 15:59:09 +00:00
orbiter
16a49c1c9d fix for graphics generation bug, see http://www.yacy-forum.de/viewtopic.php?p=10987#10987 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@886 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-09 14:46:33 +00:00
orbiter
d666b61b83 fix for news-deletion, see also http://www.yacy-forum.de/viewtopic.php?p=11000#11000 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@885 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-09 14:31:35 +00:00
borg-0300
4180c422e8 cleaned, finals, Properties 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@884 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-09 13:18:48 +00:00
theli
959eefbc4f *) Robots.txt parser/ppt 
cutting of comments at the line end
*) Adding Threadpool for stackCrawl Thread to speedup robots.txt download
   and double url checks

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@882 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-09 04:43:07 +00:00
orbiter
c1c94111b0 added new network picture at Network menu using the new image-servlet method 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@880 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-08 00:37:43 +00:00
orbiter
d98ed41f31 update to image painter and test on http://localhost:8080/imagetest.png 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@878 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-07 14:17:33 +00:00
allo
f65c939a60 userDB Auth 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@874 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-07 13:49:07 +00:00
orbiter
5153ec0f3e update to image painter 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@873 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-07 01:25:39 +00:00
orbiter
1b2db0b52a fix for file-share access; damaged some commits before by me :-( 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@870 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-06 22:30:13 +00:00
orbiter
1a5d98cd6d better imagePainter example and fix for typo http://www.yacy-forum.de/viewtopic.php?p=10920#10920 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@868 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-06 11:51:35 +00:00
borg-0300
afc5ef2819 cleaned, Properties 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@867 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-06 09:41:59 +00:00
orbiter
60ab32c313 removed obsolete htmlPlotter 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@865 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-05 16:47:29 +00:00
orbiter
06f4ac494a image-painter test; try with http://localhost:8080/imagetest.png 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@864 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-05 16:38:16 +00:00
borg-0300
fb27428674 added restart to Status.html 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@863 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-05 16:35:05 +00:00
borg-0300
0dcfc25d98 cleaned, finals, Properties 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@862 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-05 16:25:43 +00:00
borg-0300
4191b21e73 cleaned, finals, Properties 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@858 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-05 13:30:12 +00:00
orbiter
f6cf3967de fix for compile-bug in svn 583 (Martin guck mal ob das richtig ist: fifo oder filo-stack?) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@854 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-05 12:21:30 +00:00
theli
a2fa75e688 *) Asynchronous queuing of crawl job URLs (stackCrawl) 
various checks like the blacklist check or the robots.txt disallow check are now
   done by a separate thread to unburden the indexer thread(s)
   TODO: maybe we have to introduce a threadpool here if it turn out that this single
         thread is a bottleneck because of the time consuming robots.txt downloads

*) improved index transfer
   The index selection and transmission is done in parallel now to improve index 
   transfer performance.
   TODO: maybe we could speed up performance by unsing multiple transmission threads in 
         parallel instead of only a single one.

*) gzip encoded post requests
   it is now configureable if a gzip encoded post request should be send on
   intex transfer/distribution

*) storage Peer (very experimentell and not optimized yet)
   Now it's possible to send the result of the yacy indexer thread to a remote peer 
   istead of storing the indexed words locally. 
   This could be done by setting the property "storagePeerHash" in the yacy config file
   - Please note that if the index transfer fails, the index ist stored locally.
   - TODO: currently this index transfer is done by the indexer thread. 
     To seedup the indexer
     a) this transmission should be done in parallel and
     b) multiple chunks should be bundled and transfered together


*) general performance improvements  
   - better memory cleanup after http request processing has finished
   - replacing some string concatenations with stringBuffers
   - replacing BufferedInputStreams with serverByteBuffer
   - replacing vectors with arraylists wherever possible
   - replacing hashtables with hashmaps wherever possible
   This was done because function calls to verctor or hashtable functions
   take 3 time longer than calls to functions of arraylists or hashmaps.
   TODO: we should take a look on the class serverObject which is inherited from hashmap
         Do we realy need a synchronization for this class?
   TODO: replace arraylists with linkedLists if random access to the list elements is not needed

*) Robots Parser supports if-modified-since downloads now
   If the downloaded robots.txt file is older than 7 days the robots parser tries to
   download the robots.txt with the if-modified-since header to avoid unnecessary downloads
   if the file was not changed. Additionally the ETag header is used to detect changes.

*) Crawler: better handling of unsupported mimeTypes + FileExtension

*) Bugfix: plasmaWordIndexEntity was not closed correctly in 
   - query.java
   - plasmaswitchboard.java

*) function minimizeUrlDB added to yacy.java 
   this function tests the current urlHashDB for unused urls
   ATTENTION: please don't use this function at the moment because
              it causes the wordIndexDB to flush all words into the
              word directory!

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@853 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-05 10:45:33 +00:00
orbiter
01db66dc69 implemented image-servlets. the imagetest will stay there only for a limited time. Now images can be generated on-the-fly from servlets 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@852 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-05 08:40:20 +00:00
orbiter
f6a0e0f162 small bugfix to readFully 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@851 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-05 07:21:26 +00:00
orbiter
6d5d0ac801 bugfix for startup problems 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@850 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-05 00:52:55 +00:00
orbiter
5dc0d41900 bugfix in kelondroRA (hint by Martin) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@847 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-04 10:24:09 +00:00
orbiter
0c3a20d44f more + changed log for better understanding of outOfMemory bug and others 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@846 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-04 00:28:59 +00:00
orbiter
e5f8163203 fixed a bug with news; news moving could lead to shurtcut loop / 100% CPU; appeared when clicked on a 'Profile' news in Network menu 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@845 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-03 23:33:10 +00:00
orbiter
fbb5e36b80 documentation update 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@843 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-03 23:04:11 +00:00
allo
cd77078aa0 old Version restored before Release 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@842 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-03 18:10:05 +00:00
allo
a4b747fe97 ProxyAccounts based on userDB 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@841 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-03 14:26:08 +00:00
theli
0fd9aa6c6e *) Bugfix: supportedFileExt Function didn't detect the file extension correctly because of missing conversion to lower case 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@837 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-03 10:48:41 +00:00
theli
8a33c9b309 *) Bugfix: supportedFileExt Function didn't detect the file extension correctly if there was a dot 
in one of the parent directories of the file.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@836 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-03 10:21:13 +00:00
theli
28c5687ff9 *) Bugfix for "download of non supported file content" via crawler 
See: http://www.yacy-forum.de/viewtopic.php?p=10724#10724

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@835 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-03 08:45:39 +00:00
theli
2b3f964037 *) Bugfix: supportedFileExt Function didn't chop http parameters before trying to detect the file extension 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@834 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-03 08:42:55 +00:00
theli
d388292f24 *) adding function for user accounting which is called after each http request 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@827 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-30 16:02:58 +00:00
theli
c7b7c69484 *) adding some functions to userDB.Entry 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@826 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-30 15:41:01 +00:00
theli
d9c08a47f9 *) Minor changes to userDB 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@824 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-30 15:02:17 +00:00
theli
af7829df25 *) adding some functions to userDB.Entry 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@823 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-30 14:56:50 +00:00
allo
ff1d3d0680 Init of userDB 
Pagelayout of User_p.html


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@822 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-30 13:48:26 +00:00
theli
19648702cc *) adding iterator to userDB 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@819 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-30 13:21:31 +00:00
theli
dd59c1ace9 *) Adding first version of yacy user DB 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@817 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-30 10:28:49 +00:00
theli
595e0c7e56 *) Bugfix for ProxyErrormsg: Wrong base URL 
See: http://www.yacy-forum.de/viewtopic.php?p=9905#9905

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@815 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-30 06:15:22 +00:00
allo
9e1485c13b new Class for UserAccounts 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@813 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-29 20:15:31 +00:00
orbiter
9c4306e41e fixed problem with htcache path 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@811 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-29 00:24:09 +00:00
orbiter
25a59a51ad fixed problem created with last svn commit 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@810 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-28 23:16:00 +00:00
orbiter
0054d3b1a6 added age in network menu 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@809 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-28 22:54:31 +00:00
orbiter
3dd7e90cdd kbytes instead of bytes in performance settings; new default values 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@808 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-28 18:53:41 +00:00
orbiter
1669eaaa1a fixed svn 805 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@807 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-28 14:47:57 +00:00
borg-0300
ca82d690a9 changed in SVN 805 one line too much 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@806 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-28 13:58:42 +00:00
borg-0300
4bb1f849a0 Bugfix for http://www.yacy-forum.de/viewtopic.php?t=1233 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@805 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-28 13:49:57 +00:00
orbiter
2c7b490e30 memory-logging 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@804 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-28 00:52:54 +00:00
orbiter
7fc822a59b changed handling of time-zones 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@801 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-27 16:28:55 +00:00
theli
9b7f37fc37 *) Minor changes 
- more debugging output: storageTime for indexed document is logged now
   - saving memory in plasmaParserDocument.java, plasmaWordIndexEntryContainer.java (not a big deal)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@798 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-27 07:10:24 +00:00
orbiter
3c1d968d29 fix-fix for 792 and small changes in ftpc/download/dir experiments 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@797 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-26 10:36:42 +00:00
theli
b5a8992d29 *) Setting some object fields to final 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@796 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-26 09:39:54 +00:00
theli
5f95a1cf62 *) Bugfix for ProxyErrormsg: Wrong http host header 
See: http://www.yacy-forum.de/viewtopic.php?p=9905#9905

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@795 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-26 08:10:40 +00:00
theli
023be89586 *) Bugfix for "Robots.txt wird immer wieder geladen" 
See: http://www.yacy-forum.de/viewtopic.php?p=10241#10233

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@794 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-26 08:05:59 +00:00
theli
35c6c5ead7 *) Bugfix for "Blacklist und Crawlen" Bug. 
: Crawling continues even if URL is listed in Blacklist
   See: http://www.yacy-forum.de/viewtopic.php?p=10279#10279
   - missing return statement added. Thanks to allo for the
     code review.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@793 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-26 06:51:11 +00:00
orbiter
dc474aa22f various bug-fixes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@792 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-26 01:10:41 +00:00
orbiter
9e2fc7e5fe load balancing of crawl target domains 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@791 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-25 01:09:21 +00:00
orbiter
3fcc95a82c integrated crawl-profiles db in memory-performance monitor 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@788 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-24 00:33:27 +00:00
orbiter
72ce36baba cleanup in kelondroRecords 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@787 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-24 00:14:01 +00:00
theli
fe6a6abc0b *) Adding robots.txt db to Performance Settings for Memory menue 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@785 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-23 01:31:29 +00:00
orbiter
3274ae725e increased cache size of robots database; however, this should be integrated into new memory control 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@784 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-23 00:37:31 +00:00
orbiter
c6d2f50375 changed order of robots and double-check 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@783 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-23 00:18:08 +00:00
orbiter
68d5ff2ef1 added stringbuffer in condenser 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@782 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-22 23:43:45 +00:00
theli
90f02ea455 *) removing metainfo from serverargs 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@780 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-22 23:05:34 +00:00
theli
70a5681a4f *) Bugfix for inactive scp seed uploader 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@779 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-22 23:04:26 +00:00
orbiter
e380d4e55e cleanup (no functional change) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@778 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-22 21:51:21 +00:00
orbiter
de0a58d79c no more sync 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@776 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-22 20:31:55 +00:00
orbiter
495bc8bec6 removed cache-control from low and medium priority caches which reduces memory use and computation overhead 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@774 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-22 20:01:26 +00:00
theli
1dc94e7753 *) Adding support for gzip content-encoding of http post requests 
used to transferRWIs and transferURLs.
   See: http://www.yacy-forum.de/viewtopic.php?t=1167#10020

*) adding yacyVersion.java containing constants defining yacy versions
   that support a given feature.
   Needed to determine if a remote peer is able to decode gzip 
   content-encoded http post bodies properly.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@772 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-22 10:30:55 +00:00
borg-0300
a1f5027a88 finals; 
cleaned;
Properties;

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@770 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-21 23:18:12 +00:00
orbiter
979a3ee3c0 exceptions for better testing of bug http://www.yacy-forum.de/viewtopic.php?p=9852#9852 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@769 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-21 22:58:15 +00:00
orbiter
18d9e1a256 fix for http://www.yacy-forum.de/viewtopic.php?p=10026#10026 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@768 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-21 21:56:39 +00:00