Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
781 Commits 1 Branch 20 Tags 836 MiB
0054d3b1a6
Commit Graph

532 Commits

Author SHA1 Message Date
orbiter
900ab97422 change of memory-allocation blocking value for GC prevention 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@740 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-17 16:15:56 +00:00
orbiter
0ffca99886 added priority-organization to kelondroRecord cache. This should virtually double the cache capacity. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@738 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-16 12:20:59 +00:00
orbiter
2d22626386 automatic switch-off of cache control in kelondroRecords in case that cache is big enough (so that no cache-aging needs to be controled) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@737 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-16 07:50:15 +00:00
orbiter
50a9500035 fixed 100% CPU bug with news queue deletion 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@735 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-15 22:51:41 +00:00
orbiter
cddd9aaa33 fixed SERIOUS bug with kelondroStack; affected all stack processing since 729 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@732 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-15 22:17:51 +00:00
orbiter
416c126815 fix for a profile = null problem and new monitor in crawl queue 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@730 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-15 21:39:37 +00:00
orbiter
19547f1821 changed node manipulation methods in kelondro core to reduce object allocation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@729 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-15 20:45:51 +00:00
orbiter
debb207a74 removed file sync 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@725 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-14 14:25:46 +00:00
orbiter
2148c0cf49 replaced kelondro storage core; much less objects in kelondro cache now; less IO from DB 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@724 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-14 10:10:49 +00:00
theli
dff96601fe *) Bugfix for transferURL: 
URL list index was not incremented properly.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@723 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-14 07:20:14 +00:00
theli
beefddf0e8 *) Adding option which allows to do a Index-Transfer without deletion of index 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@722 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-14 07:14:24 +00:00
rramthun
4036ee812a Updated german language file 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@721 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-13 16:29:59 +00:00
theli
40925f4fb7 *) Improving complete index transfer performance by automatically increasing size of transfered word chunk 
for fast connections (much similar to normal dht behavior) 
   

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@719 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-13 10:29:04 +00:00
theli
91ab4d044b *) Adding automatic retry functionality to complete index transfer function 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@718 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-13 08:32:24 +00:00
theli
a62677f761 *) Adding additional logging output for complete index transfer 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@717 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-13 06:44:38 +00:00
rramthun
10e7d6f02b Bugfix for http://www.yacy-forum.de/viewtopic.php?t=1053 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@713 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-12 15:33:51 +00:00
theli
b991d2e7dd *) Additional logging message for complete index transfer 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@712 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-12 12:02:45 +00:00
theli
3c00c5f6c7 *) Complete Index Transfer 
See: http://www.yacy-forum.de/viewtopic.php?p=9622

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@711 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-12 11:39:32 +00:00
theli
f783061414 *) Changing redirection code from 307 to 302 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@710 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-12 11:38:46 +00:00
theli
a6a8af0f04 *) httpdFileHandler templateCache can now be disabled 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@708 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-12 10:47:27 +00:00
theli
2cb084d426 *) Complete Index Transfer 
See: http://www.yacy-forum.de/viewtopic.php?p=9622

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@707 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-12 10:37:16 +00:00
theli
d1de71e9f6 *) Suppress stacktrace on proxy error for "No route to host Exception" 
See: http://www.yacy-forum.de/viewtopic.php?t=1153

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@704 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-11 20:21:38 +00:00
theli
56160cbd01 *) Bugfix for "YaCy verzählt sich ..." Bug. 
See: http://www.yacy-forum.de/viewtopic.php?p=9559

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@701 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-11 05:26:01 +00:00
orbiter
43b42854a0 fix for null-entries and http://www.yacy-forum.de/viewtopic.php?p=8649 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@699 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-11 03:54:52 +00:00
borg-0300
801e902795 small change 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@698 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-10 17:47:48 +00:00
theli
019cc716db *) Undoing last changes on yacySeed. Seems not to work properly. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@697 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-10 17:46:17 +00:00
theli
177e8af5b7 *) Bugfix for ConcurrentModification in kelondroAbstractRA.writeMap caused by yacySeed.getMap() 
See: http://www.yacy-forum.de/viewtopic.php?p=9523

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@695 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-10 05:59:12 +00:00
theli
d3c923e6b9 *) Bugfix for "ConcurrentModificationException in hello.class" 
See: http://www.yacy-forum.de/viewtopic.php?t=723

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@694 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-10 05:42:30 +00:00
rramthun
02c242ae22 minor changes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@693 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-09 21:01:53 +00:00
theli
44b36d087e *) Implementing a Cache for the servelet template files (.html) 
should help to reduce IO
   See: http://www.yacy-forum.de/viewtopic.php?t=749

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@690 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-09 11:14:22 +00:00
theli
394b4440d2 *) adding isLoggable function to serverLog class 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@689 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-09 11:03:06 +00:00
theli
73ded2f0b6 *) Trying to fix bug for Seed-Upload-Failed 
Bug may be caused because of timing issues
   See: http://www.yacy-forum.de/viewtopic.php?p=9439

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@688 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-09 10:44:32 +00:00
theli
3587407039 *) Fixing problems of list operation if index and queue size are both 0. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@687 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-08 22:27:48 +00:00
theli
51b48a10e8 *) Suppress stacktrace on proxy error for "ValidatorException: No trusted certificate found" 
See: http://www.yacy-forum.de/viewtopic.php?t=1110

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@686 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-08 20:37:01 +00:00
theli
7809b382bf *) Bugfix for Blacklist support for https (only initial connect) 
See: http://www.yacy-forum.de/viewtopic.php?p=9419

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@684 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-08 14:48:32 +00:00
theli
7fe8784231 *) URLs pointing to a server having a private ip addess will not be indexed anymore 
See: http://www.yacy-forum.de/viewtopic.php?p=9408

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@682 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-07 21:38:03 +00:00
rramthun
7c86c36210 undoing one part of the last commit. do not know, why it didn't work... 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@681 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-07 17:40:53 +00:00
rramthun
a79913c6ea updated german language file 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@680 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-07 17:21:49 +00:00
borg-0300
718950c5da small change 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@679 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-07 15:20:12 +00:00
theli
0aafb83edc *) Bugfix for robots.txt isDisallowed Check. 
Setting path to "/" if it is null or empty.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@677 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-07 13:18:34 +00:00
theli
2cd695f376 *) Bugfix path-entries of robots.txt were not decoded correctly 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@676 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-07 11:49:53 +00:00
borg-0300
8260128ee9 changed getFreeSize(); 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@675 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-07 11:22:41 +00:00
theli
f8ad65eae1 *) First trial implementation of robots.txt support 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@674 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-07 11:17:21 +00:00
theli
9444852896 *) Correcting problems if the port number was set to -1, e.g. because of the usage of function 
URL.getPort()

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@673 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-07 09:54:11 +00:00
borg-0300
0a57fbcde5 Added new HashSet filesInUse; 
Added new Function getFreeSize();

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@672 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-07 09:37:00 +00:00
borg-0300
8cd6a52dd0 Convention 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@671 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-07 07:26:19 +00:00
borg-0300
c0e3d18bbf *) remove import java.lang 
*) Added Super()
*) replaced startsWith()
*) cleaned


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@670 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-06 16:58:12 +00:00
borg-0300
b1cd1fa917 cleaned 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@669 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-06 14:56:19 +00:00
borg-0300
da9c6857fb *) changed a misunderstand, no BUG ;) 
*) finals and other

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@668 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-06 14:17:53 +00:00
borg-0300
fbac053c03 small change 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@665 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-05 11:23:48 +00:00
theli
578f36ae18 *) Speedup of indexer. Proxy files will not be enqueued by the cachemanager 
into the sb-queue anymore if the mimeType or fileExtension is not supported
   by the installed parsers.
- Advantage: Avoiding unnecessary enqueueing and dequeueing from queue

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@664 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-05 11:17:37 +00:00
theli
1219ef99f0 *) Bugfix for NullpointerException in yacyDebugMode Init 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@663 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-05 10:51:15 +00:00
allo
9300689dde bugfix *gr* 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@662 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-05 10:46:11 +00:00
theli
e09f1fe8e4 *) IfsL: Suppressing stacktraces on further proxy errors 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@661 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-05 10:45:56 +00:00
theli
6c722706b7 *) Moving yacyDebugMode intialization to switchboard 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@660 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-05 10:34:34 +00:00
allo
ebc39a7b9a minor fixes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@659 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-05 10:34:26 +00:00
low012
e19ededd66 *) prevents problems due to HTML code in profiles (my posting from 05.Sep.2005 http://www.yacy-forum.de/viewtopic.php?t=964) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@657 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-05 10:15:59 +00:00
theli
4e07828807 *) httpdProxyHandler.java 
- harmonizing proxy exception handling
- adding malformed URL + blacklist check for http head method
- adding malformed URL check to http post method
- chunked encoding is now not used anymore for http post if clients
  are http/0.9 or http/1.0 clients (same behaviour as already implemented for get)
- now an exception will be thrown on internal httpc errors to force an error output
  to the client or a connection close. This should help to fix the "binary data in browser window" bug

*) plasmaSwitchboard.java
- fixing the following Bug
  E 2005/09/03 18:02:42 PLASMA Could not index URL http://mis04.de/FAIL/snot.php: null
  java.lang.NullPointerException
	at de.anomic.plasma.plasmaSwitchboard.processResourceStack(plasmaSwitchboard.java:1000)
	at de.anomic.plasma.plasmaSwitchboard.deQueue(plasmaSwitchboard.java:625)
	at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:585)
	at de.anomic.server.serverInstantThread.job(serverInstantThread.java:95)
	at de.anomic.server.serverAbstractThread.run(serverAbstractThread.java:243)
  This bug could occure if the cached responseHeader is null
- getting the mimeType now from the parsed document instead of the responseHeader because the 
  mimeType could have been changed during content parsing (e.g. because of the mimetypeParser)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@656 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-05 10:10:00 +00:00
allo
f90f699ab1 missing package line. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@655 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-05 10:04:27 +00:00
low012
89d7c6e45e *) Bugfix: offset of 1 prevented correct replacement of HTML 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@654 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-05 09:53:00 +00:00
theli
a7256e8f4e *) Adding X-Forwarded-For Header 
See: http://www.yacy-forum.de/viewtopic.php?t=1118&highlight=xforwardedfor
*) httpc.java: Bugfix for incorrect http response statuscode parsing 
   In some situations the statustext whas chopped
*) Adding a lot of fileheaders containing YaCy copyright and license
*) httpd.java: Adding additional debugging http header that should help du detect
   the "binary data in browser window" bug.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@653 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-05 08:01:54 +00:00
allo
06a451768f a simple robotsParser. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@652 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-05 07:35:19 +00:00
borg-0300
81cb8feb15 back to 649 :/ 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@651 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-04 22:03:44 +00:00
borg-0300
5194511e8e *) attempt to find bug 
See: http://www.yacy-forum.de/viewtopic.php?t=1121

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@650 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-04 19:08:51 +00:00
theli
8f3d19b331 *) Suppress stacktrace on proxy error for "Connection reset" 
See: http://www.yacy-forum.de/viewtopic.php?t=1107

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@646 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-03 15:27:51 +00:00
theli
6991b9e2b9 *) Suppress stacktrace on crawler error for "Connection reset" 
See: http://www.yacy-forum.de/viewtopic.php?p=9071

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@645 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-03 15:17:19 +00:00
theli
a20814291f *) Bugfix for "Race condition zwischen httpc und switchboard" 
See: http://www.yacy-forum.de/viewtopic.php?p=9036

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@644 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-03 13:40:32 +00:00
theli
286853fd39 *) Bugfix for "YACY hängt sich beim Beenden auf" Bug 
See: http://www.yacy-forum.de/viewtopic.php?p=8997

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@643 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-03 07:40:51 +00:00
theli
a47f9238fe *) Blacklist is now also used by the crawler 
See: http://www.yacy-forum.de/viewtopic.php?t=1069

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@642 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-02 12:09:45 +00:00
theli
dc0a2d4c11 *) Bugfix for Loader Queue: 
Job count was not displayed correctly
*) IndexingQueue:
- now it's possible to delete single entries from the queue
- now it's possible to clear the whole queue
  See: http://www.yacy-forum.de/viewtopic.php?t=995

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@641 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-02 11:40:40 +00:00
theli
732a107160 *) Bugfix for "-UNRESOLVED_PATTERN-" Bug on IndexCreateWWWLocalQueue_p.html and "urlEntry.url() == null" Bug 
- Logging message for "urlEntry.url() == null" is now displayed as info
   - IndexCreateWWWLocalQueue_p.html now detects null entries while looping throug the list and removes them automatically
   See: 
   - http://www.yacy-forum.de/viewtopic.php?t=532#8781
   - http://www.yacy-forum.de/viewtopic.php?t=639
   - http://www.yacy-forum.de/viewtopic.php?t=1071
   - http://www.yacy-forum.de/viewtopic.php?t=338
   - http://www.yacy-forum.de/viewtopic.php?t=980

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@640 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-02 09:33:05 +00:00
theli
33aaffbfc6 *) Displaying content size of each entry in indexing queue 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@639 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-02 08:22:11 +00:00
allo
022c1ab179 performance fix for yacyDebugMode and useYacyReferer. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@638 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-02 08:21:33 +00:00
allo
286442fbc5 do not Use YaCy-Sites as Referer, if useYacyReferer = false 
http://www.yacy-forum.de/viewtopic.php?p=8896#8896


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@637 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-02 06:26:38 +00:00
theli
f9eb550fbc *) Bugfix for NullpointerException in serverAbstractSwitch.setConfig 
See: http://www.yacy-forum.de/viewtopic.php?t=692#5575

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@636 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-02 03:34:02 +00:00
borg-0300
7626823519 BUGFIX for last 'commit' 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@635 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-01 23:43:27 +00:00
borg-0300
971756e8dd the delete size is smaller 
See: http://www.yacy-forum.de/viewtopic.php?t=1084

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@634 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-01 23:35:00 +00:00
theli
0471019606 *) IndexCreateIndexingQueue_p.html now also shows indexing jobs that are currently in process 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@633 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-01 22:05:20 +00:00
theli
3dc6845bef *) Logging error message to logging output if no errormessage can be send to the user by the proxy 
Note: This is only done if you set the logging level of PROXY to FINE

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@632 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-01 21:06:56 +00:00
borg-0300
6d43a4970c small changes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@631 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-01 15:27:41 +00:00
theli
3df5c7a6cf *) Displaying an proxy error page instead of a white page if the server has closed 
the connection before yacy was able to receive the http response line
   See: http://www.yacy-forum.de/viewtopic.php?p=8866#8866
        http://www.yacy-forum.de/viewtopic.php?t=704

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@630 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-01 11:34:32 +00:00
borg-0300
cc493ef8c1 Added change from Hermes 
See: http://www.yacy-forum.de/viewtopic.php?t=1050

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@629 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-01 11:18:41 +00:00
theli
4edb5b6f1e *) Bugfix for "ProxyAccess logging" Bug 
Loglevel was not set corretly for Proxy.access logger
   See: http://www.yacy-forum.de/viewtopic.php?p=8875#8875

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@628 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-01 10:06:47 +00:00
theli
7a7254713d *) Moving Logging directory per default to DATA/LOG 
See: http://www.yacy-forum.de/viewtopic.php?t=940#7656

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@627 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-01 08:56:35 +00:00
theli
bead8a32aa *) IndexCreate_p.java: 
Crawler StartURLs will now also added to the errorURL-DB if an error occures on this url
*) kelondroStack.java, plasmaSwitchboardQueue.java
   Adding method which returns a list of all entries in the queue. This list is used by IndexCreate_p.java 
   instead of an iterator to display the indexing-list. 
   Advantages: avoid concurrent modifications of the list while displaying it. 
               Speedup because now we have to access only one sync function instead of multiple ones 
               (one for each entry)
*) IndexCreateIndexingQueue_p.java
   Using new list() function of plasmaSwitchboardQueue
*) httpdFileHandler.java
   If a servelet returns the special value "LOCATION" the httpFileHandler does a Redirection of 
   the Browser to the URL specified by the servelet. This can e.g. be used when a http get request is
   used insead of a post request, but a refresh should not be allowed.
*) IndexCreateWWWLocalQueue_p.html
   Now it's possible to delete single entries of the local crawler queue

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@626 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-01 07:52:46 +00:00
theli
48aaf703cc *) Adding additional logging output to detect crawling problems 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@625 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-01 06:55:21 +00:00
theli
59b8a98c7e *) Bugfix for suppressing of stacktrace in log on crawler error "MalformedURLException" 
See: http://www.yacy-forum.de/viewtopic.php?p=8840

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@623 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-09-01 06:31:30 +00:00
low012
2ee4f9f4e5 *) added escape characters [= and =] to yacyWiki 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@622 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-08-31 14:43:55 +00:00
borg-0300
c1d7527929 better cache cleanup 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@621 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-08-31 13:07:08 +00:00
theli
2e6df95786 *) adding toString method 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@620 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-08-31 10:43:03 +00:00
theli
ebbd063c92 *) Making mimeTable static final 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@619 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-08-31 09:22:55 +00:00
theli
60e77dcc60 *) Adding additional loglevel finer + finest according to Thread http://www.yacy-forum.de/viewtopic.php?p=8750#8750 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@618 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-08-30 22:37:48 +00:00
borg-0300
fa54b5f38d cleanup spaces 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@617 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-08-30 22:24:38 +00:00
theli
4fd5b95b1f *) Renaming Logger function names to reflect the proper Java Logging API Loglevels 
- please use logFine instead of logDebug
   - please use logSevere instead of logFailure and logError
   See: http://www.yacy-forum.de/viewtopic.php?p=8726#8726

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@615 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-08-30 21:32:59 +00:00
theli
6adf8a4bde *) Renaming Logger function names to reflect the proper Java Logging API Loglevels 
- please use logFine instead of logDebug
   - please use logFailure instead of logError
   See: http://www.yacy-forum.de/viewtopic.php?p=8726#8726

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@614 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-08-30 21:10:39 +00:00
theli
f19c09b227 *) Suppress stacktrace on crawler error for "MalformedURLException" 
See: http://www.yacy-forum.de/viewtopic.php?p=8733#8733

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@613 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-08-30 20:25:07 +00:00
theli
cc1df08069 *) Adding missing synchronized blocks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@608 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-08-30 14:57:32 +00:00
theli
0dfa8b62e2 *) Changing Proxy-Useragent string according to thread http://www.yacy-forum.de/viewtopic.php?p=8183#8183 
A typical useragent string now e.g. looks like: 
   Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.10; YaCy 0.401/00602; yacy.net) Gecko/20050716 Firefox/1.0.6

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@607 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-08-30 13:41:47 +00:00
borg-0300
bf14e6def5 *) proxyCache, proxyCacheSize can be changed under 'Proxy Indexing' 
- path now are absolute
*) move path check from plasmaHTCache to plasmaSwitchboard
   - only one path check when starting
*) small other

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@606 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-08-30 12:50:30 +00:00
theli
3dfda1c9da *) More verbose output on ftp-seed-upload failure 
See: http://www.yacy-forum.de/viewtopic.php?p=8000#8000

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@605 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-08-30 12:18:41 +00:00