117 Commits

Author	SHA1	Message	Date
borg-0300	8cd6a52dd0	Convention ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@671 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-09-07 07:26:19 +00:00
borg-0300	da9c6857fb	*) changed a misunderstand, no BUG ;) ... *) finals and other git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@668 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-09-06 14:17:53 +00:00
theli	578f36ae18	*) Speedup of indexer. Proxy files will not be enqueued by the cachemanager ... into the sb-queue anymore if the mimeType or fileExtension is not supported by the installed parsers. - Advantage: Avoiding unnecessary enqueueing and dequeueing from queue git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@664 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-09-05 11:17:37 +00:00
theli	1219ef99f0	*) Bugfix for NullpointerException in yacyDebugMode Init ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@663 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-09-05 10:51:15 +00:00
theli	6c722706b7	*) Moving yacyDebugMode intialization to switchboard ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@660 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-09-05 10:34:34 +00:00
theli	4e07828807	*) httpdProxyHandler.java ... - harmonizing proxy exception handling - adding malformed URL + blacklist check for http head method - adding malformed URL check to http post method - chunked encoding is now not used anymore for http post if clients are http/0.9 or http/1.0 clients (same behaviour as already implemented for get) - now an exception will be thrown on internal httpc errors to force an error output to the client or a connection close. This should help to fix the "binary data in browser window" bug *) plasmaSwitchboard.java - fixing the following Bug E 2005/09/03 18:02:42 PLASMA Could not index URL http://mis04.de/FAIL/snot.php: null java.lang.NullPointerException at de.anomic.plasma.plasmaSwitchboard.processResourceStack(plasmaSwitchboard.java:1000) at de.anomic.plasma.plasmaSwitchboard.deQueue(plasmaSwitchboard.java:625) at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at de.anomic.server.serverInstantThread.job(serverInstantThread.java:95) at de.anomic.server.serverAbstractThread.run(serverAbstractThread.java:243) This bug could occure if the cached responseHeader is null - getting the mimeType now from the parsed document instead of the responseHeader because the mimeType could have been changed during content parsing (e.g. because of the mimetypeParser) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@656 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-09-05 10:10:00 +00:00
theli	a47f9238fe	*) Blacklist is now also used by the crawler ... See: http://www.yacy-forum.de/viewtopic.php?t=1069 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@642 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-09-02 12:09:45 +00:00
theli	732a107160	*) Bugfix for "-UNRESOLVED_PATTERN-" Bug on IndexCreateWWWLocalQueue_p.html and "urlEntry.url() == null" Bug ... - Logging message for "urlEntry.url() == null" is now displayed as info - IndexCreateWWWLocalQueue_p.html now detects null entries while looping throug the list and removes them automatically See: - http://www.yacy-forum.de/viewtopic.php?t=532#8781 - http://www.yacy-forum.de/viewtopic.php?t=639 - http://www.yacy-forum.de/viewtopic.php?t=1071 - http://www.yacy-forum.de/viewtopic.php?t=338 - http://www.yacy-forum.de/viewtopic.php?t=980 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@640 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-09-02 09:33:05 +00:00
theli	0471019606	*) IndexCreateIndexingQueue_p.html now also shows indexing jobs that are currently in process ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@633 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-09-01 22:05:20 +00:00
theli	48aaf703cc	*) Adding additional logging output to detect crawling problems ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@625 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-09-01 06:55:21 +00:00
theli	4fd5b95b1f	*) Renaming Logger function names to reflect the proper Java Logging API Loglevels ... - please use logFine instead of logDebug - please use logSevere instead of logFailure and logError See: http://www.yacy-forum.de/viewtopic.php?p=8726#8726 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@615 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-30 21:32:59 +00:00
theli	6adf8a4bde	*) Renaming Logger function names to reflect the proper Java Logging API Loglevels ... - please use logFine instead of logDebug - please use logFailure instead of logError See: http://www.yacy-forum.de/viewtopic.php?p=8726#8726 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@614 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-30 21:10:39 +00:00
borg-0300	bf14e6def5	*) proxyCache, proxyCacheSize can be changed under 'Proxy Indexing' ... - path now are absolute *) move path check from plasmaHTCache to plasmaSwitchboard - only one path check when starting *) small other git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@606 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-30 12:50:30 +00:00
theli	2a081c9ee5	*) Adding additional logging message for "NURL.entry() == null" Bug ... See: http://www.yacy-forum.de/viewtopic.php?p=8446 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@591 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-28 05:39:26 +00:00
theli	b70de495a0	*) Remembering Crawler-isPaused setting ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@586 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-25 09:51:24 +00:00
theli	e569a84dc0	*) Using the same configuration settings for all indexing threads on server Startup ... See: http://www.yacy-forum.de/viewtopic.php?p=8349 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@584 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-24 09:34:04 +00:00
theli	17be77a468	*) Bugfix for "Crawler data will not be removed from htcache if content parsing failed" ... See: http://www.yacy-forum.de/viewtopic.php?t=965&highlight=ramdisk *) Making ACCEPT_LANGUAGE configureable for crawler See: http://www.yacy-forum.de/viewtopic.php?p=8327 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@583 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-24 07:47:42 +00:00
allo	eb6365c069	local Bootstrapping bug. ... use yacyDebugMode=true to allow local bootstrapping git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@572 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-22 12:13:19 +00:00
theli	330eae7cf3	*) Normalizing CrawlerStartURL now before crawling is started ... *) CrawlWorker also does a URL normalization now before following the redirection URL *) CrawlWorker removes redirection URL correctly from noticeURL stack now git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@571 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-21 22:52:46 +00:00
theli	d4a045d7b1	*) Trying to solve "de.anomic.plasma.plasmaSwitchboard.deQueue': null" Bug ... See: http://www.yacy-forum.de/viewtopic.php?p=7791 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@555 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-18 06:40:26 +00:00
orbiter	25f632dbd9	more DHT bugfixes and better logging of DHT effects ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@542 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-16 00:31:15 +00:00
orbiter	5cb00889d9	enhancements to dht selection, search and search presentation ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@540 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-15 01:12:25 +00:00
orbiter	ba0a486328	moved printStackTrace() to logging ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@539 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-14 23:35:18 +00:00
orbiter	cd10370992	several bugfixes and dht selection / logging improvement ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@531 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-14 00:57:30 +00:00
orbiter	c8a7a85ce2	fix for http://www.yacy-forum.de/viewtopic.php?p=7384#7384 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@529 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-13 21:41:22 +00:00
orbiter	7db543a9fa	fixes for several dht misbehaviours ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@524 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-12 22:14:24 +00:00
orbiter	5716f8521d	bug fixes for word ordering and dht index selection ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@521 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-12 14:06:47 +00:00
orbiter	f5259f29e8	word cache behaviour fix and other fixes ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@519 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-11 23:33:19 +00:00
orbiter	2c234e1b82	better log output for search result ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@512 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-10 09:52:50 +00:00
orbiter	248c24b60a	intermission-feature usage in case of local and remote search ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@510 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-09 20:43:37 +00:00
theli	865b9490a2	*) Making DHT Transfer while Crawling configurable ... See: http://www.yacy-forum.de/viewtopic.php?p=6904 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@496 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-06 11:40:12 +00:00
orbiter	2d8557cb10	minor changes ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@487 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-03 02:02:39 +00:00
orbiter	91163db52e	fix for more time-related problems in proxy ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@486 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-03 00:52:32 +00:00
orbiter	40da910f41	bugfixes and automatic news-cleanup ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@481 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-02 16:03:35 +00:00
theli	228b04b499	*) Bugfix for "wrong seed-upload timestamp" problem ... http://www.yacy-forum.de/viewtopic.php?t=817 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@480 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-02 15:36:10 +00:00
theli	470839a16a	*) Crawler/Session pool settings will now be stored properly into configfile ... Bugfix for: - http://www.yacy-forum.de/viewtopic.php?t=502 - http://www.yacy-forum.de/viewtopic.php?t=778 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@477 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-08-02 12:20:03 +00:00
orbiter	1022fbeb65	many YaCyNews fixes ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@461 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-07-31 01:54:46 +00:00
orbiter	13abd8b6e7	added news-creation at crawl start ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@460 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-07-30 11:57:19 +00:00
orbiter	cdbbfd50fb	fixed bad remote crawl behavior ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@453 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-07-29 06:50:36 +00:00
orbiter	81e564edb8	faster crawl profile list cleanup ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@442 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-07-27 14:16:47 +00:00
orbiter	ad90f0ad13	activated RWI distribution to DHT for senior peers (default redundancy 3), necessary now for network growth ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@438 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-07-27 12:51:00 +00:00
orbiter	b9d18d40cb	configuration of proxy idle time in performance menue ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@436 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-07-26 15:17:29 +00:00
orbiter	c64970fa47	re-implemented proxy-busy-check and fixed some other things ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@421 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-07-21 11:17:04 +00:00
orbiter	b73557ed2d	better assortment monitoring and enhanced profile menue ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@416 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-07-20 13:03:41 +00:00
orbiter	9f505af7aa	preparations for bulk remote crawls ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@408 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-07-19 00:26:31 +00:00
orbiter	51962d55bf	added 'PPM', page-per-minute statistics ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@405 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-07-18 00:44:51 +00:00
orbiter	159f795f65	bugfix (null pointer exception in assortments) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@404 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-07-17 22:25:50 +00:00
orbiter	1d2155675b	changed assortment memory cache flush ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@403 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-07-17 21:22:18 +00:00
orbiter	19dbed7cc8	code clean-up ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@401 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-07-12 15:09:35 +00:00
orbiter	40036ba69c	fixed dht transmission; added url-blacklist blocking also for remote search ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@398 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-07-12 00:07:09 +00:00