316 Commits

Author	SHA1	Message	Date
orbiter	bd4f43cd66	- fixed a null pointer exception bug ... - switched off more write caches - re-enabled index-abstracts search git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2885 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-31 02:45:41 +00:00
orbiter	fe8afaf426	switched off usage of write cache for imprortant databases ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2883 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-30 02:59:22 +00:00
orbiter	985fd807cc	bugfixing in collection methods ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2882 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-30 02:39:39 +00:00
orbiter	e6044e5198	bugfix for ... http://www.yacy-forum.de/viewtopic.php?p=27207#27207 and http://www.yacy-forum.de/viewtopic.php?p=27219#27219 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2875 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-28 21:43:12 +00:00
orbiter	ebd2d629d8	added missing file for last commit ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2866 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-26 13:53:00 +00:00
orbiter	147d88cf23	re-design of database caching ... this should reduce IO a lot, because write caches are now actived for all databases - added new caching class that combines a read- and write-cache. - removed old read and write cache classes - removed superfluous RAM index (can be replaced by kelonodroRowSet) - addoped all current classes that used the old caching methods - more asserts, more bugfixes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2865 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-26 13:50:50 +00:00
orbiter	f21ede312e	bugfixes for internals of database organization ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2860 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-25 01:21:05 +00:00
orbiter	eb4bfb0e9d	fixed problem with cache.profile() ... see also: http://www.yacy-forum.de/viewtopic.php?p=27109#27109 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2859 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-24 22:34:13 +00:00
orbiter	2a9d868f6d	- removed object cache from kelondroTree ... - generalized object caching and added new object caching class - added object caching wherever kelondroTree was used - added object caching also to usage of kelondroFlex - added object buffering (a write cache) to NURLs - added many assert statements; fixed bugs here and there - added missing close methods to latest added classes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2858 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-24 13:48:16 +00:00
orbiter	dc056fabf3	small bugfix ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2847 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-23 01:22:50 +00:00
orbiter	278d8c3c7e	- more asserts ... - bugfix for reading of previously deleted nodex git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2845 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-23 00:59:55 +00:00
orbiter	83a0efc65a	better assert statements and fixes ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2833 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-21 10:50:30 +00:00
orbiter	2025e885d6	a fix for problems with remove situations in kelondroFlexSplitTable ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2831 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-21 00:19:00 +00:00
orbiter	06854988da	- full integration of new LURL database in INDEX ... - added migration method for urlHash.db into INDEX git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2819 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-19 21:14:37 +00:00
orbiter	b79e06615d	- added new LURL.Entry class for next database migration ... - refactoring of affected classes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2802 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-18 22:25:07 +00:00
orbiter	77a59a115d	refactoring of indexing methods ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2787 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-16 15:04:16 +00:00
orbiter	14490f0a83	added missing flush statement ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2786 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-16 09:42:35 +00:00
orbiter	688cbfb776	- bugfixing for flextable bug ... - bugfixing for collection index bug - several other bugfixes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2785 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-16 00:27:25 +00:00
orbiter	29a1318ef9	bugfixes for wrong database access that do not consider deleted entries ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2767 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-13 22:57:47 +00:00
orbiter	50f2578c55	- some bugfixing and code cleanup ... - now assortments can completely left out if they do not exist before startup and collection index is selected. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2757 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-13 01:19:26 +00:00
orbiter	bdf4c7c51e	added missing files for last commit ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2756 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-12 23:17:16 +00:00
orbiter	a5dd0d41af	- refactoring of plasmaCrawlLURL.Entry to prepare new Entry format ... - added test migration method to migrate the old LURL to a new LURL the new LURL will be splitted into different tables for each month this solves several problems: - the biggest table in YaCy is splitted in different parts and can also be managed in filesystems that are limited to 2GB - the oldest entries can easily be identified, used for re-crawl und deleted - The complete database can be limited to a specific size (as wanted many times) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2755 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-12 23:14:41 +00:00
orbiter	130cc76927	loop detection and termination in deletedHandles method ... see also: http://www.yacy-forum.de/viewtopic.php?p=26655#26655 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2754 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-12 19:50:09 +00:00
orbiter	6396f5971e	bugfixes and migration attempt toward new kelondroFlex db ... - more synchronization - bugfix for remove in collections - bugfix in kelondroFlex (wrong exception condition!) - options to use RAM, FLEX and TREE tables for Crawl URL stacker - default for Crawl URL stacker is now FLEX (!) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2746 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-11 00:46:45 +00:00
orbiter	86047f439d	removed very bad bug that prevented production of any remote search result ... :-((( Please update! git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2724 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-09 04:04:00 +00:00
orbiter	43614f1b36	bugfix in collection index. the index for collections was not created correctly ... The bugfix includes a migration function which starts automatically after startup of yacy. This applies only to you, if you are using the new collection index. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2711 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-05 23:47:08 +00:00
orbiter	db294687ea	enhanced logging ... - more logging output - fix in log line preparation - added filter to log page - some small bugfixes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2707 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-03 22:55:59 +00:00
orbiter	d4c239e4be	- fixed problem in collection index with deletion of single url references ... - added automatic deletion of not-found snippets after search git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2689 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-02 01:40:52 +00:00
orbiter	b033a80750	better control of failure in node seek of kelondroTree ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2686 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-10-02 00:13:19 +00:00
orbiter	df1629b05a	- code cleanup ... - version 0.471 - moved surftipps to own web page git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2676 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-09-29 22:27:20 +00:00
orbiter	ec031eb993	first version of surftipps ... see http://localhost:8080/index.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2627 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-09-18 20:14:21 +00:00
borg-0300	b174fbd0ca	"import ...*" removed; ... properties added; git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2626 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-09-18 18:31:27 +00:00
orbiter	807756150e	patch for strange bug reported by email ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2625 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-09-18 16:50:31 +00:00
orbiter	cf9884e22b	first attempt to implement a secondary search ... this is a set of search processes that shall enrich search results with specialized requests to realize a combination of search results from different peers. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2571 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-09-13 17:13:28 +00:00
orbiter	75b198bc02	- updated references to indexContainer ... - more bugfixes and debugging for indexAbstract processing git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2555 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-09-12 11:13:27 +00:00
orbiter	74d1dea30b	changes towards better join-search ... - added generation of a compressed index within remote peers during global search - added selection of specific urls within remote peers during secondary global search git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2539 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-09-10 22:36:47 +00:00
orbiter	cc97a3e9c6	fixed possibly bug with indexOutOfBoundsException ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2528 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-09-08 20:27:45 +00:00
orbiter	75b03a4580	fix for new ArrayIndexOutOfBoundException ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2524 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-09-08 19:15:55 +00:00
orbiter	96c6e4e322	- enhancements to detailed search page ... - enhancements to search ranking computation process - removed bugs in postranking git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2516 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-09-08 01:26:06 +00:00
orbiter	9340dbb501	fixed all possible problems with nullpointer exception for LURLs ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2513 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-09-07 18:24:39 +00:00
orbiter	a2525072f2	bugfix for kelondroRow - property generation ... this bug affected ranking parameters :-( git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2506 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-09-07 10:55:34 +00:00
orbiter	985dcbde7f	changed some parameters that may cause better memory usage and more indexing speed ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2457 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-08-24 23:39:52 +00:00
orbiter	b7f4a1521b	added options to switch on or off the kelondroFlexTable for NURL, EURL and PreNURL ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2456 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-08-24 22:21:22 +00:00
orbiter	c26da4893b	turned back NURL usage of kelondroTree, kelondroFlexTable has still problems with deleted entries ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2454 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-08-24 10:03:38 +00:00
orbiter	db1eae0227	* simplified initialization of database objects ... * replaced kelondroTree for NURLs by kelondroFlex * replaced kelondroTree for EURLs by kelondroFlex take care, may be very buggy please finish crawls before updating. crawls will be lost. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2452 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-08-24 02:19:25 +00:00
orbiter	135e019883	removed one superfluous line from last commit ... (hasnot is included in remove) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2435 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-08-21 01:59:44 +00:00
orbiter	1591a55963	added object cache miss-cache use for remove method ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2434 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-08-21 01:51:27 +00:00
orbiter	23dd972608	fixed memory calculation in performanceMemory web page ... fixed also maximum cache size computation git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2429 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-08-20 01:20:34 +00:00
orbiter	3e9d509c39	some small fixes ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2425 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-08-18 22:50:05 +00:00
orbiter	0187c60010	because of a bug in the JRE 1.4.2 there was no memory protection ... see http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4686462 this commit fixes the bug by using a memory-computation patch. All uses of Runtime.maxMemory had been replaced by serverMemory.max The bug is not present any more in Java 1.5 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2419 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-08-18 01:33:54 +00:00