166 Commits

Author	SHA1	Message	Date
orbiter	23ced30e83	fixed a bug in new caching ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2071 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-05-08 21:29:28 +00:00
orbiter	bab9ee38e7	added very experimental object cache for all kelondroTree ... databases. This cache now takes up 10% of all cache sizes, 90% goes to the node routing cache we had so far. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2069 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-05-08 13:37:27 +00:00
orbiter	e9aadb2e11	quickfix for http://www.yacy-forum.de/viewtopic.php?p=19765#19765 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2041 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-04-25 15:21:17 +00:00
orbiter	5defe97ccb	fixed a problem / possible bug with ordering implementation (did not rotate always) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1983 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-03-31 00:07:56 +00:00
rramthun	250864406f	... ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1955 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-03-23 20:24:53 +00:00
orbiter	a37b09e303	implemented automatic adoption of chunk-read-ahead in kelondroTree to needed chunk size ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1931 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-03-19 01:27:33 +00:00
orbiter	526407f32e	adoptions, fixes for last commit ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1919 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-03-18 00:21:34 +00:00
orbiter	488a0ed580	replaced old keyIterator and rowIterator by buffered iterators ... that are synchronized with database access Main change is done in kelondroTree, other classes are only adoptions git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1918 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-03-17 23:43:24 +00:00
orbiter	dba02f399f	starting of re-design of kelondroTree iterator ... - new access to iterator - added many IOException handling in other Classes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1914 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-03-17 20:52:43 +00:00
orbiter	f02b426073	made kelondroTree.nodeIterator private ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1910 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-03-17 18:10:48 +00:00
orbiter	431a4f3609	eliminated correcting iterator in kelondroTree ... VERY EXPERIMENTAL! NOT TESTED! git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1907 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-03-17 14:20:04 +00:00
orbiter	9cca36a107	no more strict comparator checking in map exclude method if not needed ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1901 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-03-16 22:46:32 +00:00
orbiter	4239db0d1c	fixed new ordering for backup iterator TreeSet ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1899 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-03-16 22:20:28 +00:00
orbiter	f0464042fc	fix for latest iterator-replacement-fix: ... iterator generated TreeSet which did not resprect rotations this has now be implemented using kelondroOrder Objects and by adding this rotation-rules to the ording git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1895 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-03-16 16:44:29 +00:00
orbiter	bcd99fe83e	introduced a second RAM cache for DHT transfer ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1880 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-03-13 10:43:12 +00:00
orbiter	69d26045eb	quick-fix for http://www.yacy-forum.de/viewtopic.php?p=18805#18805 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1876 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-03-10 20:45:14 +00:00
orbiter	be88687d8c	fixed some problems with new cache flush karenz ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1873 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-03-10 13:57:30 +00:00
orbiter	c51603a405	added two new kelondro classes that will handle the new index data structures (not used yet) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1789 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-03-01 00:25:02 +00:00
orbiter	ba5fe0b287	fixed (again) nullPointerException in kelondroTree ... see: http://www.yacy-forum.de/viewtopic.php?p=17091#17091 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1656 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-02-15 21:33:10 +00:00
orbiter	0017812b2e	bugfix for null compare ... see http://www.yacy-forum.de/viewtopic.php?p=16980#16980 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1644 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-02-14 22:18:45 +00:00
orbiter	01c668fe1b	experimental disabling of db write buffer ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1605 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-02-11 11:31:44 +00:00
orbiter	4438a96b87	tried to fix "Object type or Object type combination not supported" bug ... see http://www.yacy-forum.de/viewtopic.php?p=16588#16588 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1592 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-02-09 22:32:18 +00:00
orbiter	c70befbe9c	added xml read/write example file: kelondroXMLTables ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1591 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-02-09 22:16:46 +00:00
orbiter	785763d070	added debug output for "Object type or Object type combination not supported" bug. ... See also: http://www.yacy-forum.de/viewtopic.php?p=16306#16306 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1568 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-02-07 22:48:22 +00:00
hermens	56936139ae	Fix logging in kelondroTree: ... *) Use java.util.logging.* (via kelondroRecords) without de.anomic.server.logging.serverLog *) Log "CORRECTING ITERATOR" only in debug mode (happens way too often to be a useful warning and is corrected anyway) *) Log when correctedNodeIterator stumbles on incorrectly ordered data (now this one deserves throwing a warning at the user) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1503 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-31 02:01:32 +00:00
orbiter	3419b3bcdd	fix for bug that caused the peer-counter problem. ... See http://www.yacy-forum.de/viewtopic.php?p=16016#16016 The kelondroDyn now uses a generic fill character. kelondroDyn-Tables containing peer/word/url-hashes must not use '_' as fill character. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1498 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-30 22:16:55 +00:00
orbiter	03c65742ba	changes towards the new index storage scheme: ... - replaced usage of temporary IndexEntity by EntryContainer - added more attributes to word index - added exact-string search (using quotes in query) - disabled writing into WORDS during search; EntryContainers are used instead git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1485 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-30 00:42:38 +00:00
rramthun	84a00e5673	Use YaCy logging instead of something I don't understand. ... Problem was: YaCy under Linux wrote every CORRECTING ITERATOR message to syslog an your logfiles get VERY big if you run YaCy 24/7. Approx. 20MB/day. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1483 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-29 16:04:20 +00:00
orbiter	b9d73f63e7	replaced String object in loop detection by byte[] to omit String-generation ... which could cause locks. See http://www.yacy-forum.de/viewtopic.php?p=15738#15738 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1425 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-24 13:12:38 +00:00
orbiter	47843e69e2	auto-reset for switchboard queue stack ... bugfix for http://www.yacy-forum.de/viewtopic.php?p=15684#15684 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1414 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-23 12:41:08 +00:00
orbiter	a70970f993	fixed increment in content iterator ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1413 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-23 12:11:32 +00:00
orbiter	d6581c445b	added content iterator for corrupted database files ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1406 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-22 17:12:16 +00:00
orbiter	bb2095fe39	assortment files are now not deleted, but shifted to a backup directory. ... See also: http://www.yacy-forum.de/viewtopic.php?p=15458#15458 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1394 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-22 01:03:01 +00:00
orbiter	7366e39dd3	tried to fix 100% CPU bug. ... See http://www.yacy-forum.de/viewtopic.php?p=15569#15569 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1393 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-22 00:50:05 +00:00
orbiter	0371494010	tried to add word position to index ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1377 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-19 14:13:39 +00:00
hermens	84fce71d11	*) correct merging of indexes when enumerating downward ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1361 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-17 11:44:55 +00:00
orbiter	d45b1162e2	signature fix ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1358 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-17 00:43:38 +00:00
orbiter	c51e18c4ad	- added signatures to ordering ... - added storage of orderings to database - orderings can be read from database when opening a database git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1357 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-17 00:31:27 +00:00
orbiter	38b19ff56a	another bugfix to the correctedNodeIterator ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1314 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-12 15:19:09 +00:00
orbiter	45c44ca9c8	bugfix for http://www.yacy-forum.de/viewtopic.php?t=1717 ... (no rotation) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1313 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-12 07:24:03 +00:00
orbiter	c137683a5c	added patch to correct tree-iterator ... (symptoms: at the end of the iteration, more elements are iterated that are not in the correct order) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1312 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-11 13:47:16 +00:00
orbiter	2028403670	- consolidated different orderings to kelondroNaturalOrder ... - added another iteration method to rwihash-enumeration git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1309 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-11 00:32:44 +00:00
orbiter	9544c47684	added some UTF-8 handling. ... hope this will help somehow.. for shure not THE solution to our UTF-8 problem git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1308 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-10 16:48:59 +00:00
orbiter	a125fee9ce	fixed iterator ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1294 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-09 00:55:47 +00:00
orbiter	33f8920e1c	new database format: splitted trees, using an ordering to split the tree ... The new format can be tested with the dbtest git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1293 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-09 00:46:56 +00:00
orbiter	10a095d6b5	- added base64 ordering methods ... - added coding interface git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1285 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-04 01:45:23 +00:00
orbiter	9086261476	refactoring of base64 encoding: ... the kelondro database needs specific information about the order of base64-encoded keys. Since no other package depends on base64 (only the httpd uses base64 for encryption, but does not need to encode these strings) it is good to move base64 encoding to the new ordering classes in kelondro. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1284 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-04 00:39:00 +00:00
orbiter	7c0d7ed4f8	introduction of generic order interface for kelondro ... this is needed for (to be introduced) splitted tree-files git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1283 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-01-03 12:35:39 +00:00
orbiter	60fc34d9e8	iterartion bug: ... http://www.yacy-forum.de/viewtopic.php?p=14182#14182 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1235 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-12-19 00:40:06 +00:00
rramthun	45ebe477ee	Bugfix for http://www.yacy-forum.de/viewtopic.php?p=14175 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1224 6c8d7289-2bf4-0310-a012-ef5d649a1542	2005-12-17 11:13:44 +00:00