orbiter
92f4cb4d73
added option to configure the start-up delay time for kelondro database files.
...
the start-up delay is used to pre-load the database node cache
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2276 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-03 23:57:33 +00:00
orbiter
11951aed41
replaced node cache Object type in kelondroTree:
...
kelondroCollectionObjectMap replaces HashMap.
the kelondroCollectionObjectMap uses only 1/4 of the memory of HashMap,
but is slightly slower than the HashMap.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2213 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-06-18 12:04:41 +00:00
orbiter
370c481fa7
bugfixes
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2171 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-06-02 22:46:32 +00:00
orbiter
c36e9fc8d3
full integration of kelondroRow
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2167 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-06-02 12:45:57 +00:00
orbiter
4a907a570f
1st step to migrate kelondroTree to usage of kelondroRow instead of byte[][]
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2162 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-05-31 23:31:46 +00:00
orbiter
572d53506c
new kelondroRow objects now replace byte[][] objects in object cache
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2161 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-05-30 22:39:19 +00:00
orbiter
5bb565944f
integration of new kelondroRow into some parts of kelondro,
...
especially into the array storage
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2155 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-05-30 14:36:20 +00:00
orbiter
24686e50a2
- fixed a caching bug
...
- added counter for cache delete to distinguish between flush and delete
- changed some default paramenters for cache size settings
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2143 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-05-25 11:46:09 +00:00
orbiter
bd057b44dd
- automatic setting of peer-does-not-accept-remote-crawl
...
- increased percentage of object cache to node cache to 30%
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2136 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-05-23 22:03:09 +00:00
orbiter
cda087f43b
- integrated cache miss storage into object cache
...
- removed cache-miss handling from indexURL
todo: new Monitoring in PerformanceMemory_p
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2132 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-05-23 16:43:28 +00:00
orbiter
3e31820c3d
- corrections to PerformanceMemory display of object cache
...
- configuration of object cache size in kelondroTree initializer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2075 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-05-10 09:08:42 +00:00
orbiter
29b1b0823c
added monitoring of new object cache to performanceMemory page
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2072 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-05-09 10:03:12 +00:00
orbiter
23ced30e83
fixed a bug in new caching
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2071 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-05-08 21:29:28 +00:00
orbiter
bab9ee38e7
added very experimental object cache for all kelondroTree
...
databases. This cache now takes up 10% of all cache sizes,
90% goes to the node routing cache we had so far.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2069 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-05-08 13:37:27 +00:00
orbiter
e9aadb2e11
quickfix for http://www.yacy-forum.de/viewtopic.php?p=19765#19765
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2041 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-04-25 15:21:17 +00:00
orbiter
a37b09e303
implemented automatic adoption of chunk-read-ahead in kelondroTree to needed chunk size
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1931 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-19 01:27:33 +00:00
orbiter
526407f32e
adoptions, fixes for last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1919 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-18 00:21:34 +00:00
orbiter
488a0ed580
replaced old keyIterator and rowIterator by buffered iterators
...
that are synchronized with database access
Main change is done in kelondroTree, other classes are only adoptions
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1918 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-17 23:43:24 +00:00
orbiter
dba02f399f
starting of re-design of kelondroTree iterator
...
- new access to iterator
- added many IOException handling in other Classes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1914 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-17 20:52:43 +00:00
orbiter
f02b426073
made kelondroTree.nodeIterator private
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1910 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-17 18:10:48 +00:00
orbiter
431a4f3609
eliminated correcting iterator in kelondroTree
...
VERY EXPERIMENTAL! NOT TESTED!
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1907 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-17 14:20:04 +00:00
orbiter
f0464042fc
fix for latest iterator-replacement-fix:
...
iterator generated TreeSet which did not resprect rotations
this has now be implemented using kelondroOrder Objects
and by adding this rotation-rules to the ording
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1895 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-16 16:44:29 +00:00
orbiter
ba5fe0b287
fixed (again) nullPointerException in kelondroTree
...
see: http://www.yacy-forum.de/viewtopic.php?p=17091#17091
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1656 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-15 21:33:10 +00:00
orbiter
0017812b2e
bugfix for null compare
...
see http://www.yacy-forum.de/viewtopic.php?p=16980#16980
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1644 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-14 22:18:45 +00:00
orbiter
4438a96b87
tried to fix "Object type or Object type combination not supported" bug
...
see http://www.yacy-forum.de/viewtopic.php?p=16588#16588
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1592 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-09 22:32:18 +00:00
hermens
56936139ae
Fix logging in kelondroTree:
...
*) Use java.util.logging.* (via kelondroRecords) without de.anomic.server.logging.serverLog
*) Log "CORRECTING ITERATOR" only in debug mode
(happens way too often to be a useful warning and is corrected anyway)
*) Log when correctedNodeIterator stumbles on incorrectly ordered data
(now this one deserves throwing a warning at the user)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1503 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-31 02:01:32 +00:00
orbiter
03c65742ba
changes towards the new index storage scheme:
...
- replaced usage of temporary IndexEntity by EntryContainer
- added more attributes to word index
- added exact-string search (using quotes in query)
- disabled writing into WORDS during search; EntryContainers are used instead
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1485 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-30 00:42:38 +00:00
rramthun
84a00e5673
Use YaCy logging instead of something I don't understand.
...
Problem was: YaCy under Linux wrote every CORRECTING ITERATOR message to syslog an your logfiles get VERY big if you run YaCy 24/7.
Approx. 20MB/day.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1483 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-29 16:04:20 +00:00
orbiter
b9d73f63e7
replaced String object in loop detection by byte[] to omit String-generation
...
which could cause locks.
See http://www.yacy-forum.de/viewtopic.php?p=15738#15738
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1425 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-24 13:12:38 +00:00
orbiter
bb2095fe39
assortment files are now not deleted, but shifted to a backup directory.
...
See also: http://www.yacy-forum.de/viewtopic.php?p=15458#15458
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1394 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-22 01:03:01 +00:00
orbiter
7366e39dd3
tried to fix 100% CPU bug.
...
See http://www.yacy-forum.de/viewtopic.php?p=15569#15569
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1393 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-22 00:50:05 +00:00
orbiter
c51e18c4ad
- added signatures to ordering
...
- added storage of orderings to database
- orderings can be read from database when opening a database
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1357 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-17 00:31:27 +00:00
orbiter
38b19ff56a
another bugfix to the correctedNodeIterator
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1314 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-12 15:19:09 +00:00
orbiter
45c44ca9c8
bugfix for http://www.yacy-forum.de/viewtopic.php?t=1717
...
(no rotation)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1313 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-12 07:24:03 +00:00
orbiter
c137683a5c
added patch to correct tree-iterator
...
(symptoms: at the end of the iteration,
more elements are iterated that are not in the
correct order)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1312 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-11 13:47:16 +00:00
orbiter
2028403670
- consolidated different orderings to kelondroNaturalOrder
...
- added another iteration method to rwihash-enumeration
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1309 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-11 00:32:44 +00:00
orbiter
7c0d7ed4f8
introduction of generic order interface for kelondro
...
this is needed for (to be introduced) splitted tree-files
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1283 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-03 12:35:39 +00:00
orbiter
60fc34d9e8
iterartion bug:
...
http://www.yacy-forum.de/viewtopic.php?p=14182#14182
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1235 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-12-19 00:40:06 +00:00
orbiter
676e384b45
changed synchronization to fix 'contains loops' bug in kelondroTree
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1208 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-12-13 11:58:55 +00:00
orbiter
0c762daf4b
better startup failure handling
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1205 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-12-12 23:59:58 +00:00
orbiter
356231b033
changed only for testing with new hastable implementation.
...
Core funktions are unchanged.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1192 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-12-08 21:38:34 +00:00
orbiter
1041f4946a
experiments with locks to remove synchronization
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1189 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-12-08 12:30:55 +00:00
orbiter
288b2c353e
debugging of kelondroRecords ... possibly synchronization bugs found
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1188 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-12-08 02:18:44 +00:00
orbiter
37f88b4017
code cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1176 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-12-06 23:51:29 +00:00
orbiter
ec2b39c1ce
code cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1175 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-12-06 22:30:15 +00:00
orbiter
76618442e0
code cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1173 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-12-06 21:21:14 +00:00
orbiter
3d8a5ae652
code cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1166 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-12-05 14:24:13 +00:00
orbiter
a04930f025
code cleanup
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1158 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-12-04 23:51:28 +00:00
orbiter
420d56ce79
extended db-testing
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1152 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-12-02 01:51:37 +00:00
orbiter
85282b1d98
enhanced YBR recognition and search result heuristics
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1121 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-23 01:40:02 +00:00