Commit Graph

495 Commits

Author SHA1 Message Date
theli
ab7a911bb3 *) Trying to solve pool not open problem
See: http://www.yacy-forum.de/viewtopic.php?t=1798

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1482 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-29 08:54:19 +00:00
hydrox
d665f3c39c *) fixed Threadnames for stackCrawl-Threads
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1480 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-28 19:06:21 +00:00
theli
3d5347bc8e *) changing loglevel for some messages
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1479 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-28 17:40:24 +00:00
theli
0fcd113c42 *) last bugfix part. Seems to work now for the stackCrawler
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1478 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-28 17:25:19 +00:00
theli
b9c9eaeb44 *) next try todo a bugfix :-((
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1477 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-28 17:05:47 +00:00
theli
4b4b93c413 *) next try todo a bugfix :-(
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1476 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-28 16:55:05 +00:00
theli
d9fbad71b9 *) next try todo a bugfix
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1475 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-28 16:38:25 +00:00
theli
6da97bd2e4 *) next bugfix for threadpool problem
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1474 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-28 16:31:40 +00:00
theli
bea2b9edee *) further redesign of threadpools to solve too many thread problem
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1473 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-28 16:18:07 +00:00
theli
784fd50437 *) more verbose thread names
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1471 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-28 15:26:47 +00:00
theli
56e4dbeb71 *) displaying current active + current idle threads in PerformanceQueues_p.html now
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1470 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-28 15:17:04 +00:00
theli
859c6a88f5 *) testing various thread pool eviction settings to avoid outOfMemory - Thread creation problem
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1467 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-27 16:51:29 +00:00
orbiter
f2b18cede9 AND-bugfix
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1461 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-27 03:58:38 +00:00
orbiter
b946e28e61 some ranking enhancements
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1460 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-27 02:48:27 +00:00
rramthun
6c02f889f7 Cosmetic changes.
Corrected version numbering as described in http://www.yacy-websuche.de/wiki/index.php/De:Versionsnummern

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1453 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-26 15:12:44 +00:00
theli
b191f06d16 *) Adding additional logging message to locate problems with stackcrawl threads
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1452 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-26 14:24:29 +00:00
theli
d9bcd73d93 *) Bugfix for exception
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1448 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-26 12:15:59 +00:00
theli
f5abfe8d57 *) more failsafe threadpools
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1446 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-26 09:37:43 +00:00
orbiter
a56fefe0d3 added missing forced-flush for index cache
see http://www.yacy-forum.de/viewtopic.php?p=15732#15732

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1434 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-24 16:24:15 +00:00
hermens
78bcb8014a *) Limit range for selection of indexes for distribution to a DHTDistance of 0.2
(For wider ranges enough suitable targets are not probable)
*) Migrate Indexes from ClassicDB back to AssortmentCluster if transfer fails
*) Remove class iterateFiles from plasmaWordIndex
   (The class iterateFiles from plasmaWordIndexClassicDB is used instead)



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1430 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-24 14:58:15 +00:00
hermens
861aae678d *) cleanup cacheAge database when cleaning up the HTCache
*) Log directory deletes with level Fine



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1427 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-24 14:21:01 +00:00
theli
b4e2efef10 *) first test of new iteration function
ATTENTION: please don't use it at the moment

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1418 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-23 17:20:30 +00:00
orbiter
eabf4a0386 fix for null pointer exception during shut-down
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1415 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-23 13:45:14 +00:00
orbiter
47843e69e2 auto-reset for switchboard queue stack
bugfix for http://www.yacy-forum.de/viewtopic.php?p=15684#15684

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1414 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-23 12:41:08 +00:00
orbiter
d6581c445b added content iterator for corrupted database files
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1406 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-22 17:12:16 +00:00
theli
ecdc1f7547 *) Bugfix for crawling URLs with query parameters
See: http://www.yacy-forum.de/viewtopic.php?p=14065
*) Preparation for http://www.yacy-forum.de/viewtopic.php?t=1719

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1405 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-22 16:39:10 +00:00
orbiter
fc4ae899f7 added word-position to ranking (this is only a first step)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1395 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-22 02:16:09 +00:00
orbiter
bb2095fe39 assortment files are now not deleted, but shifted to a backup directory.
See also: http://www.yacy-forum.de/viewtopic.php?p=15458#15458

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1394 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-22 01:03:01 +00:00
orbiter
7366e39dd3 tried to fix 100% CPU bug.
See http://www.yacy-forum.de/viewtopic.php?p=15569#15569

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1393 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-22 00:50:05 +00:00
orbiter
f14d49fae9 enhancements, bugfixes and additions to word index attribute storage
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1392 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-22 00:07:00 +00:00
allo
4d33020f56 Migration to WORK
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1389 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-21 16:39:57 +00:00
rramthun
1e5feedf0e Fix for http://www.yacy-forum.de/viewtopic.php?p=15547#15547
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1388 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-21 11:20:26 +00:00
orbiter
f4ffa9aee5 - implemented more attributes to index entries
- implemented hand-over of new word index attributes during remote search
- implemented word-distance computation during search

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1382 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-20 15:14:21 +00:00
orbiter
90b940e90e fixed position storage problem.
Now the word position is properly stored.
No use of that now, but can be used for better ranking.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1378 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-20 00:26:55 +00:00
orbiter
0371494010 tried to add word position to index
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1377 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-19 14:13:39 +00:00
orbiter
f1cfee7703 removed tabs from condenser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1376 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-19 12:24:35 +00:00
hermens
37791fd529 *) Close indexEntities when "found not enough peers for distribution"
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1375 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-18 20:47:05 +00:00
borg-0300
c5b6154136 added CRDistOn = true/false
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1372 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-18 02:18:23 +00:00
orbiter
71d5c2b2ca better control for target peer selection for RWI transfer
see also http://www.yacy-forum.de/viewtopic.php?p=15343#15343

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1370 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-18 00:30:07 +00:00
hermens
ca7407b7e1 *) Don't change maxTime if zero or negative
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1363 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-17 14:47:21 +00:00
orbiter
3d7c8aaeae removed confusing method
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1339 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-14 02:08:11 +00:00
orbiter
4cd0c45a77 code cleanup
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1337 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-14 01:06:54 +00:00
hermens
971247b78f - rotate merged indexes after merging
see: http://www.yacy-forum.de/viewtopic.php?t=1717
- fix -rwihashlist to correctly shutdown



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1336 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-13 23:59:04 +00:00
orbiter
e2ff1767b5 fix for last DHT distribution bug-fix
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1330 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-13 16:58:27 +00:00
orbiter
060e5a0df0 fixed problem with DHT target peer selection:
- shifted selection in front of distribution
see http://www.yacy-forum.de/viewtopic.php?p=15131#15131


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1327 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-13 14:32:48 +00:00
theli
7c22afe3de *) Bugfix for NullpointerException in deleteOldHTCache
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1326 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-13 10:37:42 +00:00
orbiter
b21b9df2d0 added section headlines generation to html parser
can be viewed in cache control, but is not yet included to indexing

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1320 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-12 20:21:34 +00:00
rramthun
c4487deba9 Minor changes collected over some time.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1319 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-12 19:15:31 +00:00
allo
6822dce57b Using Orbiters function for auth
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1315 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-12 16:21:08 +00:00
orbiter
2028403670 - consolidated different orderings to kelondroNaturalOrder
- added another iteration method to rwihash-enumeration


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1309 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-11 00:32:44 +00:00