orbiter
70bd67e73e
documentation update
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3503 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-21 14:29:42 +00:00
theli
a53403e0b2
*) Updating News Feed for new release
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3502 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-21 14:13:13 +00:00
orbiter
3688ec33e5
release 0.51
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3501 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-21 14:00:17 +00:00
theli
1f61c13697
*) RSS-parser extracts the author tags now
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3500 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-21 13:35:32 +00:00
orbiter
602ac42010
fix for OOM case when a kelondroTree Node cache grows
...
See also: http://www.yacy-forum.de/viewtopic.php?p=33275#33275
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3499 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-21 13:26:18 +00:00
theli
b374812f01
*) adding rpm packager as author
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3498 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-21 13:09:12 +00:00
orbiter
beb772d6cd
fixed problem with broken notifier image, occurred only at initial start-up
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3497 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-21 12:23:27 +00:00
theli
40ce33e664
*) adding RSS feed for yacy news
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3496 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-21 12:22:18 +00:00
theli
589cbd8cbf
*) replacing all yacy-news-category strings with corresponding constants
...
Note: please use these constants from now on
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3495 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-21 11:09:15 +00:00
allo
f4af360f7c
bugfix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3494 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-20 15:37:19 +00:00
allo
bb51efbb49
"Bugfix" for Tagdisplay
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3493 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-19 13:00:33 +00:00
rramthun
43114af6d7
*) Translated robots.txt-config-page
...
*) Simplified some sentences
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3492 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-19 12:42:07 +00:00
orbiter
7af188ff9a
fix for http://www.yacy-forum.de/viewtopic.php?p=33089#33089
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3491 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-19 11:59:29 +00:00
orbiter
5bbf010107
removed synchronization of size() method from numerous classes to avoid thread locking
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3490 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-18 19:45:23 +00:00
orbiter
6b9eea3932
- removed differentiation between longTitle and shortTitle; this cannot be used for search results,
...
and it is difficult to get both types from all document types
- added some author parsing
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3489 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-18 12:33:19 +00:00
orbiter
a738b57b31
added author tag to indexing content
...
enhanced composition of title tag
TODO: insert author information for external parsers
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3488 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-17 01:18:34 +00:00
orbiter
6be57983a8
another update to the crawl balancer
...
can now alternate between top and bottom of the crawl stack
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3487 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-16 16:54:54 +00:00
orbiter
91cdc1493f
removed query to NAT or responder in case that no other peer is there.
...
this is not needed any more, there are enough peers
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3486 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-16 15:21:24 +00:00
orbiter
4783a30910
- fixed a flush problem in balancer
...
- return to idle divisor in RWI RAM cache flush
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3485 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-16 15:16:26 +00:00
theli
91c2a042a7
*) bugfix for wrong proxy traffic accounting
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3484 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-16 13:52:48 +00:00
orbiter
861f41e67e
redesigned NURL-handling:
...
- the general NURL-index for all crawl stack types was splitted into separate indexes for these stacks
- the new NURL-index is managed by the crawl balancer
- the crawl balancer does not need an internal index any more, it is replaced by the NURL-index
- the NURL.Entry was generalized and is now a new class plasmaCrawlEntry
- the new class plasmaCrawlEntry replaces also the preNURL.Entry class, and will also replace the switchboardEntry class in the future
- the new class plasmaCrawlEntry is more accurate for date entries (holds milliseconds) and can contain larger 'name' entries (anchor tag names)
- the EURL object was replaced by a new ZURL object, which is a container for the plasmaCrawlEntry and some tracking information
- the EURL index is now filled with ZURL objects
- a new index delegatedURL holds ZURL objects about plasmaCrawlEntry obects to track which url is handed over to other peers
- redesigned handling of plasmaCrawlEntry - handover, because there is no need any more to convert one entry object into another
- found and fixed numerous bugs in the context of crawl state handling
- fixed a serious bug in kelondroCache which caused that entries could not be removed
- fixed some bugs in online interface and adopted monitor output to new entry objects
- adopted yacy protocol to handle new delegatedURL entries
all old crawl queues will disappear after this update!
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3483 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-16 13:25:56 +00:00
theli
094a1482f4
*) removing yacy.exe on ant clean
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3482 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-16 12:44:51 +00:00
theli
832662ccd2
*) removing yacy.jar on ant clean
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3481 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-16 12:40:43 +00:00
hydrox
9b5fb3908d
*) a peer-message are now created when a blog-comment is written
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3480 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-15 12:58:17 +00:00
orbiter
581db87237
more debug code for
...
http://www.yacy-forum.de/viewtopic.php?p=33009#33009
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3479 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-14 15:04:06 +00:00
orbiter
81c4cc6bf7
better debugging of balancer failure
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3478 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-14 12:02:56 +00:00
orbiter
dd06d4cada
more logging to better trace bug
...
http://www.yacy-forum.de/viewtopic.php?p=33001#33001
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3477 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-14 09:36:54 +00:00
orbiter
96b79bf86d
redesigned remove method in kelondroRowSet
...
This should fix also numerous bugs like
http://www.yacy-forum.de/viewtopic.php?p=31077#31077
(java.lang.ArrayIndexOutOfBoundsException in kelondroRowCollection.removeShift)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3476 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-14 08:55:05 +00:00
orbiter
9f929b5438
better snippet handling in case of snippet load fail
...
see also http://www.yacy-forum.de/viewtopic.php?p=31096#31096
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3475 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-13 22:18:36 +00:00
auron_x
d451ad48d3
*) improved peerloadgraphic:
...
- unnecessary (0 %) pieces are removed
- percent-values of each thread displayed in legend
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3474 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-12 19:08:17 +00:00
orbiter
a5d668c0c6
added speed-buttons for easy performance setting
...
appears in crawl start and on indexing monitor page
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3473 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-12 16:24:28 +00:00
orbiter
5b0a84ce09
fix for synchronization deadlock with flushMissNameCache.
...
see also: http://www.yacy-forum.de/viewtopic.php?p=32939#32939
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3472 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-12 09:06:57 +00:00
karlchenofhell
e2ac5f62bd
- Code hübscher machen [von NNs TODO]
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3471 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-11 19:53:14 +00:00
allo
f04097c3dd
integrated tor-patch for crawling, if yacyDebugMode is set.
...
(replaces: http://yacy.deruwe.de/overlay/net-misc/yacy-tor/files/disable_dns_checks-svn3132.patch )
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3470 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-11 18:43:11 +00:00
auron_x
22fe14f292
*) first version of Peerload-graphic
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3469 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-11 17:04:11 +00:00
orbiter
432d7d4e9c
better catch
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3468 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-10 23:38:08 +00:00
orbiter
8f7e8b6ee2
auto-delete for not-fixable db error in crawl stacker.
...
see also http://www.yacy-forum.de/viewtopic.php?p=32906#32906
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3467 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-10 23:31:36 +00:00
orbiter
7a52b07fcc
better memory protection during freemen cycle
...
see also http://www.yacy-forum.de/viewtopic.php?p=32903#32903
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3466 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-10 23:22:37 +00:00
orbiter
6faa262259
fix for NURL-fix
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3465 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-09 14:30:53 +00:00
orbiter
909d7a8ae9
fixed wrong implemented row iterator in kelomdroFlexSplitTables
...
this has no effect, until now this iterator was only used on
the Index Administration page.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3464 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-09 13:55:26 +00:00
orbiter
a1fb8358b2
lets make a well-formed http link so that other crawlers don't have a problem to follow this link :-)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3463 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-09 12:35:54 +00:00
orbiter
4edb70f68b
added yacybot info-page from Roland
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3462 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-09 12:26:31 +00:00
orbiter
3ef77d2030
fix for http://www.yacy-forum.de/viewtopic.php?p=29878#29878
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3461 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-09 12:14:25 +00:00
orbiter
3bb3df3fc0
fix for http://www.yacy-forum.de/viewtopic.php?p=32298#32298
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3460 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-09 12:03:53 +00:00
orbiter
b3ca177a5d
fix for http://www.yacy-forum.de/viewtopic.php?p=32797#32797
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3459 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-09 11:49:56 +00:00
orbiter
243a2f831b
fixed problem with not found NURL-hashes
...
The cause for this problem could still not be found, but the effect
is handled much better. The NURL-pop will continue automatically until
it found a hash that can be found.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3458 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-09 11:07:20 +00:00
orbiter
6ad39bae1e
fixed shutdown problem
...
this fixes the 'inconsistency' messages during start-up
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3457 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-09 08:48:47 +00:00
orbiter
38b93f8cb8
bugfix for my last commit:
...
iterator did not consider secondary start point in case of rotation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3456 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-08 22:07:17 +00:00
karlchenofhell
264a82eec8
- fix for http://www.yacy-forum.de/viewtopic.php?t=3657
...
- fix for http://www.yacy-forum.de/viewtopic.php?p=32758#32758
- Diff takes any objects now, not only strings
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3455 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-08 22:04:15 +00:00
rramthun
045d758537
Avoid stopwords as topwords, configurable
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3454 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-08 20:50:27 +00:00