Commit Graph

7 Commits

Author SHA1 Message Date
orbiter
89b9b2b02a redesigned remote crawl process:
- instead of pushing urls to other peers, the urls are actively pulled
  by the peer that wants to do a remote crawl
- the remote crawl push process had been removed
- a process that adds urls from remote peers had been added
- the server-side interface for providing 'limit'-urls exists since 0.55 and works with this version
- the list-interface had been removed
- servlets using the list-interface had been removed (this implementation did not properly manage double-check)
- changes in configuration file to support new pull-process
- fixed a bug in crawl balancer (status was not saved/closed properly)
- the yacy/urls-protocol was extended to support different networks/clusters
- many interface-adoptions to new stack counters

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4232 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-11-29 02:07:37 +00:00
orbiter
01e0669264 re-designed some parts of DHT position calculation (effect is the same as before)
and replaced old fist hash computation by new method that tries to find a gap in the current dht
to do this, it is necessary that the network bootstraping is done before the own hash is computed
this made further redesigns in peer initialization order necessary

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4117 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-10-01 12:30:23 +00:00
karlchenofhell
6fbe31425a - some code-cleanup (no more syntax-warnings here)
- added deletion from loadedURLs of URLs to be blacklisted in IndexControl_p

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3404 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-26 12:56:50 +00:00
theli
2336f0f013 *) allow pausing/resuming of crawlJob Threads separately
- pausing/resuming localCrawls
   - pausing/resuming remoteTriggeredCrawls
   - pausing/resuming globalCrawlTrigger
   See: http://www.yacy-forum.de/viewtopic.php?t=1591

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1723 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-21 11:18:48 +00:00
borg-0300
d5a86e2287 added keywords, properties
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1402 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-22 14:35:51 +00:00
theli
319f8ffb18 *) changing file header
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1400 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-22 14:17:24 +00:00
theli
ed5ac0f81a *) adding servlet that is needed by the yacy screensaver
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1399 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-22 14:06:43 +00:00