Commit Graph

424 Commits

Author SHA1 Message Date
orbiter
08d5db6bb4 next try to fix the restart
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3939 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-29 14:45:53 +00:00
orbiter
2ff94b2fb4 another try to fix the restart on linux (it works on mac)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3938 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-29 13:56:28 +00:00
orbiter
1a45ecb356 - fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=14&p=137#p137
- fix for missing restart script in ant built target
- removed some more synchronization for size() operations
- removed blocking statement on search page

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3935 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-28 22:06:33 +00:00
orbiter
1fa4feb8e6 added restart button. should work on linux and mac, but was only tested on mac
should of course work on windows as before

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3934 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-28 20:36:15 +00:00
orbiter
87afdfc2a7 fix for long waiting time during deletion of processed news
see http://forum.yacy.de/viewtopic.php?f=6&t=6

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3932 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-28 15:03:58 +00:00
orbiter
9bbd39b67c - removed unfinished auto-updater from roland and martin
- added new download-option for releases on the status page
still mising:
- thomas-style restart for linux/mac
- untar/gunzip on shell basis
(comes next)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3931 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-28 14:52:26 +00:00
orbiter
1782ef57e5 - added SSI parser and include directive for <!--# include virtual="<file>" -->
- added chunked file transfer for non-yacy clients
- SSIs are streamed using chunked transfer, partly delivered pages can be seen in browser before transmission is finished
- added client-side network unit identification
- cleaned up code

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3926 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-26 14:37:10 +00:00
orbiter
6b4cfbd2d6 new network bootsraping method
- no more contact to yacy.net (no remote superseed any more)
- moved superseed file into new network unit definition
- fixed build; includes new network bootstraping files now

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3922 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-25 14:43:57 +00:00
orbiter
0e57a8062b added network definition for different YaCy networks
(needs much more work)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3919 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-22 14:29:14 +00:00
auron_x
52cb3208d0 *) old (lastseen > 7d) peers are now automatically removed from passive and potential seed-dbs
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3917 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-22 09:16:25 +00:00
orbiter
6518bb6c08 changed release strategy:
we will provide two different releases in the future, one standard release and one 'pro'-release.
the 'pro'-release contains all additional parsers AND has different default performance values.
The pro-version differs therefore from the previous 'all'-version by this default values.
The pro-configuration is automatically choosen if the libx-folder exists. If a version is once initialized, its configuration stays independently from an existing libx folder.
The ant targets had been changed. There are now 3 different targets to create standard and pro-releases, and one target to upgrade:
- dist: creates a standard release (only, no libx target any more)
- distPro: creates a pro-release (includes the libx)
- distExt: creates a libx-release which includes the libx-folder only. It may be used to upgrade from standard to pro
Furthermore, the naming of 'dev'-releases had been removed.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3902 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-16 14:11:52 +00:00
orbiter
069562a14d fixed problem with re-crawl; replaced error file-db with ram-db
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3900 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-15 23:47:08 +00:00
orbiter
9f7765863b bugfix for seed length control routine
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3879 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-13 13:21:19 +00:00
orbiter
4f5496062c protection against too large seeds
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3877 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-12 22:08:33 +00:00
orbiter
684ded0e09 added new news types
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3876 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-12 15:15:24 +00:00
hydrox
4a1bc4743a *)News-entries with blacklisted URLs are now ignored
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3849 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-10 08:05:18 +00:00
karlchenofhell
086239da36 - added servlet: remote crawler queue overview
- added servlet: crawl profile editor

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3731 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-05-16 10:11:25 +00:00
karlchenofhell
8ba81e0995 - added some comments (will get more in the near future)
- added missing <label> to the search field in Network.html

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3728 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-05-15 15:48:28 +00:00
orbiter
b05e2314cf another dht selection fix
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3725 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-05-14 12:52:39 +00:00
orbiter
e897eb9b4a fix for DHT selection target
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3722 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-05-14 08:17:36 +00:00
orbiter
756a27049b last-minute-feature 'newbie-selection' for workshop purpose:
for remote search, always select all peers that are less than a day old
(should be removed someday in the future if load is too high, which could mean when pph > 100)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3712 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-05-11 19:55:19 +00:00
orbiter
139c59ebbd - fixed dht selction problem: the seed tables used a wrong ordering
- cleaned some code

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3693 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-05-09 17:59:36 +00:00
orbiter
f3fee4d445 fix for http://www.yacy-forum.de/viewtopic.php?p=35322
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3689 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-05-08 09:19:36 +00:00
orbiter
7f56c8d4aa fixed some seed selection details
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3685 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-05-07 22:22:35 +00:00
orbiter
e602436fda fixed problem with cluster routing
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3684 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-05-07 20:48:24 +00:00
orbiter
0831034e07 fixed non-termination bug for robinson remote crawl peer selection
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3681 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-05-07 14:37:50 +00:00
orbiter
111ba9e359 - fixed some width problems in new status page
- fixed deadlock in dns cache
- added termination security for DHT peer selection

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3660 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-05-05 23:18:00 +00:00
orbiter
191ef16499 fixed wrong ordering that caused bad dht selection
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3646 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-05-04 14:11:50 +00:00
orbiter
7cf8981a98 - added debugging code for wrong DHT target iterator
- restricted distance constraint from 0.4 to 0.2

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3644 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-05-03 22:57:55 +00:00
orbiter
06b6e35484 fix for a null pointer exception if clusters are not defined
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3632 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-05-02 12:26:29 +00:00
orbiter
81844e85b2 - fixed more cluster routing problems
- fixed a problem in remote search when balancer caused shift process to wait too long

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3627 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-04-30 00:39:53 +00:00
orbiter
1979a167d3 fixed problem with cast
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3625 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-04-29 22:31:44 +00:00
orbiter
e48189c710 enhanced cluster routing
- cluster definitions can now contain an addition for local ip addresses
- cluster-cluster communication uses the local ip address instead the global address, if one is given

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3624 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-04-29 22:05:34 +00:00
orbiter
b33cef421e better routing for public clusters
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3620 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-04-29 00:08:38 +00:00
orbiter
f73e1e3af9 fixed bugs in remote search setting for public clusters
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3615 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-04-28 00:12:41 +00:00
orbiter
871ee1ce0f one step closer to automatic updates:
automatically aquire release information from download archives
web pages from latest.yacy-forum.net and yacy.net are retrieved, parsed,
links wihin are analysed, sorted and the most recent developer and main
releases are provided as direct download link on the status page, if it was
discovered that a more recent version than the current version is available.
This process is done only once during run-time of a peer, to protect our
download archives from DoS by YaCy peers.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3606 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-04-27 09:23:44 +00:00
orbiter
f8de19fb2f robinson cluster: added client-side protocol implementation
- the network configuration page shows a new option: robinson clusters
- when a global search is made, all robinson peers are excluded, but:
- robinson peers/clusters that provide peer tags and where search words match
  such tags, they are included in global search. Therefore, robinson peers/clusters
  support the global yacy network with their indexes, without doin DHT-exchange


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3598 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-04-26 09:51:51 +00:00
rramthun
e6fb6426a3 *) Some cosmetical changes and corrections
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3582 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-04-19 16:16:54 +00:00
(no author)
6186185775 *) Moved some comments to javadoc
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3573 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-04-14 10:11:37 +00:00
orbiter
78d04bcbcf fixed bug in search statistics
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3562 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-04-12 09:24:56 +00:00
orbiter
b79b4082e2 completed search exclusion:
- exclusion on index-level (not only from search snippets)
- exclusion hand-over at remote search protocol

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3556 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-04-10 12:27:03 +00:00
auron_x
d66b0276e3 *) removed log-output for PPM-calc
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3553 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-04-07 10:03:12 +00:00
orbiter
25070822a5 fix for http://www.yacy-forum.de/viewtopic.php?p=33925#33925
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3551 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-04-05 19:08:59 +00:00
orbiter
159bd0cab5 diverses; b.o. fix for http://www.yacy-forum.de/viewtopic.php?p=33914#33914
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3549 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-04-05 14:58:29 +00:00
orbiter
40c14a4f0e - better implementation of search query properties
- basic protection against start-up problems when database files are corrupted
- auto-delete of not-critical databases during startup when load error occurs
- on-the-fly reset option for all database tables
- automatic on-the-fly reset for seed tables during enumeration exceptions

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3547 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-04-05 10:14:48 +00:00
theli
1696606b7f *) changing loglevel of "PPM-Calculation" message
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3545 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-04-04 13:09:59 +00:00
auron_x
8ef3ad12a7 *) fix for rare bug in PPM-calc
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3523 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-25 21:46:03 +00:00
auron_x
00bc0c1b47 *) new logging for PPM-Calculation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3522 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-25 20:24:12 +00:00
auron_x
5941577076 *) added some logging to PPM-Calculation to find a rare bug
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3521 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-25 14:56:42 +00:00
hydrox
b99f9d870d *) fixed double selection of peers for the same DHT-chunk.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3513 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-22 09:08:38 +00:00