Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
2,150 Commits 1 Branch 20 Tags 836 MiB
8ba8e2b7d9
Commit Graph

714 Commits

Author SHA1 Message Date
hydrox
8ba8e2b7d9 *) added cache for blacklists urlhashs recieved by DHT. DHT does not request URLs listed in this cache. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2251 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-06-28 08:51:34 +00:00
hermens
53cbcc6d6e Implement emergency break in index receive when the limit of the ramCache is exceeded by more than cacheLimit 
See: http://www.yacy-forum.de/viewtopic.php?p=22911#22911



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2248 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-06-27 11:14:30 +00:00
orbiter
66964dc015 removed high/med/low from kelondroRecords cache control. 
this was done because testing showed that cache-delete operations
slowed down record access most, even more that actual IO operations.
Cache-delete operations appeared when entries were shifted from low-priority
positions to high-priority positions. During a fill of x entries to a database,
x/2 delete situation happen which caused two or more delete operations.
removing the cache control means that these delete operations are not
necessary any more, but it is more difficult to decide which cache elements
shall be removed in case that the cache is full. There is not yet a stable
solution for this case, but the advantage of a faster cache is more important
that the flush problem.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2244 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-06-25 10:31:38 +00:00
borg-0300
4c6083b264 network picture; 
back to old version

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2242 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-06-24 02:52:24 +00:00
borg-0300
955915385a network picture; 
small changes;

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2241 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-06-23 15:37:59 +00:00
borg-0300
027fa8ab1c network picture; 
bigger; 
more dot steps; 
small other;

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2240 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-06-23 13:53:29 +00:00
theli
b20496e42b *) make DHT DoS check configurable (requested by KoH) 
- check can be disabled via property indexDistribution.dhtReceiptLimitEnabled
   - upper bound can be configured via indexDistribution.dhtReceiptLimit

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2234 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-06-21 19:28:42 +00:00
orbiter
12af69dd86 cosmetics 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2212 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-06-18 11:49:31 +00:00
allo
67a8c74be3 Fix for dynamic login with static password. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2210 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-06-18 08:04:51 +00:00
allo
ef9eb50c3c fix for adminlogin 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2209 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-06-17 11:15:16 +00:00
allo
6fe2fed87e cookieauth works with static Admin. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2208 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-06-16 08:04:02 +00:00
theli
45b39ee1be *) solving unpacking problems with to long filename by 
a) renaming the parent folder in the tgz file to yacy
      (can be configured via build properties file)
   b) reconfiguring build file to throw an error if a file
      name is too long 
Please note that currently there is _no_ proplem with too long
class names because of step a.

      

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2207 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-06-14 15:18:41 +00:00
theli
fb090652df *) use a more compact for plasmaWordIndexAssortmentImporter.java because the long name 
caused problems during untar operation

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2206 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-06-14 14:28:46 +00:00
theli
4ca0857c0c *) Index transfer now considers the pause time send by busy peers during 
index transfer / index distribution
   See: http://www.yacy-forum.de/viewtopic.php?p=22647#22491

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2205 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-06-14 09:40:42 +00:00
orbiter
75ed507d39 some debugging of new kelondroFlexTable class 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2190 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-06-09 12:52:57 +00:00
orbiter
370c481fa7 bugfixes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2171 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-06-02 22:46:32 +00:00
orbiter
c36e9fc8d3 full integration of kelondroRow 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2167 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-06-02 12:45:57 +00:00
orbiter
c75cacda95 added a flex-width-array: this is a table where it is 
possible to add columns to an existing table

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2163 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-06-01 16:01:24 +00:00
orbiter
4a907a570f 1st step to migrate kelondroTree to usage of kelondroRow instead of byte[][] 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2162 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-31 23:31:46 +00:00
orbiter
09f780df27 more bugfixes for the new row/stack handling changes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2160 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-30 21:24:01 +00:00
orbiter
3c3c047d0a integrated kelondroRow into kelondroStack 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2156 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-30 15:28:05 +00:00
orbiter
5bb565944f integration of new kelondroRow into some parts of kelondro, 
especially into the array storage

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2155 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-30 14:36:20 +00:00
orbiter
eaa6f012f0 refactoring: better naming for classic DB (files in WORDS) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2151 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-28 11:59:16 +00:00
orbiter
5041d330ce refactoring 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2150 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-28 11:44:50 +00:00
orbiter
7b3b12888c refactoring: integrated indexContainer abstraction layer 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2149 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-28 01:09:31 +00:00
orbiter
cb295fbbdc refactoring 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2147 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-26 23:55:30 +00:00
rramthun
bc94a714b2 Better explanation for the auto-dom-filter. 
Some javadoc.
Small change to DetailedSearch.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2146 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-26 12:18:12 +00:00
orbiter
196b8abb30 refactoring 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2144 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-26 09:32:50 +00:00
hermens
b48327904a Don't disconnect peers that report 'busy' during index transfer. 
These peers are already being marked as not accepting remote index transmissions by yacyClient.transferIndex. That should by enough to prevent further transfer attempts until newer seed information is received.



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2142 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-25 00:59:14 +00:00
orbiter
4d8f8ba384 added cache-performance analysis for node caches 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2140 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-24 09:40:08 +00:00
orbiter
bd057b44dd - automatic setting of peer-does-not-accept-remote-crawl 
- increased percentage of object cache to node cache to 30%

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2136 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-23 22:03:09 +00:00
orbiter
81e79f2caf fixed new cache behaviour changes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2134 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-23 20:04:48 +00:00
orbiter
cda087f43b - integrated cache miss storage into object cache 
- removed cache-miss handling from indexURL
todo: new Monitoring in PerformanceMemory_p

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2132 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-23 16:43:28 +00:00
orbiter
757ec28430 refactoring: better data capsulation for indexURL 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2131 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-23 08:59:45 +00:00
theli
61078b3885 *) adding support for delayed shutdown 
- needed by Ismael to receive the Steering page properly on shutdown
   - now the steering page should always be displayed properly in the web browser

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2129 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-22 08:02:35 +00:00
orbiter
90d569d70f refactoring of index management: 
url storage is part of index management; moved plasmaURL to indexURL

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2122 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-19 23:50:55 +00:00
orbiter
a930be4ba3 refactoring of index management: 
generalized the index entry

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2121 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-19 23:19:20 +00:00
hermens
df7e1d9df3 Changes to plasmaURL and subclasses: 
- Improve performance of plasmaURL.exists() by remembering URL-hashes that are not present
- Use a more realistic estimation of memory usage by the existsIndex cache
- Routine cleanup of the existsIndex to limit its memory usage



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2113 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-17 13:08:57 +00:00
orbiter
a474669338 start with refactoring of index management 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2110 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-16 16:11:55 +00:00
rramthun
f08e33680c Added Blog-news-symbol as requested. 
I think I will change the character distance a little bit later.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2101 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-15 15:42:06 +00:00
theli
f331def5d8 *) Bugfix for distribution. Incorrect behavior if peerCount == selectedCount 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2098 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-15 10:03:24 +00:00
auron_x
55ea4cbfe6 *)reverted patch for memory-display issue 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2095 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-14 18:09:28 +00:00
theli
5048b05bc6 *) Index Transfer should only restart at the beginning if the delete 
option is configured. Otherwise we have an endless loop

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2092 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-14 13:27:50 +00:00
auron_x
53d9ab6db7 *)fixed bug in PerformanceMemory_p.java which caused negative memory-values on big peers 
see http://www.yacy-forum.de/viewtopic.php?t=2370

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2091 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-14 08:42:41 +00:00
theli
ddfe0f0e27 *) don't try to parse referer string if it's null 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2090 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-13 15:45:04 +00:00
theli
bcc950c533 *) Bugfix for Index Transfer 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2088 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-13 15:28:57 +00:00
orbiter
015d044c25 tried to fix some problems with latest changes to httpc 
very experimental!

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2078 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-10 16:01:14 +00:00
orbiter
3e31820c3d - corrections to PerformanceMemory display of object cache 
- configuration of object cache size in kelondroTree initializer

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2075 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-10 09:08:42 +00:00
orbiter
461548698c configuration of index transfer chunk size 
see http://www.yacy-forum.de/viewtopic.php?p=20951#20951
new properties in yacy.init:
indexDistribution.minChunkSize = 5
indexDistribution.maxChunkSize = 1000
indexDistribution.startChunkSize = 50

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2073 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-09 11:43:10 +00:00
orbiter
29b1b0823c added monitoring of new object cache to performanceMemory page 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2072 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-05-09 10:03:12 +00:00