Commit Graph

20 Commits

Author SHA1 Message Date
Michael Peter Christen
b0c408788b made class methods static where possible 2012-07-05 12:38:41 +02:00
sixcooler
c9aaa9e00a respect non-reserved Memory in GenerationMemoryStrategy
and enable it again
2012-01-16 01:46:12 +01:00
Michael Peter Christen
0d6176804b emergency disabling of GenerationMemoryStrategy because of non-working
available-method
2012-01-15 21:58:18 +01:00
sixcooler
69570fda24 bring my master to stuff from remote 2011-11-24 19:21:58 +01:00
sixcooler
249a78ff2a G1 Memory Strategy - not used now 2011-11-14 11:54:03 +01:00
sixcooler
4fec99115b Implementation of strategies for controlling memory resources.
You can toggle between previous (standard) and new (generation) strategy at PerformanceMemory_p.html.
The generation memory strategy is implemented with the objective of running more robust
but with the cost of early stopping some tasks (eg. dht) while running low on memory.
This new strategy does respect the generational way a heap is organized on most used jvms.
These changes run fine on my 3 peers for weeks now, but as I'm human, I may fail.
Please be carefull using generation memory strategy and report errors by naming
OS, jvm and java_args.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7886 6c8d7289-2bf4-0310-a012-ef5d649a1542
2011-08-22 17:50:03 +00:00
orbiter
2c58af6874 - added a short memory status simulation mode
- added a button in PerformanceMemory_p.html to set the simulated short memory status
- bugfix: added a missing lowercase in KeyList
- better concurrency in loader dispatcher

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7883 6c8d7289-2bf4-0310-a012-ef5d649a1542
2011-08-17 22:24:17 +00:00
orbiter
c64faf41e2 addon to svn 7880
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7882 6c8d7289-2bf4-0310-a012-ef5d649a1542
2011-08-15 11:07:03 +00:00
sixcooler
916d79111e Runtime.maxMemory() DOES change @ runtime:
I wondered getting Total-ram > Max-ram and MemoryControl.available() < 0
MemoryControl.available() < 0 causes some errors where its value is used for dimension of buffers for eg.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7852 6c8d7289-2bf4-0310-a012-ef5d649a1542
2011-07-19 12:48:50 +00:00
orbiter
dbea40d536 - changed snippet fetch strategy logic: do not check if entry is in cache. This should reduce IO load on the HTCACHE which is a showstopper during large number of search requests
- forced a possible short memory status when a search is started to flush caches that may cause search-heaps with resource contention effects

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7747 6c8d7289-2bf4-0310-a012-ef5d649a1542
2011-05-27 09:32:03 +00:00
orbiter
4bea3f9714 hack to reduce resource contention caused by massive UTF8 decodings which use java.nio resources:
used a ASCII String <-> byte[] conversion wherever possible. Many Strings in YaCy are hashes which are pure ASCII (base64 hashes).
The new ASCII String <-> byte[] conversion method have less computation overhead than the UTF8 conversion.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7746 6c8d7289-2bf4-0310-a012-ef5d649a1542
2011-05-27 08:24:54 +00:00
low012
3b40b98256 *) set SVN properties
*) minor changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7567 6c8d7289-2bf4-0310-a012-ef5d649a1542
2011-03-08 01:51:51 +00:00
orbiter
8d14916c74 more patches for a better out-of-memory management
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7555 6c8d7289-2bf4-0310-a012-ef5d649a1542
2011-03-07 01:45:11 +00:00
lotus
b1484299b2 same units for memory observer configuration (MiB)
old setting for DHT (RAM) will be lost after update
can be set on /Performance_p.html

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7418 6c8d7289-2bf4-0310-a012-ef5d649a1542
2011-01-02 20:38:01 +00:00
lotus
eac2daf2e8 * reenable DHT if yet enough memory is available
* reset treshold on reconfiguratoin
(thanks to sixcooler)

* display status message in web interface

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6562 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-10 19:04:43 +00:00
lotus
32972139af added nice configuration for the resource observer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6554 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-07 17:19:50 +00:00
lotus
9bee0ac780 more logging for DHTrule
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6533 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-12-21 14:02:00 +00:00
lotus
713cb26a27 update for memory observer algorithm
disable dht if memory is less than treshold
after 4 times, maximum 11 minutes between each detection

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6517 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-12-06 17:45:48 +00:00
lotus
6edc168cfe option to disable dht by memory limit:
memory.acceptDHT in kbytes
not yet pre-enabled, will clear on every startup
please review since this could break dht in freeworld

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6459 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-11-06 19:13:30 +00:00
orbiter
bea3b99aff moved table and util classes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6397 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-10 01:14:19 +00:00