97 Commits

Author	SHA1	Message	Date
sixcooler	e071d71f19	fix for yacy-banner-network-values ... http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2521 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6659 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-02-09 18:22:36 +00:00
sixcooler	787b588c33	reverted a part of svn6636: ... - didn't work on blobs >2GB - should be obsolete since svn6651 http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2652&sid=7fa98fd3edfc2a03f26394d545e3e3c1&p=19172#p19172 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6655 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-02-07 19:32:46 +00:00
lotus	11188cd7eb	resource observer now uses the Java 6 method to check for free space. thus, disk observing now needs Java 6 installed. ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6652 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-02-06 18:48:06 +00:00
sixcooler	089877f32c	my first commit - hopefully fix for merge problem ... - http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2652 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6651 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-02-05 19:38:00 +00:00
orbiter	d6391f2537	better handling of rewrite cases where the resulting rewrite blob entry is equal in size ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6648 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-02-04 23:17:47 +00:00
orbiter	ef9473d92c	added another sixcooler suggestion: recycle corrupted records ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6647 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-02-04 16:25:05 +00:00
orbiter	fe78edac32	- view API calls in correct date-order ... - execute recorded API calls in date-order git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6646 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-02-04 15:51:54 +00:00
orbiter	308a973503	refactoring of tables data organisation ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6644 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-02-04 11:26:23 +00:00
orbiter	ada0ce9de3	refactoring of bookmarks: there is a big performance problem in the bookmarks code and furthermore the bookmarks ... will loose its leading role for the re-crawl funtion when the new api tables will work. To be prepared for a replacement of such functions the bookmark class is re-organised. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6637 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-02-01 22:18:56 +00:00
orbiter	3751ab4ae2	added sixcoolers patch and more checks/removed unnecessary code ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6636 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-02-01 16:11:00 +00:00
orbiter	d8d8562c59	fill key with zeros during normalization ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6635 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-02-01 15:40:16 +00:00
orbiter	24060885b6	- added Tables abstraction in data.Tables.java ... fix for http://forum.yacy-websuche.de/viewtopic.php?p=18910#p18910 http://forum.yacy-websuche.de/viewtopic.php?p=18894#p18894 http://forum.yacy-websuche.de/viewtopic.php?p=18814#p18814 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6631 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-29 18:02:09 +00:00
orbiter	7fdf59a77f	misc NPE check ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6630 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-29 15:59:24 +00:00
orbiter	4403304957	bugfix for list() ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6616 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-22 16:00:56 +00:00
orbiter	0098e6e859	bugfix for heap iterator ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6610 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-22 10:26:50 +00:00
orbiter	db19a941cf	added new image index storage classes (not integrated yet) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6608 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-21 22:12:05 +00:00
orbiter	8ce936bcdd	added an api recording function: it shall be possible to record ... all operations on YaCy in a database that should make it possible 1) to re-create a setting on fresh peers 2) to transmit a setting from one peer to another 3) to re-create crawl starts after a complete deletion of the index This functionality will also support 4) scheduled re-crawls (new implementation) To implement this, a new database structure has been crated that stores maps into blob heaps. to encode maps the b-encoding technique was used (this is the same encoding that torrent files use) - added a b-encoder - enhanced the b-decoder - added a b-encoded map heap data structure - added a table organisation based on b-encoded heaps - added a servlet to maintain such tables (see Tables_p.html) - integrated the servlet into the Advanced Settings menu - added an api recording based on the new tables git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6606 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-21 22:06:03 +00:00
orbiter	e80e060ca6	- increased thread priority for server threads ... - decreased thread priority for crawler threads git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6596 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-19 11:29:22 +00:00
orbiter	f6731c6240	more logging etc. ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6589 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-17 00:41:50 +00:00
orbiter	4f1f4863c4	fix for deadlock when initializing a SplitTable with a file of size 0, see also: ... http://forum.yacy-websuche.de/viewtopic.php?p=18594#p18594 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6587 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-14 23:03:48 +00:00
orbiter	cc5dcf69ff	missing change for last commit ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6585 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-14 14:20:18 +00:00
orbiter	ca1ef9a079	fix for http://forum.yacy-websuche.de/viewtopic.php?p=18584#p18584 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6584 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-14 13:38:14 +00:00
orbiter	938e806182	tried to fix date problem that may have prevented that foreign peers stay in the network ... - removed unused code - removed possibly wrong utc difference correction git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6581 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-13 20:01:46 +00:00
orbiter	5df628a2a4	- added BEncoder class ... - added BEncodedHeap class that encodes B data structures and stores that to a heap - refactoring of MapView, this is now named MapHeap to fit into the naming scheme of the BEncodedHeap git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6579 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-13 16:21:37 +00:00
orbiter	82f57f79e5	more PMD enhancements ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6576 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-13 00:23:07 +00:00
orbiter	a06f7ddb33	more PMD recommendations ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6572 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-12 20:53:19 +00:00
orbiter	eb79ceb3ff	update to kelondro data structures ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6571 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-12 15:37:34 +00:00
orbiter	18172451a0	better search computation: ... - increased sort limit, now 3000 entries, before: 1000 this should cause that more results can be shown in case of strong limitating constraints, like domain navigation - enhanced the sort process - check against domain navigator bugs - fix in sort stack - showing now all naviagtion pages at first search (not only next page) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6569 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-12 15:01:44 +00:00
orbiter	2113fcd7e5	- fixed usage of isEmpty() which is not available in java 1.5 ... - increased visibility of some methods git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6564 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-11 12:33:40 +00:00
orbiter	dd459281c8	applied code changes that are recommended by PMD ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6563 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-10 23:09:48 +00:00
lotus	eac2daf2e8	* reenable DHT if yet enough memory is available ... * reset treshold on reconfiguratoin (thanks to sixcooler) * display status message in web interface git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6562 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-10 19:04:43 +00:00
orbiter	d77a8f3b3e	added some modifications recommended by PMD for better performance ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6560 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-10 01:40:26 +00:00
orbiter	7f20963b41	add-on to last commit ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6556 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-09 00:17:39 +00:00
orbiter	eeca2ded92	fix for http://forum.yacy-websuche.de/viewtopic.php?p=18500#p18500 ... - catch uncatched OOM - less wasting of memory git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6555 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-09 00:08:16 +00:00
lotus	32972139af	added nice configuration for the resource observer ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6554 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-07 17:19:50 +00:00
hermens	574f49903e	Prevent blob merge from possibly losing the last container ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6549 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-05 01:34:46 +00:00
orbiter	83d05e9176	added sixcoolers hack with some modifications: ... http://forum.yacy-websuche.de/viewtopic.php?p=15004#p15004 old index blobs where deletions have been made because of DHT transmission should be melted down to new blobs. This uses sixcoolers methods from the forum thread but modifies the process in such a way that the blobs are not merged with themselves but simply rewritten to smaller files. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6548 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-04 18:12:03 +00:00
orbiter	d0b7bf9ca2	added a decoder class for Bencoding ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6544 6c8d7289-2bf4-0310-a012-ef5d649a1542	2010-01-03 22:44:09 +00:00
low012	028657f019	*) adding more SVN properties ... *) minor changes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6542 6c8d7289-2bf4-0310-a012-ef5d649a1542	2009-12-28 13:11:07 +00:00
low012	82d740050f	*) adding more SVN properties ... *) minor changes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6541 6c8d7289-2bf4-0310-a012-ef5d649a1542	2009-12-28 12:26:50 +00:00
low012	e04cb8cef0	*) adding more SVN properties ... *) minor changes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6540 6c8d7289-2bf4-0310-a012-ef5d649a1542	2009-12-28 12:16:40 +00:00
low012	dcb1096fb0	*) adding more SVN properties ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6539 6c8d7289-2bf4-0310-a012-ef5d649a1542	2009-12-28 11:22:32 +00:00
low012	7d610e0063	*) minor changes ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6538 6c8d7289-2bf4-0310-a012-ef5d649a1542	2009-12-28 11:20:34 +00:00
lotus	9bee0ac780	more logging for DHTrule ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6533 6c8d7289-2bf4-0310-a012-ef5d649a1542	2009-12-21 14:02:00 +00:00
orbiter	c14233a933	fix for a OOM in MapView that can cause unavailability of ... - seed list - bookmarks during very low memory configuration git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6529 6c8d7289-2bf4-0310-a012-ef5d649a1542	2009-12-15 12:23:37 +00:00
orbiter	37245430c3	fix for NPE during DHT RWI selection ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6527 6c8d7289-2bf4-0310-a012-ef5d649a1542	2009-12-15 00:02:10 +00:00
orbiter	959b38b61b	fix for memory tracker ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6526 6c8d7289-2bf4-0310-a012-ef5d649a1542	2009-12-14 20:23:11 +00:00
orbiter	a37878b7d5	url parser regex performance hack ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6524 6c8d7289-2bf4-0310-a012-ef5d649a1542	2009-12-10 14:40:32 +00:00
orbiter	362b7a929b	added extensive memory protection logic to avoid out of memory errors that may be caused by the RowCollection memory allocation function ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6521 6c8d7289-2bf4-0310-a012-ef5d649a1542	2009-12-09 23:27:26 +00:00
orbiter	8281e29963	- more configuration for profiling graph (number of events) ... - more logging for a shutdown: print reason and accessing IP into log git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6520 6c8d7289-2bf4-0310-a012-ef5d649a1542	2009-12-08 14:25:51 +00:00