2640 Commits

Author	SHA1	Message	Date
orbiter	68f5d64ae6	replaced yacy logo by better version ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3675 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-05-07 08:54:27 +00:00
orbiter	d6480dc670	fix for long transfer pauses ... see http://www.yacy-forum.de/viewtopic.php?p=35243#35243 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3672 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-05-06 21:43:20 +00:00
theli	cb43ae11ba	*) Bugfix ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3668 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-05-06 12:57:22 +00:00
theli	0b5fc3c28c	*) moving date functions to serverDate class ... *) Sitemap-parser - logging added - parsing of modDate added git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3667 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-05-06 12:36:49 +00:00
theli	6f46245a51	*) Bookmarks: Ajax icon is displayed while loading title ... *) First version of a sitemap parser added - currently only autodetection of sitemap files is supported *) DB-Import restructured - pause/resume should work again now git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3666 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-05-06 09:52:04 +00:00
theli	269d5ca45b	*) Bugfix for IllegalMonitorException ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3664 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-05-06 08:54:02 +00:00
theli	e8610f049a	*) new method allowing the updater to wait until yacy has finished startup ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3662 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-05-06 08:22:18 +00:00
orbiter	111ba9e359	- fixed some width problems in new status page ... - fixed deadlock in dns cache - added termination security for DHT peer selection git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3660 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-05-05 23:18:00 +00:00
theli	74dd6cac95	*) signal yacy shutdown to updater ... *) some javadoc added git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3658 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-05-05 16:20:01 +00:00
theli	43748f87fb	*) changes required for the uploader ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3655 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-05-05 15:41:05 +00:00
rramthun	e12e934ade	*) Fixed broken compile process. ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3650 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-05-04 21:33:37 +00:00
rramthun	d6811ac243	*) Moving tar.jar from libx to lib ... *) Enhanced interface git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3649 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-05-04 19:46:23 +00:00
theli	469583ea80	*) new interface class. should be implemented by the updater to allow communication between the updater and yacy ... (not yet functional) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3648 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-05-04 14:22:10 +00:00
orbiter	191ef16499	fixed wrong ordering that caused bad dht selection ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3646 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-05-04 14:11:50 +00:00
orbiter	7cf8981a98	- added debugging code for wrong DHT target iterator ... - restricted distance constraint from 0.4 to 0.2 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3644 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-05-03 22:57:55 +00:00
rramthun	abb63e3289	*) disabled system.exit() in case of YaCy shutdown as it kills the whole VM (including updater and other management threads) ... *) UpdateCheck-Thread now pauses for given interval correctly git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3642 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-05-03 19:16:43 +00:00
orbiter	dd44a1394f	disabled automatic performance setting change ... - during crawl start - each indexing cycle - for delay values - for short memory cycles git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3634 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-05-02 15:39:27 +00:00
orbiter	b9add5cf37	some bugfixes: ... - dht iterator start point - wordIndex synchronization - surftipps url check git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3633 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-05-02 14:20:43 +00:00
orbiter	06b6e35484	fix for a null pointer exception if clusters are not defined ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3632 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-05-02 12:26:29 +00:00
orbiter	35c660654d	more debugging lines to fix bug for ... http://www.yacy-forum.de/viewtopic.php?p=34935#34935 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3629 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-30 23:05:19 +00:00
orbiter	47e90f31b2	fix for deadlock in plasmaWordIndex.addPageIndex ... synchronization for class method not necessary see also: http://www.yacy-forum.de/viewtopic.php?p=34959#34959 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3628 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-30 22:30:09 +00:00
orbiter	81844e85b2	- fixed more cluster routing problems ... - fixed a problem in remote search when balancer caused shift process to wait too long git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3627 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-30 00:39:53 +00:00
orbiter	304ed3f4d2	fix for remote crawl requests in clusters ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3626 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-29 22:52:07 +00:00
orbiter	1979a167d3	fixed problem with cast ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3625 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-29 22:31:44 +00:00
orbiter	e48189c710	enhanced cluster routing ... - cluster definitions can now contain an addition for local ip addresses - cluster-cluster communication uses the local ip address instead the global address, if one is given git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3624 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-29 22:05:34 +00:00
theli	e9d87b2fce	*) changes required for the uploader ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3621 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-29 15:56:45 +00:00
orbiter	b33cef421e	better routing for public clusters ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3620 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-29 00:08:38 +00:00
theli	7c902996b5	*) changes required for the uploaderWrapper ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3618 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-28 16:04:37 +00:00
orbiter	f73e1e3af9	fixed bugs in remote search setting for public clusters ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3615 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-28 00:12:41 +00:00
orbiter	485bf1ea83	bugfix for robinson/remote crawl bug ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3614 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-27 21:35:43 +00:00
orbiter	62c947b4aa	next try to fix deadlock in plasmaWordIndex ... see also: http://www.yacy-forum.de/viewtopic.php?p=34821#34821 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3607 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-27 12:26:36 +00:00
orbiter	871ee1ce0f	one step closer to automatic updates: ... automatically aquire release information from download archives web pages from latest.yacy-forum.net and yacy.net are retrieved, parsed, links wihin are analysed, sorted and the most recent developer and main releases are provided as direct download link on the status page, if it was discovered that a more recent version than the current version is available. This process is done only once during run-time of a peer, to protect our download archives from DoS by YaCy peers. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3606 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-27 09:23:44 +00:00
theli	ec225f9ab6	*) SOAP: adding methods to get the comment and MD5 checksum of a single file ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3604 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-27 05:10:31 +00:00
theli	2399ed817c	*) robots.txt parser now extracts the sitemap-URL (will be used later) ... *) some javadoc added *) junit testclass for robots.txt parser added git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3602 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-26 15:42:38 +00:00
orbiter	fa012789b2	tried to fix a deadlock problem durin shutdown ... see also: http://www.yacy-forum.de/viewtopic.php?p=34753#34753 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3601 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-26 15:15:40 +00:00
orbiter	e192f616a2	collection of small bugfixes ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3600 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-26 14:28:57 +00:00
orbiter	64a6d6e5e6	added new set iterator (needed for last commit) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3599 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-26 09:52:37 +00:00
orbiter	f8de19fb2f	robinson cluster: added client-side protocol implementation ... - the network configuration page shows a new option: robinson clusters - when a global search is made, all robinson peers are excluded, but: - robinson peers/clusters that provide peer tags and where search words match such tags, they are included in global search. Therefore, robinson peers/clusters support the global yacy network with their indexes, without doin DHT-exchange git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3598 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-26 09:51:51 +00:00
theli	50e1e61fa5	*) SOAP: adding functions to rename and move files ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3595 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-25 05:36:28 +00:00
theli	63a004abff	*) bugfix for Nullpointerexception ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3594 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-25 05:22:09 +00:00
orbiter	29fe2beac7	possibly fixed a deadlock ... cannot find forum link now for that git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3593 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-24 21:55:57 +00:00
(no author)	4f4d3d71dd	*) Faster appearance of ConfigBasic by bypassing UPNP-scan in case of existing external connects ... *) Marked two deprecated source-points *) Added possibility to dump words from indexing to file. Should not affect performance in the current form. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3592 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-24 16:33:31 +00:00
orbiter	657585fe0d	network functions for robinson peers: server-side protection ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3591 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-24 15:11:12 +00:00
orbiter	89c1511738	- added new Network Configuration menu, can be found in basic settings ... - new cluster functions will be available in this menu, but currently not enabled, because corresponding interface methods are not ready yet - shifted remote crawl settings to new network configuration menu - shifted DHT distribution/receive to the new network configuration menu - adopted some string constants - added cluster configuration settings to yacy.init git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3589 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-23 20:47:07 +00:00
orbiter	62b79aa0a9	bugfix for http://www.yacy-forum.de/viewtopic.php?p=34558#34558 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3586 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-20 21:38:12 +00:00
orbiter	2f3b518169	temporary patch for startup-problem: ... http://www.yacy-forum.de/viewtopic.php?t=3854 This is a serious problem that is caused by the database bug between 0.511 - 0.513 which produced a large number of double-entries in the RWI index. The uniq()-method tries to fix this, and it does not terminate when the index is large and the number of double-occurrences is also large. This patch does simply implement a time-controlled termination, which does not heal the inconsistency problem. The uniq-method itself is correct and does not need a bugfix, the non-termination is simply caused by the large number of data that is shifted during the process. It was possible to reproduce this behaviour in a test environment. A real fix would need to: - enhance the uniq()-method by using a recursive, binary segmentation of the array to be fixed - uniq() must report the entries that are double - the double-entries must be deleted from the collection index (from the index and the collections) to heal the problem git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3583 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-20 07:53:58 +00:00
rramthun	e6fb6426a3	*) Some cosmetical changes and corrections ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3582 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-19 16:16:54 +00:00
orbiter	ba525ebf52	- re-enabled path optimization that was disabled during testing ... - re-implemented index load/extend optimization that was removed from kelondroFlexTable, this is now part of kelondroIntBytesIndex git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3580 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-19 14:55:19 +00:00
orbiter	595ee10468	fixed datatabase inconsistency bugs ... inserted many debug lines added a huge number of asserts extended database test methods git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3579 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-19 13:37:02 +00:00
orbiter	ca79362b9d	disabling auto-setting of remote crawl performance ... see also http://www.yacy-forum.de/viewtopic.php?t=3849 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3577 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-18 18:36:06 +00:00
orbiter	7a7a1c7c29	fight against problems with remove-methods and synchronization ... - some bugs may have been fixed with wrong removal operations - removed temporary storage of remove-positions and replaced by direct deletions - changed synchronization - added many assets - modified dbtest to also test remove during threaded stresstest git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3576 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-17 15:15:47 +00:00
orbiter	b6a5f53020	removed double synchronization from kelondroRecords.USAGE to prevent thread locking. ... The method synchronization should be sufficient git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3574 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-15 21:13:54 +00:00
(no author)	6186185775	*) Moved some comments to javadoc ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3573 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-14 10:11:37 +00:00
orbiter	063063aa0c	fix for 100% cpu bug during dht selection ... see also: http://www.yacy-forum.de/viewtopic.php?p=34068#34068 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3570 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-13 13:40:19 +00:00
michitux	4990909178	Some bugfixes, new layout/style for image search results: ... * removed divide by zero bug when 20_dhtdistribution_busysleep is 0 * replaced German comment with wrong charset in source/de/anomic/plasma/plasmaCrawlBalancer.java by an English one * replaced the table-fix for floating behind snipped images by a br with clear * removed unnecessary old xhtml-files (were not in use, they were created when we weren't having xhtml for testing) * new layout for image-search results: replaced the old one with spans and tables inside (not valid) with new divs, now each image snippet container has the same size TODO: * the ids of the snippetLoading-divs aren't valid because ids must start with an alphabetic letter or an underscore, they have to be prefixed * in the returned snippet-xml is an unresolved pattern for status (the status is only set for text snippets) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3566 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-12 18:21:17 +00:00
orbiter	78d04bcbcf	fixed bug in search statistics ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3562 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-12 09:24:56 +00:00
orbiter	b79b4082e2	completed search exclusion: ... - exclusion on index-level (not only from search snippets) - exclusion hand-over at remote search protocol git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3556 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-10 12:27:03 +00:00
auron_x	d66b0276e3	*) removed log-output for PPM-calc ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3553 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-07 10:03:12 +00:00
low012	4400acc27d	*) created new 8 bit oldschool style font for possible future use ... *) main method is generalization of main method of ymageFontGenerator: it does not matter how many lines of how many bits a font is made of as long as the values stay the same within the font -> use this class as a template for your own font generators and be a happy camper *) main method checks if font is valid (96 characters, all letters must have same number of lines and same number of bits per line) *) ***** I have not checked if the result is really a valid font so far. ***** git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3552 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-05 21:05:16 +00:00
orbiter	25070822a5	fix for http://www.yacy-forum.de/viewtopic.php?p=33925#33925 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3551 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-05 19:08:59 +00:00
orbiter	06a7978730	moved url pattern matching for search to better place ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3550 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-05 15:08:14 +00:00
orbiter	159bd0cab5	diverses; b.o. fix for http://www.yacy-forum.de/viewtopic.php?p=33914#33914 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3549 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-05 14:58:29 +00:00
orbiter	cdc7b77a62	fix for http://www.yacy-forum.de/viewtopic.php?p=33916#33916 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3548 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-05 14:47:45 +00:00
orbiter	40c14a4f0e	- better implementation of search query properties ... - basic protection against start-up problems when database files are corrupted - auto-delete of not-critical databases during startup when load error occurs - on-the-fly reset option for all database tables - automatic on-the-fly reset for seed tables during enumeration exceptions git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3547 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-05 10:14:48 +00:00
theli	1696606b7f	*) changing loglevel of "PPM-Calculation" message ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3545 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-04 13:09:59 +00:00
auron_x	f30bf1683e	*) corrected spelling of captcha ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3544 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-04 10:13:58 +00:00
orbiter	fcdf000fbc	bugfix for http://www.yacy-forum.de/viewtopic.php?p=33838#33838 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3543 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-03 22:08:40 +00:00
low012	d7edc9740b	*) added correct (c) and Last*-data ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3542 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-03 20:58:45 +00:00
low012	ee241f32e6	*) very basic capcha class (see coding sections of forum for more details) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3541 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-03 20:54:14 +00:00
orbiter	6e7340ef52	added exclusion search ... (you can now search and exclude words from the result with '-') git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3540 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-03 15:35:29 +00:00
orbiter	e4734a8b6b	fix for fix in SVN 3537 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3539 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-03 14:42:29 +00:00
orbiter	356033aceb	fixed bug with continuous reset of balancer file index ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3537 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-03 12:36:24 +00:00
orbiter	ba2c307ab3	optimized memory allocation in kelondroRow.Entry ... such an entry cannot be instantiated without allocation of new byte[]; instead it can re-use memory from other kelondroRow.Entry objects. during bugfixing also other bugs may have been solved, maybe the INCONSISTENCY problem could have been solved. One cause can be missing synchronization during bulk storage when a R/W-path optimization is done. To test this case, the optimization is currently switched off. More memory enhancements can be done after this initial change to the allocation scheme. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3536 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-03 12:10:12 +00:00
theli	24ea4ca631	*) adding first version of postscript parser ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3535 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-04-01 15:02:07 +00:00
theli	def0d6124e	*) trying to solve SecurityManager problem during init of soap engine ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3534 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-31 16:01:52 +00:00
theli	75eb65028a	*) adding a test if a seucrity manager is active ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3533 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-31 14:30:09 +00:00
orbiter	210ede8230	added a class for byte-array management. This was the result of a very large experiment ... to replace byte[] objects within kelondro. Frequent System.arraycopy are common when kelondroRow.Entry objects are handled. This class may be used to prevent this. However, experimental replacement of byte[] by kelondroByteArray in kelondroRow.Entry resulted in complete re-write of large parts of kelondro. This experiment did not completely lead to a result, because then the interface to kelondro had to be changed also from byte[] to kelondroByteArray, which may have caused a rewrite of large parts of YaCy. The experiment is therefore abanonded, but this class remains here without any function but possibly for future use. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3531 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-30 08:44:43 +00:00
theli	1b7fda12ee	*) SOAP: separate function to get the active/passive/potential peer list ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3526 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-28 07:34:44 +00:00
orbiter	6488ec8a80	no deletions in index in case that snippet-loading fails and there is no network connection ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3525 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-27 08:21:45 +00:00
orbiter	847349358b	less memory usage during collectionIndex-rebuild ... should also speed up that process a little bit git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3524 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-27 08:21:03 +00:00
auron_x	8ef3ad12a7	*) fix for rare bug in PPM-calc ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3523 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-25 21:46:03 +00:00
auron_x	00bc0c1b47	*) new logging for PPM-Calculation ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3522 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-25 20:24:12 +00:00
auron_x	5941577076	*) added some logging to PPM-Calculation to find a rare bug ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3521 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-25 14:56:42 +00:00
orbiter	5c3afb3202	added option to configure a path to a secondary index location. ... this shall be used to store a fragment of the index on another physical device, to split IO load and enhance access speed. The index is splitted in such a way that the LURLs are stored to the secondary location, and the RWIs to the primary location. This is especially useful for environments where symbolic links are not possible and may cause IO access even if there is no write access to the device which hosts the symbolic link. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3519 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-24 15:28:17 +00:00
theli	c2e6afbd69	*) bugfix: setting mimeType properly for dir listing with e.g. "?format=xml" ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3516 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-23 05:37:19 +00:00
orbiter	242c19b480	completed TLD categorization ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3515 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-22 13:52:00 +00:00
hydrox	b99f9d870d	*) fixed double selection of peers for the same DHT-chunk. ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3513 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-22 09:08:38 +00:00
theli	f20b596dc0	*) adding servlet to display all deployed SOAP Services ... - soap related servlets are located in htroot/soap *) new serverContext class for soap git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3511 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-22 08:30:57 +00:00
theli	75d90834a2	*) adding additional file extension for powerpoint ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3507 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-21 16:18:58 +00:00
orbiter	2cb16824e3	removed support for old database structures. ... The new collection index will be more generalized to support other indexes i.e. YBR block-rank computation. A clean-up of the many conditions to support the old database was necessary. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3506 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-21 15:35:35 +00:00
theli	81b4598487	*) peer profile can now be displayed as vcard ... e.g. http://localhost:8080/ViewProfile.vcf?hash=localhash git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3504 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-21 15:08:18 +00:00
orbiter	3688ec33e5	release 0.51 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3501 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-21 14:00:17 +00:00
theli	1f61c13697	*) RSS-parser extracts the author tags now ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3500 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-21 13:35:32 +00:00
orbiter	602ac42010	fix for OOM case when a kelondroTree Node cache grows ... See also: http://www.yacy-forum.de/viewtopic.php?p=33275#33275 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3499 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-21 13:26:18 +00:00
theli	b374812f01	*) adding rpm packager as author ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3498 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-21 13:09:12 +00:00
orbiter	beb772d6cd	fixed problem with broken notifier image, occurred only at initial start-up ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3497 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-21 12:23:27 +00:00
theli	40ce33e664	*) adding RSS feed for yacy news ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3496 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-21 12:22:18 +00:00
theli	589cbd8cbf	*) replacing all yacy-news-category strings with corresponding constants ... Note: please use these constants from now on git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3495 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-21 11:09:15 +00:00
allo	f4af360f7c	bugfix ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3494 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-20 15:37:19 +00:00
orbiter	7af188ff9a	fix for http://www.yacy-forum.de/viewtopic.php?p=33089#33089 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3491 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-19 11:59:29 +00:00
orbiter	5bbf010107	removed synchronization of size() method from numerous classes to avoid thread locking ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3490 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-18 19:45:23 +00:00
orbiter	6b9eea3932	- removed differentiation between longTitle and shortTitle; this cannot be used for search results, ... and it is difficult to get both types from all document types - added some author parsing git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3489 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-18 12:33:19 +00:00
orbiter	a738b57b31	added author tag to indexing content ... enhanced composition of title tag TODO: insert author information for external parsers git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3488 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-17 01:18:34 +00:00
orbiter	6be57983a8	another update to the crawl balancer ... can now alternate between top and bottom of the crawl stack git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3487 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-16 16:54:54 +00:00
orbiter	91cdc1493f	removed query to NAT or responder in case that no other peer is there. ... this is not needed any more, there are enough peers git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3486 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-16 15:21:24 +00:00
orbiter	4783a30910	- fixed a flush problem in balancer ... - return to idle divisor in RWI RAM cache flush git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3485 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-16 15:16:26 +00:00
theli	91c2a042a7	*) bugfix for wrong proxy traffic accounting ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3484 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-16 13:52:48 +00:00
orbiter	861f41e67e	redesigned NURL-handling: ... - the general NURL-index for all crawl stack types was splitted into separate indexes for these stacks - the new NURL-index is managed by the crawl balancer - the crawl balancer does not need an internal index any more, it is replaced by the NURL-index - the NURL.Entry was generalized and is now a new class plasmaCrawlEntry - the new class plasmaCrawlEntry replaces also the preNURL.Entry class, and will also replace the switchboardEntry class in the future - the new class plasmaCrawlEntry is more accurate for date entries (holds milliseconds) and can contain larger 'name' entries (anchor tag names) - the EURL object was replaced by a new ZURL object, which is a container for the plasmaCrawlEntry and some tracking information - the EURL index is now filled with ZURL objects - a new index delegatedURL holds ZURL objects about plasmaCrawlEntry obects to track which url is handed over to other peers - redesigned handling of plasmaCrawlEntry - handover, because there is no need any more to convert one entry object into another - found and fixed numerous bugs in the context of crawl state handling - fixed a serious bug in kelondroCache which caused that entries could not be removed - fixed some bugs in online interface and adopted monitor output to new entry objects - adopted yacy protocol to handle new delegatedURL entries all old crawl queues will disappear after this update! git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3483 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-16 13:25:56 +00:00
hydrox	9b5fb3908d	*) a peer-message are now created when a blog-comment is written ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3480 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-15 12:58:17 +00:00
orbiter	581db87237	more debug code for ... http://www.yacy-forum.de/viewtopic.php?p=33009#33009 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3479 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-14 15:04:06 +00:00
orbiter	81c4cc6bf7	better debugging of balancer failure ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3478 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-14 12:02:56 +00:00
orbiter	dd06d4cada	more logging to better trace bug ... http://www.yacy-forum.de/viewtopic.php?p=33001#33001 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3477 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-14 09:36:54 +00:00
orbiter	96b79bf86d	redesigned remove method in kelondroRowSet ... This should fix also numerous bugs like http://www.yacy-forum.de/viewtopic.php?p=31077#31077 (java.lang.ArrayIndexOutOfBoundsException in kelondroRowCollection.removeShift) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3476 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-14 08:55:05 +00:00
orbiter	9f929b5438	better snippet handling in case of snippet load fail ... see also http://www.yacy-forum.de/viewtopic.php?p=31096#31096 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3475 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-13 22:18:36 +00:00
auron_x	d451ad48d3	*) improved peerloadgraphic: ... - unnecessary (0 %) pieces are removed - percent-values of each thread displayed in legend git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3474 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-12 19:08:17 +00:00
orbiter	a5d668c0c6	added speed-buttons for easy performance setting ... appears in crawl start and on indexing monitor page git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3473 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-12 16:24:28 +00:00
orbiter	5b0a84ce09	fix for synchronization deadlock with flushMissNameCache. ... see also: http://www.yacy-forum.de/viewtopic.php?p=32939#32939 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3472 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-12 09:06:57 +00:00
karlchenofhell	e2ac5f62bd	- Code hübscher machen [von NNs TODO] ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3471 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-11 19:53:14 +00:00
allo	f04097c3dd	integrated tor-patch for crawling, if yacyDebugMode is set. ... (replaces: http://yacy.deruwe.de/overlay/net-misc/yacy-tor/files/disable_dns_checks-svn3132.patch) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3470 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-11 18:43:11 +00:00
auron_x	22fe14f292	*) first version of Peerload-graphic ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3469 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-11 17:04:11 +00:00
orbiter	432d7d4e9c	better catch ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3468 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-10 23:38:08 +00:00
orbiter	8f7e8b6ee2	auto-delete for not-fixable db error in crawl stacker. ... see also http://www.yacy-forum.de/viewtopic.php?p=32906#32906 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3467 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-10 23:31:36 +00:00
orbiter	7a52b07fcc	better memory protection during freemen cycle ... see also http://www.yacy-forum.de/viewtopic.php?p=32903#32903 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3466 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-10 23:22:37 +00:00
orbiter	6faa262259	fix for NURL-fix ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3465 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-09 14:30:53 +00:00
orbiter	909d7a8ae9	fixed wrong implemented row iterator in kelomdroFlexSplitTables ... this has no effect, until now this iterator was only used on the Index Administration page. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3464 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-09 13:55:26 +00:00
orbiter	a1fb8358b2	lets make a well-formed http link so that other crawlers don't have a problem to follow this link :-) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3463 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-09 12:35:54 +00:00
orbiter	4edb70f68b	added yacybot info-page from Roland ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3462 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-09 12:26:31 +00:00
orbiter	3ef77d2030	fix for http://www.yacy-forum.de/viewtopic.php?p=29878#29878 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3461 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-09 12:14:25 +00:00
orbiter	3bb3df3fc0	fix for http://www.yacy-forum.de/viewtopic.php?p=32298#32298 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3460 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-09 12:03:53 +00:00
orbiter	243a2f831b	fixed problem with not found NURL-hashes ... The cause for this problem could still not be found, but the effect is handled much better. The NURL-pop will continue automatically until it found a hash that can be found. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3458 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-09 11:07:20 +00:00
orbiter	6ad39bae1e	fixed shutdown problem ... this fixes the 'inconsistency' messages during start-up git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3457 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-09 08:48:47 +00:00
orbiter	38b93f8cb8	bugfix for my last commit: ... iterator did not consider secondary start point in case of rotation git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3456 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-08 22:07:17 +00:00
karlchenofhell	264a82eec8	- fix for http://www.yacy-forum.de/viewtopic.php?t=3657 ... - fix for http://www.yacy-forum.de/viewtopic.php?p=32758#32758 - Diff takes any objects now, not only strings git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3455 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-08 22:04:15 +00:00
orbiter	d755a8026d	- better OOM protection ... - better memory allocation for FlexTable indexes - splitting between static index and dynamic index (only the dynamic part must grow) - to enable a merge-iteration of new splittet index, a huge number of classes needed to be adopted for new iterator classes - added new iterator classes that support cloneable iterators - adopted all iterator classes to implement cloneable itarators git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3453 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-08 16:15:40 +00:00
orbiter	23338d2070	small fix for RAM computation ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3447 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-07 23:55:52 +00:00
orbiter	33f97cff7a	changed startup initialization sequence slightly ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3446 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-07 23:24:16 +00:00
orbiter	4e8eb1dbe3	some minor changes here and there ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3441 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-07 14:22:10 +00:00
karlchenofhell	03c5906ae7	- minor bugfixes for url-fetcher & http://www.yacy-forum.de/viewtopic.php?t=3646 ... - PerformanceMemory_p.html is valid XHTML again git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3440 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-07 11:50:03 +00:00
orbiter	3499a364ef	a little bit better memory protection ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3439 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-07 09:38:14 +00:00
orbiter	313f6a7680	fix for http://www.yacy-forum.de/viewtopic.php?p=31553#31553 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3438 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-07 09:26:01 +00:00
orbiter	958ebea5c5	fix for http://www.yacy-forum.de/viewtopic.php?p=32470#32470 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3437 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-07 09:08:13 +00:00
orbiter	5d5e6ebfcc	fix for http://www.yacy-forum.de/viewtopic.php?p=32631#32631 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3436 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-07 08:54:07 +00:00
orbiter	1cba31de43	redesigned ram organization for database caches ... - each cache can now allocate as much memory as is available - no more fixed limits - replaced old performance memory monitor by new one - added supervision methods as static functions into the classes that provide cache functionality - steering of ram allocation is done with two simple limits that are ram availability-relative git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3434 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-06 22:43:32 +00:00
theli	26450a1d9a	*) avoid nullpointerException on seed.getAddress() (reported by netbude) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3431 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-06 16:11:36 +00:00
orbiter	db235f2d61	added some memory protection in collection index multiple merge ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3429 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-04 22:54:04 +00:00
theli	c72605ecab	*) adding a function to determine if a given URL is bookmarkt ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3428 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-03 11:57:49 +00:00
theli	bd03c6b874	*) bugfix in bookmarksDB: ... - NullpointerException when trying to get an unknown bookmark - bookmarks can either start with http or https git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3427 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-03 11:56:46 +00:00
orbiter	b466baa574	added some memory protection ... too large collection arrays are now avoided. By default, the biggest collection index is 7. larger collections are dumped into a commons directory, but cannot yet be used. Bevore doing a dump, the collection is splittet into a part which has only root-references, and stored back to the collection; the remaining part goes to commons git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3426 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-03 00:55:51 +00:00
low012	ce360ef43e	*) no more HTML in plasmaCrawlProfile.java anymore ... *) <br> will not be displayed in items in Auto Filter Content on WatchCrawler_p.html anymore *) removed unnecessary replaceHTML() git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3425 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-02 21:09:28 +00:00
karlchenofhell	88245e44d8	- improved version of robots.txt (delete your old htroot/robots.txt before updating): ... - robots.txt is a servlet now - no need to rewrite the whole file each time a section is added or removed - user-defined disallows, added manually, won't be overwritten anymore - new config-setting: httpd.robots.txt, holding names of the disallowed sections git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3423 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-02 01:19:38 +00:00
karlchenofhell	9623bf7bbe	- removed call of java 1.5 method ... - added config servlet for local robots.txt - removed YPStats_p as it is of no use anymore - supertemplates use XHTML now - quick-fix for http://www.yacy-forum.de/viewtopic.php?p=32296#32296 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3422 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-03-01 13:54:14 +00:00
orbiter	51e12049fa	third generation of R/W head path optimization ... - data from collection arrays are read in order - merged data is written in order git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3419 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-28 11:13:23 +00:00
karlchenofhell	a1d68fe092	- use .class rather than Class.forName for classes in class-path ... - added Bost's patch for Diff.findDiagonale() from: http://www.yacy-forum.de//files/patch_685.txt - fixed minor bugs in Blog git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3416 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-27 22:52:22 +00:00
orbiter	10a3c20b8d	some more enhancements to R/W Head path optimization ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3415 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-27 15:54:02 +00:00
orbiter	f4cfd19835	second Generation of collection R/W head path optimization: ... - permanent cache flush is switched off. The optimized cache flush works better if it is a large number of collections that is flushed together - the flush size can be configured instead the flush divisor. There is only one size for all flushes - collection records that shall be removed during collection transition (jump from one collection file to another) are now not really removed but only marked in RAM. add-operations to the collection use these marked collection spaces - index bulk write operations are now separated for each file of a kelondroFlex git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3414 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-27 13:01:22 +00:00
orbiter	1fda50fd3c	correct R/W head positioning in kelondroFlex ... and some enhancements git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3409 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-26 22:25:39 +00:00
orbiter	304412a049	first generation of collection index R/W head path optimization ... - collections are now hand-over as collection lists to collection index for merge opertations - collection index lists are separated into 'new' and 'extend' lists - lists are written separately - write operations are done into array sets and array indexes. These are now serialized - write operations into index files are sorted by index; that means that a R/W head does not need to go forward and backward, only forward More enhancements are possible git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3407 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-26 15:49:23 +00:00
hydrox	54fef3574f	*) missing files for last commit ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3406 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-26 14:38:34 +00:00
hydrox	cb89c74d52	*) added blog-comments ... *) removed debug-output when deleting news git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3405 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-26 14:36:01 +00:00
karlchenofhell	6fbe31425a	- some code-cleanup (no more syntax-warnings here) ... - added deletion from loadedURLs of URLs to be blacklisted in IndexControl_p git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3404 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-26 12:56:50 +00:00
orbiter	32867580ee	update to kelondroRecords needed fo last commit ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3403 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-26 11:55:36 +00:00
orbiter	e3480d4ad3	fix for warning in crawl balancer ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3402 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-26 11:54:43 +00:00
orbiter	8668ac5d91	preparations for collection index cache flush optimization ... (hand-over commit, no functional change to current code) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3399 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-25 21:06:26 +00:00
karlchenofhell	39a2000d8b	- added support for [[Bookmark:$bookmarkTag|description]]-link-listings (requested by theli) to wiki-parser ... - added support for <pre>-tags to wiki-parser git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3393 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-24 21:26:48 +00:00
karlchenofhell	619653c054	- fix for last commit ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3392 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-24 15:40:56 +00:00
karlchenofhell	26f5757b40	- added support for multiple paths per domain to default-blacklist ... warning: an interface-change had been neccessary: - remove(String, String) has been renamed to removeAll(String, String), because it removes all path-entries for the specified host - remove(String, String, String) has been added to delete only a path-entry - geBlacklistType(String) has been renamed to getBlacklistType(String) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3391 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-24 13:56:32 +00:00
karlchenofhell	a5a36d9252	- hopefully last fix fo 1.5 methods (sorry for that, eclipse isn't that helpful in identifying those methods) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3387 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-22 08:04:09 +00:00
karlchenofhell	e97b6f0458	- we still use Java 1.4 ... ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3386 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-21 22:43:31 +00:00
karlchenofhell	0c7b8cf632	- added first version of new wiki-parser ... - added blacklist support to manual URLFetcher stack fill - fix for NPE: http://www.yacy-forum.de/viewtopic.php?t=3559 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3385 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-21 22:31:36 +00:00
orbiter	f7803a6ce4	enhanced crawl balancer ... - new domains now get a chance to get crawled early - less IO operations - new balancing method - better dump order at shutdown time - bugfixes regarding not found url hashes (no more superfluous cache kill) - domain access time is now shared over all balancer stacks - viewing the stack does no more disturbish the balancing algorithm that much - intelligent selection of best next domain using domain access times - extra double-check (to double-check the double-check) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3384 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-21 16:23:31 +00:00
low012	801eea8849	*) Fixed bug where pairReplace() got caught in infinite recursion. (http://www.yacy-forum.de/viewtopic.php?t=3466) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3383 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-20 22:07:59 +00:00
orbiter	c3e8c23f5d	fix for 'CANNOT FETCH ENTRY: hash is null' bug ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3380 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-20 13:53:21 +00:00
orbiter	badab8d924	fixed some more bugs in new db handling ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3379 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-20 12:29:12 +00:00
orbiter	e72d253577	fixed problem with initial cache load ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3378 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-20 11:20:48 +00:00
orbiter	2d8e472cfd	emergeny bugfix for last commit ... (kelondroTree should work again) the cache prefill is broken and will be fixed later git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3377 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-20 10:25:17 +00:00
orbiter	dc0c06e43d	PLEASE MAKE A BACK-UP OF YOUR COMPLETE DATA DIRECTORY BEFORE USING THIS ... redesign for better IO performance enhanced database seek-time by avoiding write operations at distant positions of a database file. until now, a USEDC counter was written at the head-section of a kelondroRecords database file (which is the basic data structure of all kelondro database files) to store the actual number of records that are contained in the database. Now, this value is computed from the database file size. This is either done only once at start-time, or continuously when run in asserts enabled. The counter is then updated only in RAM, and written at close of the file. If the close fails, the correct number can be computed from the file size, and if this is not equal to the stored number it is a strong evidence that YaCY was not shut down properly. To preserve consistency, the complete storage-routine had to be re-written. Another change enhances read of nodes in some cases, where the data-tail can be read together with the data-head. This saves another IO lookup during each DB node fetch. Includes also many small bugfixes. IF ANYTHING GOES WRONG, ALL YOUR DATA IS LOST: PLEASE MAKE A BACK-UP git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3375 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-20 08:35:51 +00:00
karlchenofhell	c016fcb10f	- added streaming-support to CrawlURLFetchStack_p servlet ... - bug for NPE in list.java - use more constants git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3373 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-19 12:47:46 +00:00
karlchenofhell	d114a0136e	- crawl profile: don't add null-values ... - added some settings and statistics for url-fetcher 'server'-mode - added own stack for fetchable URLs - added possibility to fill stack via shift from peer's queues, via POST (addurls=$count and url$num=$url) or via file-upload - added "htroot" to classpath of linux start-script git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3370 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-17 19:16:53 +00:00
karlchenofhell	b2a9d32f29	why do I always forget some lines? sorry... ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3368 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-14 15:11:03 +00:00
theli	e1edb23689	*) Bugfix for IllegalMonitorStateException ... See: http://www.yacy-forum.de/viewtopic.php?t=3522 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3358 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-09 19:32:49 +00:00
orbiter	bf69a721cb	more protection against mis-use of YaCyHop interface: ... - target must not be at port 80 - target access not more than every 3 seconds - requester may not access more than every 10 seconds git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3357 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-09 15:25:10 +00:00
orbiter	a15963ff98	better balancing: if element from top would force a busy waiting, ... an element from the bottom of the stack is used instead. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3356 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-09 10:32:58 +00:00
orbiter	dda24fcb85	ups ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3355 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-09 09:55:21 +00:00
orbiter	8c1d2e0227	protection against crawl balancer failure: ... a minimum of 500 milliseconds distance between two acesses to the same domain is now ensured git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3354 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-09 09:48:23 +00:00
orbiter	1f1f398bfa	enhanced speed of RAM cache flush by factor 20 (twenty times faster) ... - the speed was doubled by avoiding read access during the dump - the speed was dramatically increased at least by factor 10 by using a temporary ram-file where the structures are flushed to before it is dumped then as a whole byte-chunk to the file system. The speed enhancements also affects some other parts of the database. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3353 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-08 23:21:46 +00:00
orbiter	30d79d69a6	fix for wrong display of search statistics ... see http://www.yacy-forum.de/viewtopic.php?p=31242#31242 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3352 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-08 10:42:35 +00:00
orbiter	daf2e15f59	some storage process enhancements (write without preceding read) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3348 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-07 23:23:24 +00:00
orbiter	9c2101a852	small enhancement to cache dump ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3346 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-07 00:02:54 +00:00
orbiter	c464157a6e	replaced some toString() ... see http://www.yacy-forum.de/viewtopic.php?p=31151#31151 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3345 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-06 16:26:56 +00:00
orbiter	7673f0869b	minor enhancements ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3344 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-06 16:01:03 +00:00
orbiter	b4aa195c27	added user-agent check for yacy-hop proxy authentication ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3343 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-06 09:53:02 +00:00
orbiter	2d7f7da7ce	fix for null pointer exception ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3342 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-06 09:50:24 +00:00
orbiter	d25caa07bf	redesigned some parts of http authentication ... added another access check for peer hops git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3340 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-05 19:46:50 +00:00
hydrox	9184113284	*) fixed News deletion. News are now removed if they are no longer in a news-stack. This does not effect News-entires in the news-db that have no stack-entries. ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3336 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-05 13:35:36 +00:00
(no author)	e218940293	The copyright sign "\u00A9" is already replaced by "&copy;". String "(C)" is not a unicode sequence! ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3334 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-04 18:16:27 +00:00
low012	1bc4d8d470	*) If there is more than one pair of patterns in a line, all of them (and not only one pair) will be replaced. ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3333 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-04 15:53:40 +00:00
low012	ea7a8cf7aa	*) <hr> and <br> tags are XHTML compliant now. ... *) Avoid superflous trailing blank in non-proportional sections. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3332 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-04 15:03:13 +00:00
orbiter	d03cd41266	fix for http://www.yacy-forum.de/viewtopic.php?t=3411 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3331 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-04 04:39:47 +00:00
karlchenofhell	f2e6f19b90	- added versioning to Wiki ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3327 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-03 15:20:12 +00:00
orbiter	10ebcb5e1e	fixed bug in disconnection time ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3326 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-03 11:22:23 +00:00
karlchenofhell	02a73dce87	- added Diff-class for wiki-versioning (forthcoming, first need suitable serverObjects.put() for it) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3325 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-03 05:24:44 +00:00
karlchenofhell	2401e748a3	- fixed wrong replacement of POST-parameters in httpd ('<' and '>' are still replaced, don't know why): http://www.yacy-forum.de/viewtopic.php?t=3466 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3324 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-03 01:26:05 +00:00
orbiter	b2f4087400	redesign of last-seen fieln inside seed: ... the field contains now a time in UDC-0 (instead relative to local UDC offset) this fixes a bug in peer selection, where an iteration over all seeds ordered by lastseen did not work correctly. Problems may occur because the new meaning of this field may mix with the different meaning of that field in older peers git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3322 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-02 23:54:27 +00:00
karlchenofhell	e68cdeeeb3	- reverted parseArg(String) to use a byte-array to handle correct UTF-8 parsing ... - arguments aren't passed html-escaped to the servlets anymore, bug-fix for http://www.yacy-forum.de/viewtopic.php?p=30573 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3321 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-02 21:20:53 +00:00
orbiter	e00e850a98	removed constants (no connection with yacySeed.dna identifier) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3320 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-02 14:52:54 +00:00
orbiter	fcc11391a8	some redesign attempts because sorting of lastseen does not work correctly ... not finished yet target: better selection of peer-ping targets, which should enhance stabilization of the net git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3319 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-02 13:12:31 +00:00
orbiter	c2d6edf21d	integrated number of remote targets as 'partitions' into remote search protocol ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3317 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-01 13:27:23 +00:00
orbiter	f696d3c1eb	added double computation to kelondroMapObjects ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3316 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-01 09:48:31 +00:00
orbiter	819ff21c92	fixed QPM output ... QPM is temporarily called QPH (until more search requests are present?) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3313 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-02-01 00:17:35 +00:00
auron_x	89e7af037a	*) used more switchboard-vars instead of config-vars ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3310 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-31 17:05:15 +00:00
orbiter	306c50ac40	QPM (queries per minute) statistic stub ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3308 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-31 15:39:11 +00:00
orbiter	7598e1243e	removed unused variables/imports ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3306 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-31 09:28:47 +00:00
hydrox	27e4429645	*) fixed minimizeurldb-function ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3305 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-31 09:22:22 +00:00
orbiter	47ab83a7c0	added flag for YaCyHop - proxy access for all paths that start with /yacy/ ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3304 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-31 00:09:51 +00:00
karlchenofhell	9f74b128dd	- added many more commented constants (please use constants rather than i.e. config-setting strings directly) ... - not all constants may be located correctly in the switchboard. Please relocate if you know the appropriate place for them git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3303 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-30 14:18:35 +00:00
orbiter	e4910f03d1	tag storage fix ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3302 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-30 11:52:15 +00:00
orbiter	991182b29b	more space for bookmarks ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3299 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-30 00:20:03 +00:00
orbiter	88fa764b64	implemented new kelondroObjects into bookmarkDB ... - Bookmark-Objects are stored inside the kelondroObjects cache - removed superfluous classes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3298 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-30 00:17:55 +00:00
orbiter	9c05e2a820	re-design ob kelondroMap ... - this class is replaced by an object that can hold any type of object - this object must be defined as a class that implements kelondroObjectsEntry - the kelodroMap is now implemented as kelondroMapObjects git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3297 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-29 23:51:10 +00:00
allo	669c21db05	first version of abstracted kelondroMap Cache. ... get returns a kelondroCachedObject(or in most cases a subclass of it), or a map, which can be used to construct a kelondroCachedObject. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3295 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-29 19:10:55 +00:00
allo	98cb777e18	abstract wikiCode in putWiki ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3293 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-29 15:09:58 +00:00
orbiter	f25c0e98d1	- replaced String by StringBuffer in condenser ... - added CamelCase parser in condenser - added option to switch on or off indexing for proxy git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3292 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-29 01:11:22 +00:00
allo	14f2068daf	some more bookmark changes towards multiuser bookmarks ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3291 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-28 17:38:43 +00:00
allo	25c7d4e25e	fix for form (cookie) login ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3284 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-27 17:22:49 +00:00
allo	ff79c52fc0	bookmark users can now edit bookmarks. ... TO COME: tag bookmarks with username, list bookmarks of a special user, filter private bookmarks for users. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3274 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-23 10:24:26 +00:00
theli	49972b6193	*) SOAP: getPeerHash returns the proper value now ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3273 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-23 09:47:13 +00:00
karlchenofhell	15f0334cd3	- fixed IllegalThreadStateException in LogParser ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3265 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-21 14:45:52 +00:00
allo	f40169fcd7	preparing multiuser bookmarks ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3256 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-19 19:42:50 +00:00
hydrox	814a09a0ed	*) reversed r3250 and parts of r3252 (nanotime() is an java1.5 function) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3253 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-19 11:10:57 +00:00
karlchenofhell	d311e258f8	- adjusted LogStatistics to nano-seconds ... - removed patches of SVNs 3184/3185 preventing fast DHT git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3252 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-19 10:39:22 +00:00
hydrox	f7623f5d24	*) added missing measuring points for Parser-Runtime ... *) changed precision of Parser-Runtime from ms to ns git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3250 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-19 09:25:04 +00:00
orbiter	f3f99b19c6	extended search statistics ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3249 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-19 01:45:29 +00:00
orbiter	c0851ee943	refactoring: moved and renamed de.anomic.data.searchResults to plasma package ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3248 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-19 00:38:03 +00:00
allo	c39dda2374	finished refactoring of searchtemplates. ... now plasmaSwitchboard.searchFromLocal calculates a searchResults structure, which is parsed in the yacysearch/detailedSearch Servlets. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3244 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-18 10:42:36 +00:00
allo	35039982da	refactoring of search process: store results in a searchResults structure. At the moment, its just stored in it, and read from it again. ... Next step: return searchResults instead of serverObjects, and parse the results in the servlets. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3241 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-18 07:41:15 +00:00
orbiter	76fab83395	fixed bugs in seach statistics ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3240 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-18 00:26:16 +00:00
karlchenofhell	139810d504	- truncate attributes of new newsRecords if too long ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3239 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-17 23:45:53 +00:00
karlchenofhell	3c43e605ba	- don't accept malformed bookmarks, fix for: http://www.yacy-forum.de/viewtopic.php?t=3414 (first report) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3238 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-17 23:39:03 +00:00
orbiter	d07b132a0d	- fixed colors of network grafic ... - added option to activate write cache for seed-db - did not activate write cache because it did not work git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3236 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-17 19:39:31 +00:00
karlchenofhell	5d540b219e	- LogalizerHandler skips interfaces again ... - added LogParser stats to LogStatistics git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3234 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-17 17:01:20 +00:00
allo	e1fb3550ab	fix for profile ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3231 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-17 14:52:51 +00:00
hydrox	6faf9b70b7	*) LogParserPLASMA now counts its total runtime. ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3229 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-17 13:35:33 +00:00
hydrox	e5f854bc37	*) added LogalizerHandler-settings to yacy.logging. ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3228 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-17 13:25:11 +00:00
karlchenofhell	7c40197e42	- fixed error pages and <label>s for index.html ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3226 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-17 04:20:19 +00:00
allo	29aa7031d3	workaround for the snippets ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3225 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-16 21:35:25 +00:00
allo	b4457763e5	fix for putSafeXML and supertemplates. ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3223 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-16 21:06:31 +00:00
karlchenofhell	77b73aa7a8	- log-entries 'Indexed' are parsed correctly now ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3222 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-16 18:42:34 +00:00
karlchenofhell	aea199cb7b	- IndexTransfer is working again ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3220 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-16 17:18:25 +00:00
karlchenofhell	71112b1fe6	- added LogStatistics_p.html servlet based on the logalizer (indexing values not functional yet due to charset/regex problems) ... add the following to DATA/LOG/yacy.logging: --- # Properties for the LogalizerHandler de.anomic.server.logging.LogalizerHandler.enabled = true de.anomic.server.logging.LogalizerHandler.debug = false de.anomic.server.logging.LogalizerHandler.parserPackage = de.anomic.server.logging.logParsers --- and "de.anomic.server.logging.LogalizerHandler" to the list of global handlers git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3219 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-16 16:13:21 +00:00
allo	0c81bd39d4	XSS-safe put as default. ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3217 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-16 14:07:54 +00:00
karlchenofhell	bdda9e802f	- added some commented string constants to ease use of the result-table ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3215 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-16 05:51:39 +00:00
orbiter	5515571950	redesign of ymage classes ... - less memory usage - better usage of awt classes - drawing abstractions: preparations for movable objects for animation class - test applet for animations - known bugs: wrong colours for network picture git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3214 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-15 23:31:50 +00:00
orbiter	52c6461e6b	some bugfix for statistics ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3211 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-15 16:03:00 +00:00
(no author)	fe72b772cf	added a monitor page for search requests ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3206 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-15 01:50:57 +00:00
karlchenofhell	b3a2c79fec	- fix for last fix ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3202 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-14 17:47:25 +00:00
karlchenofhell	4b23e79f51	- quick-fix for http://www.yacy-forum.de/viewtopic.php?t=3358 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3201 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-14 17:31:08 +00:00
karlchenofhell	4dce5ec261	- if mem is too low but former GCs helped, the word-cache limit is only decreased now, if a subsequent GC doesn't ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3197 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-13 06:30:17 +00:00
hydrox	c27e88104c	*) getResults() should now work and compile properly with Java 1.4 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3191 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-12 11:12:09 +00:00
hydrox	a2fb54afff	*) Quickfix for http://www.yacy-forum.de/viewtopic.php?p=29973#29973 getResults() used a java 1.5 function (Output is temporally disabled until a sulution with 1.4 functions is implemented) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3190 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-12 10:39:22 +00:00
karlchenofhell	b873ad51ab	- fix for http://www.yacy-forum.de/viewtopic.php?t=3369 ... - merged netBude's alternative for tables in yacysearch.html & search results valid - added statistic info to index.html as proposed here: http://www.yacy-forum.de/viewtopic.php?p=29762#29762 - fixed error-log in httpTemplate git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3189 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-12 00:52:38 +00:00
auron_x	7fb30de8a4	*) made combined2prettyversion less restrictive and made it again easier to read ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3187 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-11 15:27:20 +00:00
hydrox	3acd90033c	*) added functions to get results from log-parsers (not documented yet) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3186 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-11 09:18:10 +00:00
borg-0300	1aa74bbd2b	update for last commit ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3185 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-09 23:57:04 +00:00
borg-0300	23e613b2ab	CPU & IO reduce (Index Distribution) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3184 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-09 17:23:29 +00:00
(no author)	c67d22116e	added exists-check based only on RAM index lookup: ... - faster double-check during crawling - less IO git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3179 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-08 13:13:30 +00:00
(no author)	37e53b4a6a	replaced tree database structure for seed db by flex data structure ... I don't know if this helps, we will find out... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3177 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-07 23:34:13 +00:00
karlchenofhell	340dc52a9d	- ConfigProfile_p.html now transmits usable encoding for other than 7-bit ASCII charset, see TODO in httpd.parseArg(String) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3174 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-07 02:07:27 +00:00
karlchenofhell	35fb671721	- updated DetailedSearch and ViewFile ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3173 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-06 12:21:13 +00:00
karlchenofhell	00aa9472d6	- added decode of HTML-entities in request lines ... - removed Bookmark symbol on search pages and surftips if not authenticated git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3172 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-06 11:05:50 +00:00
theli	d157201e08	*) IfesL for "Unexpected end of ZLIB" error message ... See: http://www.yacy-forum.de/viewtopic.php?t=3327 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3169 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-05 13:45:31 +00:00
karlchenofhell	0336480a3e	- the maxMemory-fix for the Sun JVM 1.4.2 wrongly also applied to 1.6, thx to NN ... - added logging of reducing word-cache (log-level fine) - disabled memprereq field in PerformanceQueues_p.html, because it is now set by the collections db - minor changes to ConfigSkins / -Language git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3165 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-05 08:00:05 +00:00
borg-0300	735c1f87b1	restart re-activated (windows) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3164 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-04 22:03:32 +00:00
hydrox	2c01508ada	*) fix for http://www.yacy-forum.de/viewtopic.php?p=29575#29575 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3162 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-04 11:36:38 +00:00
allo	6ff8359b98	possibility to use anonther bindPort than the externally reachable port. ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3161 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-03 21:00:07 +00:00
borg-0300	d2be3c674d	wrong cache values fixed ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3159 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-03 13:41:41 +00:00
karlchenofhell	df6281ba1f	- removed JS from DetailedSearch => valid ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3151 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-02 21:43:21 +00:00
hydrox	fb1d8b91af	*) changed Startpoints of IndexCleaner and IndexTransfer from ------------ to AAAAAAAAAAAA. ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3150 6c8d7289-2bf4-0310-a012-ef5d649a1542	2007-01-01 12:27:05 +00:00
karlchenofhell	d6eb699e8e	- fix for last commit (didn't know that the paragraph sign has an UTF-8-specific location) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3134 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-23 14:04:21 +00:00
karlchenofhell	41bc31d2c2	- ConfigAdvanced_p => XHTML (no invalid IDs) ... - removed unmappable characters from code git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3133 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-23 13:35:34 +00:00
orbiter	9b726ac366	release 0.50 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3132 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-23 04:26:05 +00:00
orbiter	036a0c828e	fix for auto-configuration of crawler thread memory ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3131 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-23 03:04:58 +00:00
orbiter	16f88c49d9	applied patch from http://www.yacy-forum.de/viewtopic.php?p=27392#27392 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3130 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-23 02:34:57 +00:00
orbiter	2aa6f978c2	fix for http://www.yacy-forum.de/viewtopic.php?p=29125#29125 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3129 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-23 01:54:28 +00:00
orbiter	a4e90bc1dc	fix + debug-code for http://www.yacy-forum.de/viewtopic.php?p=29126#29126 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3128 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-23 01:39:00 +00:00
borg-0300	6b5f28b746	answer for last commit: no ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3126 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-22 19:26:01 +00:00
borg-0300	d98ba7bc33	fix for memory limit computation ? ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3125 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-22 19:17:23 +00:00
orbiter	49ffedfd8b	patch from Fuchs against deadlock, ... see http://www.yacy-forum.de/viewtopic.php?p=28804#28804 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3122 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-22 15:26:29 +00:00
borg-0300	a4f63d187d	better map2string and NullPointer fix ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3119 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-22 13:13:41 +00:00
orbiter	c48374d14a	new memory limit computation for indexing queue ... shall better prevent outofmemory errors git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3118 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-22 12:54:56 +00:00
theli	a9f1d3a065	*) SOAP: adding function to get the local peer hash ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3117 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-22 11:31:48 +00:00
borg-0300	9659c004c3	fix for wrong date (Lotus-1 Peer) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3111 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-21 13:14:13 +00:00
orbiter	08ac4c5ed0	bugfix for http://www.yacy-forum.de/viewtopic.php?p=29045#29045 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3110 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-21 10:43:18 +00:00
orbiter	8e3bd17554	adopted DetailedSearch page to new ranking options ... - fixed bug http://www.yacy-forum.de/viewtopic.php?t=3265 - more attributes on page - attributes can be set as default for main search page - option to re-set the attributes to built-in values git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3109 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-21 03:09:46 +00:00
orbiter	93a7e88245	more ranking parameter usage ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3108 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-20 23:51:55 +00:00
orbiter	2dbea612c9	fixed display bug for image search preview ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3107 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-20 23:31:57 +00:00
orbiter	0a050bc043	enhanced ranking ... - redesign of data storage in plasmaSearchRankingProfile - profiles are extended by new ranking parameters - new RWI ranking parameters are considered during ranking - appearance attributes (i.e. emphasised text) is now considered - faster ranking - some attributes that had been checked during post-ranking can now be checked during pre-ranking phase - removed old ranking parameter on index.html page (will be replaced by profiles in the future) - ranking can now consider appearances of media content - snippet-loading for media types now work correctly (fetches only from the wanted media) - ranking-profiles can be handed over the remote peers and apply there also - re-search of same query with different domain now also re-triggers remote search git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3105 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-20 15:44:29 +00:00
orbiter	d0c32c6aeb	better protection against fraud peers ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3104 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-20 01:07:49 +00:00
orbiter	61798f0ae6	added option to distinguish between text crawl and media crawl ... - for each crawl start, there is now a flag for text and media - the localCrawl flag is superfluous - added new crawl profiles - if an image search is done, only media links are crawled for the snippets git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3100 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-19 03:10:46 +00:00
karlchenofhell	be941c4475	- "javascript:"-URLs are recognized as well (as intended formerly I assume) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3097 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-18 20:19:02 +00:00
karlchenofhell	a619ba3f49	- fix for String index out of range during URL parsing ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3096 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-18 19:32:01 +00:00
orbiter	febe6b114a	design update of crawler monitor ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3094 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-18 01:18:28 +00:00
orbiter	40049e0635	fixed media search snippet flow ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3092 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-17 22:52:58 +00:00
karlchenofhell	5763d20795	- added DATA/yacy.running file to indicate whether YaCy is running or not. Helpful for i.e. the updater and linux init-scripts ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3089 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-17 18:06:39 +00:00
borg-0300	05d0464377	only do 16 checks if "address" starts with "172."; ... better readably; git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3087 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-17 13:54:11 +00:00
rramthun	1a525710c1	*) cursor jumps now to searchbox on searchpages again ... *) added missing private IP-ranges for APIPA/Zeroconf and 172.16.0.0–172.31.255.255 *) Changed some seed-download-errors to warnings git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3086 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-17 13:21:17 +00:00
karlchenofhell	e17591acc3	- parse HTML arguments as UTF-8 strings ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3085 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-16 21:40:59 +00:00
karlchenofhell	d30932c7d8	- fix for fix... sry ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3084 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-16 16:43:52 +00:00
karlchenofhell	6118fb73ec	- added decode of UTF-16 escapes in url-arguments (%u0123), bugfix for http://www.yacy-forum.de/viewtopic.php?t=2762 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3083 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-16 16:40:40 +00:00
allo	782db9099d	version independent name for commons-pool lib ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3082 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-15 07:12:33 +00:00
orbiter	7ff86d6ba6	- image search now shows thumbnails (in bad order, but it works) ... - repaired DHT selection git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3081 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-14 02:48:37 +00:00
orbiter	ee3d91cb6b	print-out of links that result from contraint-filtering ... in search result git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3078 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-13 01:39:34 +00:00
borg-0300	f4737eebd6	Properties ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3076 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-12 12:51:31 +00:00
borg-0300	1fc75c0f67	better logging ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3075 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-12 12:46:51 +00:00
orbiter	e4570bffaf	-implemented a specialized snippet-fetch for media content ... -changed search result preparation for media search presentation git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3073 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-12 02:09:25 +00:00
low012	694a6e4f44	*) better text snipptes: any possible searchword (welt, linux, tag) in welt-linux-tag will be marked correctly now ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3072 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-11 15:19:35 +00:00
orbiter	bddc197453	reverted by-mistake removed change from low012/SVN 3068 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3070 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-11 11:07:36 +00:00
orbiter	1377c53aa3	extraction of media links from search results ... these links are mixed to the snippets for testing purpose (a final version will handle this differently) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3069 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-11 01:31:23 +00:00
low012	586add4c6c	*) Better snippets: words like GNU/Linux will not prevent Linux or GNU from being marked if they are searchword (see http://www.yacy-forum.de/viewtopic.php?t=2891) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3068 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-10 23:13:53 +00:00
borg-0300	29922d5dd2	changed writeMap ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3066 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-10 14:06:58 +00:00
borg-0300	5b00a04359	changed writeMap ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3065 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-10 13:56:32 +00:00
karlchenofhell	52abbd4131	- fix for wrong public IP if no hostname was set and IP was from range 192.168.*.* ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3063 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-09 22:00:34 +00:00
borg-0300	8b7c543885	NullPointer fix ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3061 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-09 15:51:34 +00:00
orbiter	937ccd4e76	fix for snippet-generation ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3060 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-09 02:13:43 +00:00
orbiter	559f41a001	fix for http://www.yacy-forum.de/viewtopic.php?p=28607#28607 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3059 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-08 23:35:32 +00:00
auron_x	c086c71f17	*) fixed ArrayIndexOutOfBoundsException ... --> http://www.yacy-forum.de/viewtopic.php?t=3210 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3058 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-08 14:56:18 +00:00
orbiter	c93cfdc23a	fix for http://www.yacy-forum.de/viewtopic.php?p=28564#28564 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3057 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-08 13:03:03 +00:00
orbiter	93a5ace330	fix for http://www.yacy-forum.de/viewtopic.php?p=28544#28544 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3056 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-08 12:57:17 +00:00
orbiter	bf0d820659	- added correct flagging of word properties ... - added self-healing to database in case that wrong free-pointers exist - added presentation of media links in snippets (does not yet work correctly) - code cleanup git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3055 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-08 02:14:56 +00:00
orbiter	10d888e70c	- added a media search for images, audio, video and applications ... - new search options on search page - new option in ViewInfo to display all links of a file - enhanced collection data structure git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3054 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-07 02:40:57 +00:00
orbiter	a603c4d5e8	more code simplifications ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3052 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-06 13:13:55 +00:00
orbiter	9a85f5abc3	cleanup ... - removed 'deleteComplete' flag; this was used especially for WORDS indexes - shifted methods from plasmaSwitchboard to plasmaWordIndex git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3051 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-06 12:51:46 +00:00
borg-0300	fbe1ee402b	plasmaCrawlLURL$kiter cleanup ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3050 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-06 12:49:13 +00:00
orbiter	773ba1e91a	- generalized object order handling ... - controlled object order for all database tables - migrated DHT position computation to correct base64-decoded values this also closed the 'gaps' in the dht positions git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3049 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-06 03:02:57 +00:00
borg-0300	15381cbf73	other bugfix ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3048 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-05 23:45:51 +00:00
borg-0300	ad65cc9d2f	NullPointer fixes ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3047 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-05 18:43:05 +00:00
borg-0300	d33745a7ea	NullPointer ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3046 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-05 17:48:01 +00:00
orbiter	3a4933b63c	bugfix for ... http://www.yacy-forum.de/viewtopic.php?p=28493#28493 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3045 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-05 12:32:19 +00:00
orbiter	109ed0a0bb	- cleaned up code; removed methods to write the old data structures ... - added an assortment importer. the old database structures can be imported with java -classpath classes yacy -migrateassortments - modified wordmigration. The indexes from WORDS are now imported to the collection database. The call is java -classpath classes yacy -migratewords (as it was) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3044 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-05 02:47:51 +00:00
orbiter	052f28312a	removed assortments from indexing data structures ... removed options to switch on assortments git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3041 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-02 19:34:59 +00:00
orbiter	2372b4fe0c	release 0.49 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3040 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-02 01:57:49 +00:00
orbiter	f8efb3c948	fixed a null pointer exception problem reported in the forum. ... I cant find the forum entry any more because my girlfriend switched off the power while the forum window was open. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3039 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-01 22:36:32 +00:00
orbiter	ad1e4aa88e	added selection of audio, video, image and application resources ... to search procedure. This function can currently not used through the search interface, but only through remote search. added accumulation of search attributes to enable the audio, video, image and application selection. fixed a problem with external URL representation generation git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3036 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-01 16:21:17 +00:00
orbiter	fb7902aa68	fix for http://www.yacy-forum.de/viewtopic.php?p=26142#26142 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3033 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-01 11:01:56 +00:00
orbiter	7cc4cec9c9	bugfix for assertion bugs documented in ... http://www.yacy-forum.de/viewtopic.php?p=28261#28261 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3030 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-12-01 01:30:05 +00:00
orbiter	97596af843	patch for wrong old RWI entries ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3028 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-11-30 14:24:41 +00:00
hydrox	2c69cc969a	*) more special chars removed ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3027 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-11-30 07:39:09 +00:00
hydrox	ebb42906f8	*) removed special characters ... *) added Copyright comments git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3026 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-11-30 07:33:54 +00:00
theli	954db729db	*) Bugfix for ArrayIndexOutOfBoundsException during SSL detection ... See: http://www.yacy-forum.de/viewtopic.php?p=28247#28247 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3025 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-11-30 06:38:53 +00:00
orbiter	5afd2f283c	fix for http://www.yacy-forum.de/viewtopic.php?p=27633#27633 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3024 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-11-30 01:17:28 +00:00
orbiter	9dfae90da8	better control of sortBound; should help to avoid an indexOutOfBound Exception ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3023 6c8d7289-2bf4-0310-a012-ef5d649a1542	2006-11-30 00:24:12 +00:00