2872 Commits

Author	SHA1	Message	Date
orbiter	8d6a13bc07	refactoring of parsing-condensing-indexing process: ... - separated parts - removed storagePeer function next step will be parallelization of processes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4600 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-24 22:51:26 +00:00
orbiter	d3b06913ec	protection against seed-db failure during enumeration ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4598 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-23 23:47:41 +00:00
orbiter	5aa96dbc36	fix for shutdown configuration ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4596 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-23 13:14:57 +00:00
orbiter	93633abed8	- removed some debugging code from search process - should speed up now ... - added some profiling code to search event - more time details in PerformanceSearch_p.html git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4594 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-23 00:55:04 +00:00
orbiter	fba46c51d7	fixed non-termination bug in qsort ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4593 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-22 23:15:28 +00:00
orbiter	541b817502	refactoring of switchboard queueing ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4591 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-22 01:28:37 +00:00
orbiter	fc94fbe224	another improvement to the collection sorting ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4589 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-20 23:11:04 +00:00
orbiter	11270d450e	better quicksort-pivot computation: 30% faster (measured with test program) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4588 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-20 22:01:12 +00:00
orbiter	3e44293f07	- fixed a problem with thread pools in row collection ... - added a line-viewing feature in threaddump git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4587 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-20 14:21:58 +00:00
danielr	e43051b125	- fixed Threaddump output (html-escaped ie. <init>) ... - in EcoFS converted comments to javadoc git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4586 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-20 10:20:55 +00:00
orbiter	433ff855f7	- fixed another concurrency problem in collection sorting ... - fixed a typing problem that was introduced in svn 4579 and caused the crawl monitor to fail git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4585 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-19 23:47:24 +00:00
orbiter	19286fa2d1	tried to fix seed2.old.db-problem ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4584 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-19 22:35:19 +00:00
orbiter	f3996e63b8	tried to fix more deadlocks: ... - changed connection modes in ftpc - replaced sort tread pool in row collections by new one using util.concurrent. the old pool had caused blockings git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4582 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-19 11:23:43 +00:00
danielr	7008a218b3	avoid ConcurrentModificationException in plasmaCrawlerQueues ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4579 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-17 13:51:56 +00:00
orbiter	7150b463ff	changed handling of default values and database paths: ... - the default files yacy.init and for the network definition is now moved to the path defaults - the httpProxy.conf is renamed to yacy.conf - the DATA/INDEX/PUBLIC is renamed to the actual network nickname, which should be freeworld or sciencenet more menu entries - added apfelmaennchens alternative search page to the menu - added the new thread dump page to the server log menu point as submenu modifications - modified the thread dump page: sorting by thread type git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4575 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-16 22:31:54 +00:00
lulabad	25f5035f23	typo ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4571 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-16 15:13:12 +00:00
orbiter	7fd094fcbe	small bug in ftpc: did cot compile in Java 1.5 ... Please set compiler to Java 1.5-compliance git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4570 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-16 13:41:49 +00:00
danielr	f51bad8ae5	FTP: ... - report connection status (to break if no connection possible) - fixed isFolder() - additional error output - fixed paths with encoded symbols (ie. a%20file.txt) - refactoring git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4567 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-15 21:57:55 +00:00
danielr	820641938e	ftpc: fixed date parsing, some refactoring ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4566 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-15 10:56:47 +00:00
orbiter	4c584dff87	disabled soLinger to prevent that too many connections stay open (it's a TEST!) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4565 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-15 10:46:55 +00:00
orbiter	9c989fe5f7	fixed deadlock ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4562 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-15 00:49:16 +00:00
danielr	c565906050	FTP: ... - added maxFileSize-check - added timeout for download - fixed dirlist (when all filenames have spaces, change to absolute links) - enhanced isFolder() - make sure data connection is closed, so a new can be opened - refactoring git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4561 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-14 16:28:27 +00:00
danielr	1a7870df0d	FTP: source cleanup (added finals, indention for easier diffs) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4559 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-14 12:35:53 +00:00
orbiter	fa1090113d	- next try to fix the networking problem: ... set the maximum transfer size to less than MTU=1500-52: buffer size <= 1448 - some refactoring of transfer methods (naming) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4558 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-14 00:16:04 +00:00
orbiter	d87d295c68	one more try to fix the connection problem ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4556 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-12 13:13:11 +00:00
orbiter	a3dadcd89b	preventing that peer which return a false search result are disconnected ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4555 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-12 00:56:18 +00:00
orbiter	ba622bb240	addendum to svn 4553 ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4554 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-12 00:24:20 +00:00
orbiter	5530b8e1ca	reverted changes to yacy protocol classes: they caused the sciencenet to loose connections ... a comparisment with the main release 0.57 had been made: this showed a stable network This is an emergency operation to ensure availability of the sciencenet network. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4553 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-12 00:05:18 +00:00
orbiter	b664a53553	fix for NPE during search ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4552 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-11 15:30:26 +00:00
orbiter	b4ed937f1e	- modified zone navigation (does still not work correctly) ... - added dht switch in network definition - 0.574 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4550 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-11 11:09:38 +00:00
orbiter	8d0470a5c6	new method to compute search history IDs ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4549 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-10 23:40:56 +00:00
orbiter	65785da8f2	new method for best hash computation ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4548 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-10 23:28:05 +00:00
orbiter	9eddc1506b	- one try to fix the httpd problem ... - fix for handling of collection index that appears when removing elements - added another navigation method (stub, not working yet) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4543 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-09 23:58:22 +00:00
orbiter	7cc4ff05c9	some code enhancements and bugfixes ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4542 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-09 23:48:24 +00:00
danielr	6788f8f7c1	fixed error 'FTPC cannot change directory' ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4531 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-06 11:59:23 +00:00
orbiter	7ce76c8ff8	added missing file ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4530 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-05 22:57:53 +00:00
orbiter	bfed9c2da6	- some refactoring in search process ... - separated sidebars in new search interface and placed them in their own files which can be put in into the search page like plug-ins git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4529 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-05 21:46:55 +00:00
borg-0300	3445b1e10b	*better logging ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4526 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-05 13:41:54 +00:00
borg-0300	4b0339fec0	*fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=927 ... *remove some cast *Properties added git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4525 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-05 13:29:42 +00:00
orbiter	275a226cc5	refactoring ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4524 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-04 22:45:45 +00:00
apfelmaennchen	bc3d3b4c97	fixed rebuildTags() to correctly rebuild folders... ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4523 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-03 22:36:27 +00:00
danielr	fbe335db73	consistent use of de.anomic.server.serverMemory to get information about memory statistics ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4522 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-02 15:42:50 +00:00
orbiter	8c06436c4a	removing the error-db upon each time a start-up is made. ... This is necessary because the table uses a lot of RAM and the content is never re-used after Start-Up. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4520 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-03-01 09:44:33 +00:00
orbiter	4fdf695064	- fixed a bug in remote search that prevented that any results had been generated (!) ... - added a great number of printStackTrace and new exceptions that shall be used to find the cause for a bug in yacy client-server communication which causes the interruption of data transfer which then causes the parser bug for the seed strings. - tried to fix the communication bug on server-side (copy functions) Be aware that the log may be full of errors and bugs - there should not be more bugs but there is more to see git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4519 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-02-27 23:12:43 +00:00
borg-0300	0ddbed9451	Less memory consumption at start ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4518 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-02-27 20:09:22 +00:00
orbiter	1dce2f1079	more multithreading support: ... - replaced some synchronized classes by classes from util.concurrent - used a util.concurrent.SynchronousQueue to implement a persistent sorting thread in the very basic kelondroRowCollection which supports sorting with a second thread in case that a double-core processing CPU is used git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4517 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-02-27 15:16:47 +00:00
orbiter	6779b455d7	another fix for the punycode parser/generator (should work now!) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4516 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-02-26 23:00:20 +00:00
orbiter	1b127406d0	update to punycode encoding (still not working) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4515 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-02-26 22:37:23 +00:00
orbiter	83860507c9	- added punycode class from gnu idn library ... - added parser for international domains in yacyURL git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4514 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-02-26 22:18:40 +00:00
orbiter	253a453413	removed possible synchronization deadlock ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4511 6c8d7289-2bf4-0310-a012-ef5d649a1542	2008-02-26 14:05:43 +00:00