Commit Graph

6555 Commits

Author SHA1 Message Date
orbiter
34354cf9b2 added a servlet that has been removed in SVN 4881; this servlet is now splitted and will be used for a simple crawl start and a remote crawl monitor (not yet integrated into the interface)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6582 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-13 23:19:45 +00:00
orbiter
938e806182 tried to fix date problem that may have prevented that foreign peers stay in the network
- removed unused code
- removed possibly wrong utc difference correction

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6581 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-13 20:01:46 +00:00
orbiter
bd05e57d3b fix for http://forum.yacy-websuche.de/viewtopic.php?p=18563#p18563
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6580 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-13 18:41:33 +00:00
orbiter
5df628a2a4 - added BEncoder class
- added BEncodedHeap class that encodes B data structures and stores that to a heap
- refactoring of MapView, this is now named MapHeap to fit into the naming scheme of the BEncodedHeap

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6579 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-13 16:21:37 +00:00
orbiter
81035e7080 moved a sub-menu entry
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6578 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-13 16:18:23 +00:00
suessthomas
5f1565cc95 Fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2540&sid=507e4dd9e234af001ba60c0d8adf45be, and small Layout Changes.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6577 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-13 07:57:36 +00:00
orbiter
82f57f79e5 more PMD enhancements
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6576 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-13 00:23:07 +00:00
orbiter
5d930c96f0 more fixes to search result page navigation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6575 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-13 00:04:37 +00:00
suessthomas
1375d0214d minor changes on the de language File - Thanks Copro
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6574 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-12 21:28:16 +00:00
orbiter
8c520f128d reverted a change in ranking process committed this afternoon
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6573 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-12 20:56:37 +00:00
orbiter
a06f7ddb33 more PMD recommendations
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6572 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-12 20:53:19 +00:00
orbiter
eb79ceb3ff update to kelondro data structures
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6571 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-12 15:37:34 +00:00
orbiter
9bbd546e64 in live search, show at least 20 entries instead of only 10
this is a work-around for the problem that the search widget
does not load a second page if the first page did not fill up
the window with enough lines such that a scrollbar ist visible.
Because the scrollbar triggers loading of following pages, this
must be enforced with the trick that more result lines must be
shown after the first search.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6570 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-12 15:23:01 +00:00
orbiter
18172451a0 better search computation:
- increased sort limit, now 3000 entries, before: 1000
  this should cause that more results can be shown in case
  of strong limitating constraints, like domain navigation
- enhanced the sort process
- check against domain navigator bugs
- fix in sort stack
- showing now all naviagtion pages at first search (not only next page)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6569 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-12 15:01:44 +00:00
orbiter
d126d6c1b5 renamed the servlet WatchCrawler_p to Crawler_p
this was done because that servlet may be used for wget/cronjob
triggered crawl starts and it appears to be confusing that the
name of the crawl start servlet looks like a pure monitoring tool.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6568 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-12 10:05:28 +00:00
orbiter
66c0a8e849 more PMD recommendations
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6567 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-11 22:18:38 +00:00
orbiter
909a4f91c7 added a logging output for crawl starts that shows the URL that can be used to start the crawl again
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6566 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-11 18:10:39 +00:00
orbiter
bc96d74813 - clean-up of robots.txt parser
- added 'yacybot' as key to recognize robots.txt entries for YaCy
- removed unused method to get robots.txt from database

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6565 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-11 16:36:30 +00:00
orbiter
2113fcd7e5 - fixed usage of isEmpty() which is not available in java 1.5
- increased visibility of some methods

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6564 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-11 12:33:40 +00:00
orbiter
dd459281c8 applied code changes that are recommended by PMD
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6563 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-10 23:09:48 +00:00
lotus
eac2daf2e8 * reenable DHT if yet enough memory is available
* reset treshold on reconfiguratoin
(thanks to sixcooler)

* display status message in web interface

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6562 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-10 19:04:43 +00:00
lotus
0752634b8b log YaCy version on startup
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6561 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-10 16:10:11 +00:00
orbiter
d77a8f3b3e added some modifications recommended by PMD for better performance
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6560 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-10 01:40:26 +00:00
orbiter
d1973bae2a code cleanup: removed unused code and unused methods
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6559 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-10 00:42:49 +00:00
orbiter
a3b8b7b5c5 some redesign of the main menu structure:
- moved all index generation servlets to it's own main menu item, including proxy indexing
- removed external index import because this operation is not recommended any more. Joining an index can simply be done by moving the index files from one peer to the other peer; they will be merged automatically
- fix to prevent endless loops when disconnecting http sessions
- fix to prevent application of bad blacklist entries that can cause a 'Dangling meta character' exception

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6558 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-10 00:10:43 +00:00
lotus
ab3cf60dbe fix for npe
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6557 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-09 14:10:27 +00:00
orbiter
7f20963b41 add-on to last commit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6556 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-09 00:17:39 +00:00
orbiter
eeca2ded92 fix for http://forum.yacy-websuche.de/viewtopic.php?p=18500#p18500
- catch uncatched OOM
- less wasting of memory

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6555 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-09 00:08:16 +00:00
lotus
32972139af added nice configuration for the resource observer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6554 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-07 17:19:50 +00:00
orbiter
bb2e03761c - fix for deadlock with 100% CPU during search
- fix for failure of ranking because of a ConcurrentModificationException

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6553 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-07 12:41:43 +00:00
orbiter
3f771d2a16 fix for rss parser: be lazy when rss is not well-formed
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6552 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-07 01:02:23 +00:00
orbiter
dff4f95c78 some patches to get the torrent parser working
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6551 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-07 00:42:12 +00:00
suessthomas
75dfe4098c Clean up the de.language File, removed all unused Blocks. All unused Blocks copied in a new File and archived this on my Homepage (http://tokeek.de). The Cleanup de.language File is easy readable, is the basic for more improvements.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6550 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-05 21:24:36 +00:00
hermens
574f49903e Prevent blob merge from possibly losing the last container
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6549 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-05 01:34:46 +00:00
orbiter
83d05e9176 added sixcoolers hack with some modifications:
http://forum.yacy-websuche.de/viewtopic.php?p=15004#p15004
old index blobs where deletions have been made because of DHT transmission should be melted down to new blobs. This uses sixcoolers methods from the forum thread but modifies the process in such a way that the blobs are not merged with themselves but simply rewritten to smaller files.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6548 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-04 18:12:03 +00:00
orbiter
fbd24c2d84 integrated the torrent parser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6547 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-04 16:07:31 +00:00
orbiter
bd32f8b8cb added a torrent metadata file parser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6546 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-03 23:26:15 +00:00
orbiter
610e3ffffb Added new classes for the implementation of concurrent greedy algorithms.
These classes can be used to produce an abstract worker process that can be used for common problems in artificial intelligence, such as game playing and problem solving. These classes will be used as abstraction layer for a new search process in YaCy. These classes had been created while searching for an abstraction of the current search process. It turned out that the abstraction of the YaCy search process is also an abstraction for problems in artificial intelligence and therefore the classes had been designed in such a way that it covers not only the YaCy-specific problem but also the more generic problems in ai. To test the classes they had been used in a ConnectFour implementation (game playing). 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6545 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-03 22:51:14 +00:00
orbiter
d0b7bf9ca2 added a decoder class for Bencoding
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6544 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-01-03 22:44:09 +00:00
low012
04d91c67d6 *) added message if unrestricted access from localhost is possible
*) added link to admin access config page

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6543 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-12-28 14:01:41 +00:00
low012
028657f019 *) adding more SVN properties
*) minor changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6542 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-12-28 13:11:07 +00:00
low012
82d740050f *) adding more SVN properties
*) minor changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6541 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-12-28 12:26:50 +00:00
low012
e04cb8cef0 *) adding more SVN properties
*) minor changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6540 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-12-28 12:16:40 +00:00
low012
dcb1096fb0 *) adding more SVN properties
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6539 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-12-28 11:22:32 +00:00
low012
7d610e0063 *) minor changes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6538 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-12-28 11:20:34 +00:00
low012
82198acc06 *) minor changes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6537 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-12-28 11:06:49 +00:00
low012
b75547fc60 *) minor changes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6536 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-12-28 11:05:50 +00:00
low012
e8fb332567 *) script does not allow empty passwords anymore
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6535 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-12-28 10:16:39 +00:00
suessthomas
d933763b12 Small Changes on the de. language File and small correction to the ConfigAppearance_p.html page. Please test and report bugs to me (th-suess@gmx.de).
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6534 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-12-26 21:18:20 +00:00
lotus
9bee0ac780 more logging for DHTrule
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6533 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-12-21 14:02:00 +00:00