Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
6,261 Commits 1 Branch 20 Tags 836 MiB
007f8297de
Commit Graph

78 Commits

Author SHA1 Message Date
orbiter
4f1f4863c4 fix for deadlock when initializing a SplitTable with a file of size 0, see also: 
http://forum.yacy-websuche.de/viewtopic.php?p=18594#p18594

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6587 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-14 23:03:48 +00:00
orbiter
cc5dcf69ff missing change for last commit 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6585 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-14 14:20:18 +00:00
orbiter
ca1ef9a079 fix for http://forum.yacy-websuche.de/viewtopic.php?p=18584#p18584 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6584 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-14 13:38:14 +00:00
orbiter
938e806182 tried to fix date problem that may have prevented that foreign peers stay in the network 
- removed unused code
- removed possibly wrong utc difference correction

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6581 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-13 20:01:46 +00:00
orbiter
5df628a2a4 - added BEncoder class 
- added BEncodedHeap class that encodes B data structures and stores that to a heap
- refactoring of MapView, this is now named MapHeap to fit into the naming scheme of the BEncodedHeap

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6579 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-13 16:21:37 +00:00
orbiter
82f57f79e5 more PMD enhancements 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6576 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-13 00:23:07 +00:00
orbiter
a06f7ddb33 more PMD recommendations 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6572 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-12 20:53:19 +00:00
orbiter
eb79ceb3ff update to kelondro data structures 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6571 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-12 15:37:34 +00:00
orbiter
18172451a0 better search computation: 
- increased sort limit, now 3000 entries, before: 1000
  this should cause that more results can be shown in case
  of strong limitating constraints, like domain navigation
- enhanced the sort process
- check against domain navigator bugs
- fix in sort stack
- showing now all naviagtion pages at first search (not only next page)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6569 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-12 15:01:44 +00:00
orbiter
2113fcd7e5 - fixed usage of isEmpty() which is not available in java 1.5 
- increased visibility of some methods

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6564 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-11 12:33:40 +00:00
orbiter
dd459281c8 applied code changes that are recommended by PMD 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6563 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-10 23:09:48 +00:00
lotus
eac2daf2e8 * reenable DHT if yet enough memory is available 
* reset treshold on reconfiguratoin
(thanks to sixcooler)

* display status message in web interface

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6562 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-10 19:04:43 +00:00
orbiter
d77a8f3b3e added some modifications recommended by PMD for better performance 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6560 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-10 01:40:26 +00:00
orbiter
7f20963b41 add-on to last commit 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6556 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-09 00:17:39 +00:00
orbiter
eeca2ded92 fix for http://forum.yacy-websuche.de/viewtopic.php?p=18500#p18500 
- catch uncatched OOM
- less wasting of memory

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6555 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-09 00:08:16 +00:00
lotus
32972139af added nice configuration for the resource observer 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6554 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-07 17:19:50 +00:00
hermens
574f49903e Prevent blob merge from possibly losing the last container 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6549 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-05 01:34:46 +00:00
orbiter
83d05e9176 added sixcoolers hack with some modifications: 
http://forum.yacy-websuche.de/viewtopic.php?p=15004#p15004
old index blobs where deletions have been made because of DHT transmission should be melted down to new blobs. This uses sixcoolers methods from the forum thread but modifies the process in such a way that the blobs are not merged with themselves but simply rewritten to smaller files.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6548 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-04 18:12:03 +00:00
orbiter
d0b7bf9ca2 added a decoder class for Bencoding 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6544 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-03 22:44:09 +00:00
low012
028657f019 *) adding more SVN properties 
*) minor changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6542 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-28 13:11:07 +00:00
low012
82d740050f *) adding more SVN properties 
*) minor changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6541 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-28 12:26:50 +00:00
low012
e04cb8cef0 *) adding more SVN properties 
*) minor changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6540 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-28 12:16:40 +00:00
low012
dcb1096fb0 *) adding more SVN properties 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6539 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-28 11:22:32 +00:00
low012
7d610e0063 *) minor changes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6538 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-28 11:20:34 +00:00
lotus
9bee0ac780 more logging for DHTrule 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6533 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-21 14:02:00 +00:00
orbiter
c14233a933 fix for a OOM in MapView that can cause unavailability of 
- seed list
- bookmarks
during very low memory configuration

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6529 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-15 12:23:37 +00:00
orbiter
37245430c3 fix for NPE during DHT RWI selection 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6527 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-15 00:02:10 +00:00
orbiter
959b38b61b fix for memory tracker 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6526 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-14 20:23:11 +00:00
orbiter
a37878b7d5 url parser regex performance hack 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6524 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-10 14:40:32 +00:00
orbiter
362b7a929b added extensive memory protection logic to avoid out of memory errors that may be caused by the RowCollection memory allocation function 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6521 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-09 23:27:26 +00:00
orbiter
8281e29963 - more configuration for profiling graph (number of events) 
- more logging for a shutdown: print reason and accessing IP into log


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6520 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-08 14:25:51 +00:00
lotus
713cb26a27 update for memory observer algorithm 
disable dht if memory is less than treshold
after 4 times, maximum 11 minutes between each detection

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6517 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-06 17:45:48 +00:00
orbiter
29fde9ed49 better control of ranking order in sort stack 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6514 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-03 00:36:07 +00:00
orbiter
93caa38d55 fix for bug in SortStack (did not appear to shrink according to required size) - caused bad and unsufficient search results 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6513 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-03 00:20:53 +00:00
orbiter
e34e63a039 preset of proper HashMap dimensions: should prevent re-hashing and increase performance 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6511 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-02 14:01:19 +00:00
orbiter
4a5100789f replaced _all_ size() == 0 with isEmpty() and all size() > 0 with !isEmpty(). The isEmpty() method is much faster in some cases, especially when used to access badly balanced hashtables where an size() operation becomes a large iteration. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6510 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-02 00:37:59 +00:00
orbiter
491ba6a1ba - some refactoring in workflow 
- some refactoring in search process
- fixed image search for json and rss output
- search navigation on bottom of search result page in cases where there are more than 6 results on page
- fixes for number of displayed documents
- disabled pseudostemming

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6504 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-24 11:13:11 +00:00
orbiter
23aef43786 - better synchronization in SortStack 
- better ThreadGroup organization
- less worker threads for media search (64 was too much...)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6497 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-20 14:35:33 +00:00
orbiter
7b1f5b0430 - better media search ranking 
- better concurrency with enhanced synchronization in sort stack

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6496 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-20 13:19:12 +00:00
orbiter
4df88a4e7a - fixes for missing or bad hashCode computation 
- fixes for bad equals() methods that had not been used by hash maps and therefore some classes did not work as objects in hash maps.
- this may also affect some cases where double-checks should have been, but did not work.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6495 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-20 12:11:56 +00:00
orbiter
dbdf2570ba added comparator and more fixes for SortStack/SortStore 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6494 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-20 03:30:48 +00:00
orbiter
1dff620181 Better implementation of SortStack and SortStore and adoptions in all using classes to implement the necessary Comparable interface and hash code computation. 
The better SortStack performance affects crawling and image search speed and quality.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6492 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-19 13:49:28 +00:00
orbiter
4c6312d103 enhanced image search 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6489 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-18 23:56:05 +00:00
orbiter
fd0658ce7c avoid forced execution of InetAddress.getLocalHost() at startup, because that hangs at some strangely declared linux configurations. The Domains.localHostAddresses object is first instantiated with a more simple logic and enriched with more host addresses using a concurrent thread that will not block a startup process. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6482 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-16 23:08:20 +00:00
orbiter
013f337d3f - avoid unnecessary host name lookups for localhost 
- avoid unnecessary reverse domain name lookups for remote access

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6481 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-16 23:00:54 +00:00
orbiter
5afd9f7a91 fix for crlf writing 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6477 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-11 22:50:44 +00:00
orbiter
2d3c98b742 less computation within synchronized blocks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6475 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-11 16:07:40 +00:00
orbiter
29fe436e36 - fixed post-ranking including prefer mask 
- enhanced a core database access method / less wasted ram

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6473 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-09 19:14:51 +00:00
orbiter
5399d1e2bc refactoring (reason: get more abstraction to use the blacklist class; for integration in other servlets) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6471 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-08 22:58:57 +00:00
orbiter
534182559c removed concurrency hacks from SplitTable because it showed deadlock-like situation. 
see thread dump at http://forum.yacy-websuche.de/viewtopic.php?p=18081#p18081

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6468 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-07 11:52:03 +00:00