Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
6,190 Commits 1 Branch 20 Tags 836 MiB
713cb26a27
Commit Graph

6190 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
lotus
713cb26a27 update for memory observer algorithm 
disable dht if memory is less than treshold
after 4 times, maximum 11 minutes between each detection

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6517 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-06 17:45:48 +00:00
orbiter
a44112b562 - moved index cleaner to blacklist submenu, because the index cleaner cleans the index with the blacklist 
- version switch to 0.93 to reflect advancements

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6516 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-03 14:26:30 +00:00
orbiter
4782d2c438 fix for search bug that appeared when looking at page 3 of results or further 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6515 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-03 12:25:03 +00:00
orbiter
29fde9ed49 better control of ranking order in sort stack 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6514 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-03 00:36:07 +00:00
orbiter
93caa38d55 fix for bug in SortStack (did not appear to shrink according to required size) - caused bad and unsufficient search results 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6513 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-03 00:20:53 +00:00
orbiter
66923ebc6c - modified method in RequestHeader that delivers the host name of requester: no more reverse domain lookup (may have killed interface performance in some cases) 
- added logging output for shutdown servlet: show ip of requester of the shutdown

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6512 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-02 22:56:28 +00:00
orbiter
e34e63a039 preset of proper HashMap dimensions: should prevent re-hashing and increase performance 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6511 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-02 14:01:19 +00:00
orbiter
4a5100789f replaced _all_ size() == 0 with isEmpty() and all size() > 0 with !isEmpty(). The isEmpty() method is much faster in some cases, especially when used to access badly balanced hashtables where an size() operation becomes a large iteration. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6510 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-02 00:37:59 +00:00
orbiter
f4946eaf27 - better thread dump 
- suppressed one server exception

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6509 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-01 22:53:36 +00:00
orbiter
9743b70d1c disabled keep-alive of server, not really needed for speed but a cause for much trouble and memory occupancy 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6508 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-01 19:14:16 +00:00
lotus
a4e72928ad classpath correction 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6507 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-29 19:56:25 +00:00
low012
173aa84611 *) Ooops! 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6506 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-29 19:37:49 +00:00
low012
e47e8a8bfe *) corrected classpath settings 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6505 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-29 19:36:15 +00:00
orbiter
491ba6a1ba - some refactoring in workflow 
- some refactoring in search process
- fixed image search for json and rss output
- search navigation on bottom of search result page in cases where there are more than 6 results on page
- fixes for number of displayed documents
- disabled pseudostemming

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6504 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-24 11:13:11 +00:00
orbiter
969123385b added json and rss output for image search 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6503 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-23 16:10:50 +00:00
orbiter
016fdfa310 stub for json image search (not yet ready) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6502 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-23 00:43:51 +00:00
lotus
82258b2882 default skin: rounded fieldsets 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6501 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-21 12:42:59 +00:00
low012
40900eebb4 *) better comment 
*) r6500!

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6500 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-20 21:53:43 +00:00
low012
19fd5a3c33 *) script is much more userfriendly now 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6499 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-20 21:49:39 +00:00
orbiter
d183f8d980 refactoring (moved code from ContentTransformer to TemplateEngine) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6498 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-20 14:57:00 +00:00
orbiter
23aef43786 - better synchronization in SortStack 
- better ThreadGroup organization
- less worker threads for media search (64 was too much...)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6497 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-20 14:35:33 +00:00
orbiter
7b1f5b0430 - better media search ranking 
- better concurrency with enhanced synchronization in sort stack

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6496 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-20 13:19:12 +00:00
orbiter
4df88a4e7a - fixes for missing or bad hashCode computation 
- fixes for bad equals() methods that had not been used by hash maps and therefore some classes did not work as objects in hash maps.
- this may also affect some cases where double-checks should have been, but did not work.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6495 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-20 12:11:56 +00:00
orbiter
dbdf2570ba added comparator and more fixes for SortStack/SortStore 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6494 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-20 03:30:48 +00:00
orbiter
d2938c44a1 - added bmp parser to the document parsers 
- image parser that implement the document parser interface return itself in the list of images of the document which should cause that the parsed images contribute to the image search

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6493 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-19 23:22:53 +00:00
orbiter
1dff620181 Better implementation of SortStack and SortStore and adoptions in all using classes to implement the necessary Comparable interface and hash code computation. 
The better SortStack performance affects crawling and image search speed and quality.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6492 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-19 13:49:28 +00:00
orbiter
fe41a84330 some enhancements in web caching: avoid double loading of response metadata and/or content 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6491 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-19 10:17:26 +00:00
orbiter
06d0dcde20 more enhancements to image search 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6490 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-19 00:43:42 +00:00
orbiter
4c6312d103 enhanced image search 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6489 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-18 23:56:05 +00:00
orbiter
2d8f3ee301 some performance hacks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6488 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-18 16:03:28 +00:00
orbiter
36fbfdcb21 more performance for remote search 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6487 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-18 15:13:06 +00:00
orbiter
5c7b32a4fa better performance for list api (blacklist transfer) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6486 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-18 15:11:52 +00:00
orbiter
94b2a664f3 - use a static DiskFileItemFactory (one instantiation is enough) 
- use more memory for the DiskFileItemFactory to avoid IO when POST commands come

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6485 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-18 15:05:51 +00:00
orbiter
267108470f testing jmx console for yacy: 
- start YaCy with startYACY.sh -l
- open jconsole and open localhost:9999

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6484 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-18 14:54:30 +00:00
suessthomas
d9835d8568 Conversion of the de. Language to UTF-8 format. This version will replace the now used language File in the future. Please test and report bugs to me (th-suess@gmx.de). 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6483 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-17 16:19:46 +00:00
orbiter
fd0658ce7c avoid forced execution of InetAddress.getLocalHost() at startup, because that hangs at some strangely declared linux configurations. The Domains.localHostAddresses object is first instantiated with a more simple logic and enriched with more host addresses using a concurrent thread that will not block a startup process. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6482 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-16 23:08:20 +00:00
orbiter
013f337d3f - avoid unnecessary host name lookups for localhost 
- avoid unnecessary reverse domain name lookups for remote access

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6481 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-16 23:00:54 +00:00
low012
141712ec95 *) small changes to UI 
*) password will not be deleted anymore when changing to unlimited access from localhost

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6480 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-15 22:15:25 +00:00
lotus
12dd8ece3e enabled memory protection from 6459 with 50000kb (disables dht-in) 
this should only apply if there is really little memory available because it is checked by threads explictly requesting memory

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6479 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-13 16:26:45 +00:00
orbiter
20c5d78a5c fix for a ConcurrentModificationException 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6478 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-11 23:31:12 +00:00
orbiter
5afd9f7a91 fix for crlf writing 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6477 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-11 22:50:44 +00:00
orbiter
7144d2df6e added crawlReceipt servlet as individual class to examine OOM problem as documented in 
http://forum.yacy-websuche.de/viewtopic.php?p=18120#p18120

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6476 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-11 16:12:00 +00:00
orbiter
2d3c98b742 less computation within synchronized blocks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6475 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-11 16:07:40 +00:00
orbiter
1a146b0d73 added a patch to ignore bad mime-ignore patterns 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6474 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-11 15:49:53 +00:00
orbiter
29fe436e36 - fixed post-ranking including prefer mask 
- enhanced a core database access method / less wasted ram

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6473 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-09 19:14:51 +00:00
orbiter
e9ab130ad7 fixed start/stop using ant 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6472 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-09 19:12:33 +00:00
orbiter
5399d1e2bc refactoring (reason: get more abstraction to use the blacklist class; for integration in other servlets) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6471 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-08 22:58:57 +00:00
orbiter
a97fdb4566 catch for NPE in image parser 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6470 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-07 23:39:31 +00:00
low012
9ee7862710 *) added configuration script 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6469 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-07 13:21:27 +00:00
orbiter
534182559c removed concurrency hacks from SplitTable because it showed deadlock-like situation. 
see thread dump at http://forum.yacy-websuche.de/viewtopic.php?p=18081#p18081

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6468 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-11-07 11:52:03 +00:00