Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
7,214 Commits 1 Branch 20 Tags 836 MiB
bed79402be
Commit Graph

7214 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
orbiter
bed79402be introduction of a new remote search load control: the remote search has taken 10 results per peer with a time-out of 3 seconds so far. The attributes of number of results per peer and time-out time can now be configured. 
This has two aspects: the user who searches may want to increase these values to get more results and more load on the remote side and the user of the server which is accessed for this search may want to restrict the load. Both sides can now be configured. The server-site maximum load parameters are defined by a network definition and the client-side search request load can be defined by each user individually but when the remote search is done the requested service is limited to the network definition.

You can find now in the network definition file:
network.unit.remotesearch.maxcount and network.unit.remotesearch.maxtime
and in the yacy.conf file:
remotesearch.maxcount and remotesearch.maxtime

There is currently no web interface to define the client-side remote search attributes, please set them manually
    

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7548 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-04 13:44:00 +00:00
orbiter
6dfaf6fef7 fix for bug in deletion of old seeds 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7547 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-04 10:00:37 +00:00
lotus
ac13343f12 jre update 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7546 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-03 20:44:53 +00:00
lotus
fa5b911082 remove old libraries in case of update 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7545 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-03 18:34:32 +00:00
orbiter
993b9bc1a8 memory/performance hacks, less synchronization, better concurrency 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7544 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-03 11:30:04 +00:00
sixcooler
65bcc60808 stupid me: revert placement of closing connection which caused unclosed connections 
+ reuse sockets

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7543 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-03 04:16:33 +00:00
sixcooler
e3d75d6cd5 Not storing external header in an Header-Array and reduce a loop for its conversion. 
Ensure connection close if a OOM is thrown.
Ensure setting resolved host is set at the request.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7542 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-01 13:50:11 +00:00
orbiter
42d90664f3 - fixed a memory leak in the httpc.post method (no finish) 
- patched some more memory-saving relevant code
- some more minor bug fixes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7541 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-01 09:03:33 +00:00
orbiter
38dce547c0 better concurrency (less locking on date formatting) more logging and minor bug fixes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7540 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-28 06:28:29 +00:00
mikeworks
85f5c02deb de.lng: Updated German translation and removed old unused strings, e.g. 8080 -> 8090 and Search Portal translations 
Bookmarks.html, Ranking_p.html, base.css: Fixed XHTML errors to make pages compatible again - switched div -> span inside dt and replaced css definition of id (unique) with class (class of elements)
header.template: Fixed link to berliOS changelog by replacing & -> & and adding translation for German page by refering to German berliOS UI ;-]

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7539 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-28 00:22:05 +00:00
f1ori
59dea3a284 * implement url proxy, a proxy via the url http://peer:port/proxy.html?url=http://domain.tld/path 
* enable with proxyURL = true
* could be useful to browse specific pages with proxy or use own improvements in proxy

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7538 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-27 21:39:38 +00:00
mikeworks
8b7b783c49 Tray.java: Broke the build on with wrong non UTF-8 encoded file and french umlauts (unmappable character for encoding UTF8) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7537 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-27 15:01:46 +00:00
mikeworks
db65ada467 Tray.java: Added localization for french tray icon command - although this can probably also done better than with if statements. (preferably also from the locales file) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7536 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-27 11:42:33 +00:00
mikeworks
073ee5525f de.lng: Updated German translation added missing line no change when no other network was selected 
YaCy logos: removed outer white line that made the logo blitz on black or dark themes http://forum.yacy-websuche.de/viewtopic.php?f=15&t=1724#p21655

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7535 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-27 01:19:46 +00:00
orbiter
89d337841c more logging for OOMs 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7534 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-26 09:27:15 +00:00
orbiter
b1781d7aae some more performance hacks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7533 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-26 01:24:49 +00:00
orbiter
b2f147d28e performance hack: excluded map encoding in many cases from synchronization block, especially when doing an iteration 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7532 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 21:29:55 +00:00
orbiter
5e186e0122 continuing the fight against deadlocks during time formatting: better caching. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7531 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 21:11:53 +00:00
orbiter
dec24244cf added convenience class to generate UTF StringBody objects with a default UTF8 charset. 
Reason: if this is not used in StringBody-Class initialization, a default charset name is parsed.
This is a synchronized process and all classes using default charsets synchronize at that point
Synchronization is omitted if this class is used
 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7530 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 13:26:09 +00:00
orbiter
1110d16af9 performance hack: replaced generic row.getColBytes() call with row.getPrimaryKeyBytes() where the column is 0 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7529 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 12:41:27 +00:00
orbiter
19b2a50578 - enhanced date formatter cache 
- added more instances of formatter objects to different classes to make them independent in case of lockings that may applay during synchronization of the date formatter object (date formatting is not thread-safe and must be synchronized therefore)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7528 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 12:23:00 +00:00
orbiter
48a61c39a3 speed hacks in BLOB ArrayStack: 
- more concurrency if possible
- less threads if no concurrency necessary

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7527 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 11:58:01 +00:00
orbiter
a92d80a545 performance enhancements using an alternative to a insensitive collator (a complex string compare): 
- less synchronizations
- better speed
..at most important and commonly used classes: http headers, url parsing and html parsing

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7526 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 11:23:57 +00:00
orbiter
f2e8ffd768 enhancement in synchronisation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7525 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 01:19:39 +00:00
sixcooler
bcea497644 next try to fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=3193&start=0&sid=b98aa9a7466397602b436eb45f4a9d39 
tested proxy, crawl, updatedownload - please do further testing!

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7524 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 01:16:15 +00:00
orbiter
ad7fcb9d61 Enhanced Base64Order transformation: less overhead (transformation between StringBuilder and byte[]) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7523 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 00:56:31 +00:00
orbiter
f95e50ec3d more explanation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7522 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 00:06:31 +00:00
orbiter
bb36bf841a emergency commit (sorry sixcooler for not waiting) because without that automatic updating peers would not be able to do the next update. 
Please see http://forum.yacy-websuche.de/viewtopic.php?p=22059#p22059

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7521 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 00:02:04 +00:00
sixcooler
8ad4e10491 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=3193&start=0&sid=b98aa9a7466397602b436eb45f4a9d39 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7520 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-24 23:15:27 +00:00
orbiter
0ce17d823a - fixed bug in ordering 
- fixed ConcurrentModificationException in set join


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7519 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-24 10:32:46 +00:00
orbiter
dec4f36700 - fix for missing favicons in search widgets 
- fix for bad digest/hash computation in case of interrupts to class

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7518 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-23 23:04:05 +00:00
lotus
28c7615f24 fix missing translation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7517 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-23 15:02:00 +00:00
orbiter
804ae2275b - do not delete idx and gap files if the heap is not modified 
this change may have bugs in it which may cause damage to your existing data. please use with care.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7516 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-23 11:28:12 +00:00
orbiter
e3ef4e3021 - increased default peer ping time from 2 minutes to 1 minute 
- filtering out too old peers when reading seed lists (limit is now 240 minutes)
- added concurrent host names resolving in front of the http client because the http client uses the java built-in DNS resolve which is not multithreading-safe (i have seen deadlocks in thread dumps showing that this bug in jdk is still there)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7515 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-23 09:42:01 +00:00
orbiter
5e45ded8e2 - removed locks from WordReference 
- refactoring of HeapReader/Writer

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7514 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-23 00:32:16 +00:00
orbiter
cd19d0517e added dns resolve to HTTPClient POST using a dns cache to prevent that that not-thread-safe built-in dns cache inside apache http client is used 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7513 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-22 22:58:19 +00:00
orbiter
d28f8040e0 removed unnecessary recording function that caused also a performance problem after serving too much files 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7512 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-22 13:33:28 +00:00
orbiter
af87af0d4c - removed synchronization in serverSwitch which should improve speed 
- fixed wrong assert in network graph
- enhanced double check method in table class

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7511 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-22 12:56:25 +00:00
orbiter
4bd65532da initialization of libraries concurrently (faster start-up) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7510 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-22 10:57:38 +00:00
orbiter
24909b3006 slightly less restrictive values for DoS 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7509 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-21 15:24:09 +00:00
orbiter
311f57d360 DoS to prevent online snippet fetch: allow read from cache. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7508 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-21 15:11:03 +00:00
orbiter
83a38b16bb added link to svn log 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7507 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-21 12:11:12 +00:00
orbiter
57e6728cb7 - removed usage of /etc/alternatives/www-browser because of problems with lynx, see: 
http://forum.yacy-websuche.de/viewtopic.php?p=21959#p21959
  please look if the browser that is linked with /etc/alternatives/www-browser can be detected and insert call again if
  it can be made sure that this does not call lynx
- replaced severe warnings with just warnings in yacyClient

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7506 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-21 11:53:45 +00:00
orbiter
91eeaf2cff fix in ftp client 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7505 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-21 10:51:15 +00:00
orbiter
09a98a6ae4 - disabled arrow keys in search (pages switch) again -- too disturbing when editing the query line 
- fixed alt text in AccessGrid

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7504 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-21 10:30:58 +00:00
orbiter
e717bf74ba more logging, more care about OOMs 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7503 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-21 09:51:05 +00:00
orbiter
d84b4a072e healing for some OOM problems 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7502 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-21 00:38:49 +00:00
orbiter
4aa406fb0f added log output to find bug in url parser for short hosts 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7501 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-21 00:07:00 +00:00
orbiter
82f262f685 - enhanced circle drawing speed 
- beautified 'moving dot' feature (using smaller and correctly positioned dots)
- added moving dots to DHT transfer

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7500 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-21 00:03:11 +00:00
suessthomas
6e93a3eb2f Small correction 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7499 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-19 22:02:44 +00:00