Commit Graph

111 Commits

Author SHA1 Message Date
theli
b594ee9a5a *) Adding possibility to configure if the http proxy should send the
X-forwarded-for header (requested by TeeSee)
   See: http://www.yacy-forum.de/viewtopic.php?t=2577

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2257 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-06-29 16:01:03 +00:00
theli
b20496e42b *) make DHT DoS check configurable (requested by KoH)
- check can be disabled via property indexDistribution.dhtReceiptLimitEnabled
   - upper bound can be configured via indexDistribution.dhtReceiptLimit

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2234 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-06-21 19:28:42 +00:00
orbiter
9937730560 added localization for Slovenky/Slovenian language, provided by Rostislav Svoboda
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2148 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-05-28 00:04:04 +00:00
orbiter
24686e50a2 - fixed a caching bug
- added counter for cache delete to distinguish between flush and delete
- changed some default paramenters for cache size settings

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2143 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-05-25 11:46:09 +00:00
theli
a520ab2e8c *) adding possibility to use an existing PKCS12 certificate for https
instead of creating a new one.

   Notes:
   This import is done automatically on startup if the following properties 
   are set in the config file:
     pkcs12ImportFile = 
     pkcs12ImportPwd = 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2139 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-05-24 07:15:42 +00:00
theli
df068cf23c *) adding first version of native SSL support for yacy
VERY EXPERIMENTAL!
   See: http://www.yacy-forum.de/viewtopic.php?p=18516

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2096 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-05-15 09:41:29 +00:00
orbiter
0c2cbc572b reverted a change to idle/busy time default setting.
There was a misunderstanding of the meaning of these values:
this is not the time that the process may take, instead it is the time
that the proces pauses after each loop.
increased the busysleep time pause from 2 seconds to 10 seconds.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2094 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-05-14 17:59:11 +00:00
rramthun
fb064fdbee Many additions/corrections to the German language file
Increased default time for DHT distribution, because many people complained about the high load on their systems, see http://www.yacy-forum.de/viewtopic.php?p=20922#20922

Avoided problems with some browsers and ampersands, see http://www.htmlhelp.com/tools/validator/problems.html#amp

Removed nearly invisible "bug" in menu

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2087 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-05-12 21:25:46 +00:00
orbiter
461548698c configuration of index transfer chunk size
see http://www.yacy-forum.de/viewtopic.php?p=20951#20951
new properties in yacy.init:
indexDistribution.minChunkSize = 5
indexDistribution.maxChunkSize = 1000
indexDistribution.startChunkSize = 50

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2073 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-05-09 11:43:10 +00:00
orbiter
c67125d8d3 better default values for queue times to prevent (D)DoS-similar situations
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2062 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-05-06 20:34:09 +00:00
orbiter
fe4ad214f1 configuration of time-out for fetching seed-lists during bootstraping
(weil das beim linuxtag nur auf meinem Notebook nicht funtioniert)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2054 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-05-03 21:27:30 +00:00
orbiter
93eb4f14e6 release 0.45
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2047 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-05-01 00:01:08 +00:00
orbiter
00e768b259 added Picture button to search results
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2011 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-04-07 13:38:09 +00:00
orbiter
ac114d69c0 tried to fix some problems with time-outs during search
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1994 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-04-03 23:26:08 +00:00
orbiter
63f39ac7b5 added 3 new crawling steering options:
- re-crawl by age of page (enter in minutes)
- auto-domain-filter
- maximum number of pages per domain
NOT YET TESTED!

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1949 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-23 16:05:16 +00:00
allo
59b9540eb6 save the current Skin
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1912 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-17 20:11:16 +00:00
hydrox
8da13088e9 *)removed multiple DHT_Distribution_Threads
*)boosted DHT_Distribution sending chunk parallel to multiple peers

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1890 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-15 11:27:43 +00:00
orbiter
bcd99fe83e introduced a second RAM cache for DHT transfer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1880 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-13 10:43:12 +00:00
theli
5ee0125046 *) adding possibility to configure the server port for seed uploading via scp.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1861 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-08 16:34:05 +00:00
orbiter
f0a38873eb * added yacysearch page with better view on search results
the old search page is obsolete and will be removed
* ConfigBasic.html is now the default page instead of index.html
  as long as no password is set

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1815 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-04 18:52:04 +00:00
orbiter
f0041d504d remove of several results from a single domain is stopped if the result set is smaller than the wanted number of results
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1811 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-04 00:10:49 +00:00
rramthun
c2f8318b4f Ok, the 08.03 commit ;-)
Added nice graphic for the 1-2-3-interface.
Used one graphic less (check.png-->ok.png). Saves disk/download-space. 
Updated italian translation.
Deleted my old version of the changelog as we have a new one.
Many corrections to the spelling.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1791 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-01 19:21:22 +00:00
auron_x
8c6f38fe70 *) added Blog to YaCy (atm not reachable through interface) -> Blog.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1790 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-01 07:40:25 +00:00
orbiter
87e90b9d8c refinements in ram cache flush procedure and default timing
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1768 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-25 21:05:19 +00:00
theli
fbbbf5f411 *) remote trigger for proxy-crawl
- remote crawling can now be enabled for the proxy crawling profile
   See: http://www.yacy-forum.de/viewtopic.php?p=17753#17753
   

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1758 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-24 09:35:54 +00:00
theli
2336f0f013 *) allow pausing/resuming of crawlJob Threads separately
- pausing/resuming localCrawls
   - pausing/resuming remoteTriggeredCrawls
   - pausing/resuming globalCrawlTrigger
   See: http://www.yacy-forum.de/viewtopic.php?t=1591

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1723 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-21 11:18:48 +00:00
hydrox
e2af2a3f45 *) it's now possible to run more then one indexDistribution-Thread
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1673 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-16 15:22:25 +00:00
theli
8fcb25f9f9 *) Setting via header according to rfc
- can be disabled via settings dialog

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1662 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-16 09:20:57 +00:00
allo
40199cea1f migration with svn Numbers
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1623 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-12 16:46:43 +00:00
allo
7bd61ab0e5 Locales will now be in DATA/HTDOCS. So it works with readonly htroot.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1527 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-04 10:50:22 +00:00
borg-0300
c5b6154136 added CRDistOn = true/false
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1372 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-18 02:18:23 +00:00
borg-0300
f1643228f5 more media extension (7z,asx,db,lx,lxl,scr,tbz,war)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1244 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-12-22 13:06:25 +00:00
borg-0300
395626dbe3 new media extension (dcm)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1242 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-12-22 12:20:40 +00:00
orbiter
5e2673e86c new version number to check performance enhancements
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1207 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-12-13 01:02:23 +00:00
orbiter
6b1a49ea23 fix for last commit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1181 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-12-07 10:46:14 +00:00
theli
44fa94ac52 *) Modifications for dbImport functionality
- dbImporter threads are now shutdown by the switchboard on server shutdown
   - adding possibility to pause a importer thread via GUI
   - Bugfix for abort function
     See: http://www.yacy-forum.de/viewtopic.php?p=13363#13363

*) Modification of content parser configuration
   - now it's possible to configure which parsers should be enabled for the proxy,
     crawler, icap, etc. separately
   - 

*) htmlFilterContentScraper.java
   - adding regular expression to normalize URLs containing /../ and /./ parts

*) httpc.java
   - adding functionality to unzip gzipped content
   - requested by roland: should be used later to allow gzipped seed lists

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1170 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-12-06 10:41:19 +00:00
theli
7e670894d9 *) Suppressing stackTraces in proxyError message for "connect timed out" errors
See: http://www.yacy-forum.de/viewtopic.php?t=1504
*) Increasing default http client timeout

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1129 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-25 00:40:35 +00:00
allo
4776f3f815 squid like redirctors
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1120 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-22 22:07:29 +00:00
orbiter
7ad4353fc6 fix for last commit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1111 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-20 19:01:57 +00:00
orbiter
0ec54d9c5f enhanced CR-file handling and added first RCI-evaluation tests
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1110 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-20 18:55:35 +00:00
theli
d0dfccdb77 *) Making CrawlStacker pool configurable via GUI and config file
See: http://www.yacy-forum.de/viewtopic.php?t=1448

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1087 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-15 12:46:22 +00:00
theli
86a9210264 *) indexing queue slots are now configurable via config file
See: http://www.yacy-forum.de/viewtopic.php?t=1480

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1081 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-15 08:25:46 +00:00
theli
fd58d5f8e6 *) Adding possibility to specify the interface / IP-Address where YaCy should bind to.
- e.g. Port = 192.168.0.1:8080
          Port = #eth0:8080
          Port = 8080

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1071 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-13 17:03:52 +00:00
orbiter
79818a320f introduced citation-rank transmission protocol and activate transport for anonymisation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1055 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-10 23:48:20 +00:00
orbiter
d2731418bf added creation of global ranking files and changed url normal form usage
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1046 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-07 12:33:02 +00:00
theli
fb766413d1 *) Changes on httpc dns caching
- Bugfix: old dns cache did not handle case insensitive hostnames correctly. 
   - adding a possibility to set domain name patterns defining hostnames that should not be cached by the httpc dns cache
     e.g. borg-300.dyndns.org
     This can be done by setting the new httpc.nameCacheNoCachingPatterns property
   - using httpc.dnsResolve wherever possible within the sourcecode
     [httpd.java,plasmaCrawlStacker.java]

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1044 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-07 10:57:54 +00:00
hydrox
295aff52a3 *)added offline-browsing-support (onlineMode=0)
*)online-mode now can be changed in Status.html

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1010 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-10-31 12:25:40 +00:00
orbiter
5b0911d7ea added new performance menu for search sequence configuration and monitoring
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@990 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-10-27 00:42:08 +00:00
theli
02d9af1a70 *) Restructuring and extending of Remote Proxy Support
- remote proxy configuration can now be "really" changed on the fly and takes effect immediately
   - adding possibility to disable remote proxy usage for yacy->yacy communication
   - adding possibility to disable remote proxy usage for ssl
   - restructuring proxy configuration so that it is stored in a single place now

*) Adding possibility to import a foreign word DB (or even more of them in parallel) 
   at runtime into the peers DB
   - this can be done by calling IndexImport_p.html 
   - ATTENTION: please not that at the moment this thread must be aborted via gui
     before a normal server shutdown is done. 
   - TODO: integrating IndexImport Thread into normal server shutdown
   - TODO: Adding posibility to import crawl-queues, etc. from foreign peers
   - TODO: removing old import function from yacy.java and calling the new routines instead

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@968 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-10-22 13:28:04 +00:00
allo
43a127ff3a allow httpsTunnels to other Ports than 443. (if secureHttps=false)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@940 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-10-14 12:51:56 +00:00