Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity

new limitation documentation

Browse Source 
just replaced two by four
This commit is contained in:
Michael Peter Christen 2020-12-22 16:33:12 +01:00
parent 3078b74e1d
commit 256fa3d985
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
htroot/CrawlStartSite.html
View File

@ -105,7 +105,7 @@
</fieldset>
<h3>Hints</h3>
<ul>
<li><h4>Crawl Speed Limitation</h4> No more that two pages are loaded from the same host in one second (not more that 120 document per minute) to limit the load on the target server.</li>
<li><h4>Crawl Speed Limitation</h4> No more that four pages are loaded from the same host in one second (not more that 120 document per minute) to limit the load on the target server.</li>
<li><h4>Target Balancer</h4> A second crawl for a different host increases the throughput to a maximum of 240 documents per minute since the crawler balances the load over all hosts.</li>
<li><h4>High Speed Crawling</h4> A 'shallow crawl' which is not limited to a single host (or site)
can extend the pages per minute (ppm) rate to unlimited documents per minute when the number of target hosts is high.