yacy_search_server/source/net/yacy/crawler/retrieval
orbiter 8952153ecf update to Balancer algorithm:
- create a load list from the current list of known hosts
- do not create this list for each Balancer.pop access
- create the list from those hosts which have a zero-waiting time
- select 1/3 from that list which have the most urls waiting
- get hosts from the wainting list in random order
- fixes for some delta-time computations
- always load all urls from hosts which have never been loaded before
2012-10-28 13:24:49 +01:00
..
FileLoader.java removed the option to prevent removal of & parts inside of the 2012-10-10 11:46:22 +02:00
FTPLoader.java update to Balancer algorithm: 2012-10-28 13:24:49 +01:00
HTTPLoader.java update to Balancer algorithm: 2012-10-28 13:24:49 +01:00
ImporterException.java
Request.java replaced some more .getBytes() with UTF8/ASCII.getBytes() 2012-10-09 12:14:28 +02:00
Response.java replaced some more .getBytes() with UTF8/ASCII.getBytes() 2012-10-09 12:14:28 +02:00
RSSLoader.java removed the option to prevent removal of & parts inside of the 2012-10-10 11:46:22 +02:00
SitemapImporter.java - removed dependencies from URIMetadataRow and made direct access to 2012-10-16 18:11:57 +02:00
SMBLoader.java removed the option to prevent removal of & parts inside of the 2012-10-10 11:46:22 +02:00
URLRewriterLibrary.java added an url rewriter which can be used to remove session ids from urls 2012-10-09 11:24:48 +02:00