yacy_search_server/source/net/yacy/crawler/robots
2013-07-09 14:42:39 +02:00
..
RobotsTxt.java replaced usage of log4j by ConcurrentLog where possible 2013-07-09 14:42:39 +02:00
RobotsTxtEntry.java better balancing and duetime-cumputation also for no-delay intranet 2012-10-30 11:28:49 +01:00
RobotsTxtParser.java fix for sitemap detection: the sitemap url was not visible if it 2013-05-10 04:56:58 +02:00