Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-22 00:00:59 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
ea033f8f8e
yacy_search_server/source/net/yacy/crawler/robots
History
orbiter 5aa5202adf fixes for filesystem indexing
2012-11-24 10:27:29 +01:00
..
RobotsTxt.java fixes for filesystem indexing 2012-11-24 10:27:29 +01:00
RobotsTxtEntry.java better balancing and duetime-cumputation also for no-delay intranet 2012-10-30 11:28:49 +01:00
RobotsTxtParser.java replaced the custom robots.txt loader by the standard http loader 2012-10-28 22:48:11 +01:00