Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
0f0393e5e3
yacy_search_server/source/net/yacy/crawler/robots
History
luccioman f0639d810c Customized name for Threads still using the default "Thread-n" pattern. 
This makes threads monitoring easier to read.
2016-10-22 17:17:21 +02:00
..
RobotsTxt.java Customized name for Threads still using the default "Thread-n" pattern. 2016-10-22 17:17:21 +02:00
RobotsTxtEntry.java Robots : properly handle URLs including non ASCII characters 2016-10-12 11:25:36 +02:00
RobotsTxtParser.java support for multiple sitemaps in robots.txt 2014-03-14 13:33:23 +01:00