Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
0f3d8890db
yacy_search_server/source/net/yacy/crawler/robots
History
Michael Peter Christen 47b1c81d08 - refactoring 
- generalized writing of url attributes to solr documents
- added more url attributes to error documents
2013-08-20 15:46:04 +02:00
..
RobotsTxt.java Added 'final' for all exception blocks as this helps the Java compiler 2013-07-17 18:31:30 +02:00
RobotsTxtEntry.java - refactoring 2013-08-20 15:46:04 +02:00
RobotsTxtParser.java fix for sitemap detection: the sitemap url was not visible if it 2013-05-10 04:56:58 +02:00