Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
d77ed28e2f
yacy_search_server/source/de/anomic/plasma/parser/rpm
History
orbiter e81be7d4f2 added many missing user-agent declarations for yacy http client connections. 
the most important fix was the addition of the yacybot user-agent for robots.txt loading,
because web masters look for that access to see if the crawler behaves correctly.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4968 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-07-04 11:03:03 +00:00
..
build.xml
rpmParser.java added many missing user-agent declarations for yacy http client connections. 2008-07-04 11:03:03 +00:00