Michael Peter Christen
|
85a427ec54
|
support for multiple sitemaps in robots.txt
|
2014-03-14 13:33:23 +01:00 |
|
cominch
|
cef8ebc41d
|
getpageinfo: Checks if there is a OAI repository behind the URL.
This check is only performed if oai parameter is set when calling e.g. getpageinfo_p.xml?actions=oai
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8039 6c8d7289-2bf4-0310-a012-ef5d649a1542
|
2011-11-15 12:22:19 +00:00 |
|
orbiter
|
eb1c7c041d
|
write info about robots.txt evaluation into getpageinfo_p.xml
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8038 6c8d7289-2bf4-0310-a012-ef5d649a1542
|
2011-11-15 00:33:54 +00:00 |
|
orbiter
|
f8b8c82421
|
- refactoring of getpageinfo_p.xml (moved out of util)
- added more logging in getpageinfo_p.xml
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8037 6c8d7289-2bf4-0310-a012-ef5d649a1542
|
2011-11-15 00:22:40 +00:00 |
|