yacy_search_server/source/net/yacy/document/content
orbiter 54af9e6b49 - added parsing of robots meta-tag in html headers to detect a noindexing request
- added evaluation and indexing prevention in case that a noindexing is given in a html file

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6709 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-03-03 23:32:56 +00:00
..
dao more PMD recommendations 2010-01-11 22:18:38 +00:00
DCEntry.java - added parsing of robots meta-tag in html headers to detect a noindexing request 2010-03-03 23:32:56 +00:00
RSSMessage.java
SurrogateReader.java applied code changes that are recommended by PMD 2010-01-10 23:09:48 +00:00