Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
3ca98fee42
yacy_search_server/source/de/anomic/plasma/parser
History
orbiter e81be7d4f2 added many missing user-agent declarations for yacy http client connections. 
the most important fix was the addition of the yacybot user-agent for robots.txt loading,
because web masters look for that access to see if the crawler behaves correctly.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4968 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-07-04 11:03:03 +00:00
..
bzip more generics 2008-01-29 10:12:48 +00:00
doc more generics 2008-01-29 10:12:48 +00:00
gzip more generics 2008-01-29 10:12:48 +00:00
mimeType fixed "java.lang.NoClassDefFoundError: org/a" 2008-05-10 08:42:31 +00:00
odt added many missing user-agent declarations for yacy http client connections. 2008-07-04 11:03:03 +00:00
pdf pdfParser: updated lib, fixed ClassNotFoundException: CMSError 2008-05-08 16:55:45 +00:00
ppt - added parsing of Dublin Core - compliant metadata (see RFC 5013 and ISO 15836) to html parser 2008-01-22 11:51:43 +00:00
ps - added parsing of Dublin Core - compliant metadata (see RFC 5013 and ISO 15836) to html parser 2008-01-22 11:51:43 +00:00
rpm added many missing user-agent declarations for yacy http client connections. 2008-07-04 11:03:03 +00:00
rss - organize imports 2008-06-06 16:01:27 +00:00
rtf - added parsing of Dublin Core - compliant metadata (see RFC 5013 and ISO 15836) to html parser 2008-01-22 11:51:43 +00:00
sevenzip - added parsing of Dublin Core - compliant metadata (see RFC 5013 and ISO 15836) to html parser 2008-01-22 11:51:43 +00:00
swf - added parsing of Dublin Core - compliant metadata (see RFC 5013 and ISO 15836) to html parser 2008-01-22 11:51:43 +00:00
tar - organize imports 2008-06-06 16:01:27 +00:00
vcf added many missing user-agent declarations for yacy http client connections. 2008-07-04 11:03:03 +00:00
xls - added parsing of Dublin Core - compliant metadata (see RFC 5013 and ISO 15836) to html parser 2008-01-22 11:51:43 +00:00
zip - organize imports 2008-06-06 16:01:27 +00:00
AbstractParser.java joined anomic.net.URL, plasmaURL and url hash computation: 2007-09-05 09:01:35 +00:00
Parser.java - added parsing of Dublin Core - compliant metadata (see RFC 5013 and ISO 15836) to html parser 2008-01-22 11:51:43 +00:00
ParserException.java refactoring: moved all crawler classes into their own package 2008-05-06 00:32:41 +00:00
ParserInfo.java more generics 2008-01-19 00:40:19 +00:00