yacy_search_server/source/net/yacy/crawler/retrieval
Michael Peter Christen a88a62f7aa added a feature to set a collection for a crawl result based on a
regular expression on th url: the collection attribut for a crawl start
may be now either a token or a list of tokens, seperated by ',' where a
token is either a string or a pair <string,pattern> where the string is
separated to the pattern with a ':' and the string is assigned to the
document as collection only if the pattern matches with the url.
2013-08-25 00:13:48 +02:00
..
FileLoader.java Redesign of crawler identification and robots steering. A non-p2p user 2013-08-22 14:23:47 +02:00
FTPLoader.java added collection_sxt to error documents 2013-07-17 15:20:56 +02:00
HTTPLoader.java Redesign of crawler identification and robots steering. A non-p2p user 2013-08-22 14:23:47 +02:00
ImporterException.java
Request.java Added 'final' for all exception blocks as this helps the Java compiler 2013-07-17 18:31:30 +02:00
Response.java extend content type detection on file extension for .tif .tiff .htm 2013-07-21 22:57:21 +02:00
RSSLoader.java added a feature to set a collection for a crawl result based on a 2013-08-25 00:13:48 +02:00
SitemapImporter.java Redesign of crawler identification and robots steering. A non-p2p user 2013-08-22 14:23:47 +02:00
SMBLoader.java Redesign of crawler identification and robots steering. A non-p2p user 2013-08-22 14:23:47 +02:00
URLRewriterLibrary.java Added 'final' for all exception blocks as this helps the Java compiler 2013-07-17 18:31:30 +02:00