Michael Peter Christen 2d9e577ad0 replaced the custom robots.txt loader by the standard http loader		2012-10-28 22:48:11 +01:00
..
Blacklist.java	Refactoring and redesign of data architecture to make URIMetadataRow	2012-10-18 14:29:11 +02:00
BlacklistFile.java	Rewrote all String blacklist types to enum 'BlacklistType', closes bug	2012-06-11 00:17:30 +02:00
FilterEngine.java	- replaced all length() == 0 and size() == 0 with isEmpty()	2012-07-10 22:59:03 +02:00
LoaderDispatcher.java	replaced the custom robots.txt loader by the standard http loader	2012-10-28 22:48:11 +01:00
RegexHelper.java	*) Invalid crawl profiles (containing invalid mustmatch/mustnotmatch filters) will be moved from active crawls to invalid crawls (new file: DATA/INDEX/freeworld/QUEUES/crawlProfilesInvalid.heap). This file can not be edited yet, but it shoudl be easy to extend the CrawlProfileEditor accordingly.	2011-07-03 23:55:55 +00:00