Commit Graph

3 Commits

Author SHA1 Message Date
Michael Peter Christen
f23c4142e0 added option to configure a custom user agent within allip networks 2014-06-01 01:02:03 +02:00
Michael Peter Christen
a2fba6584f use submitted default userAgent if cloning a crawl 2014-04-30 05:05:02 +02:00
orbiter
d29b6db270 made crawl start pages public since they do not reveal individual
information and they are also not used as servlet to actually start the
crawl (which is Crawler_p.html).
2014-03-31 20:42:39 +02:00