Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
9,014 Commits 1 Branch 20 Tags 836 MiB
2d9e577ad0
Commit Graph

4 Commits

Author SHA1 Message Date
Michael Peter Christen
2d9e577ad0 replaced the custom robots.txt loader by the standard http loader 2012-10-28 22:48:11 +01:00
Michael Peter Christen
a33e2742cb - removed unnecessary synchronized and deadlock in crawler 
- removed problem with monitoring object on Balancer.wait
- added missing user agent settings
 2012-10-28 19:56:02 +01:00
Michael Peter Christen
5f0ab25382 removed the option to prevent removal of & parts inside of the 
MultiProtocolURI during normalform computation because that should
always be done and also be done during initialization of the
MultiProtocolURI Object. The new normalform method takes only one
argument which should be 'true' unless you know exactly what you are
doing.
 2012-10-10 11:46:22 +02:00
Michael Peter Christen
00c1c777fa refactoring 2012-09-21 15:48:16 +02:00