Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
e81be7d4f2
yacy_search_server/source/de/anomic
History
orbiter e81be7d4f2 added many missing user-agent declarations for yacy http client connections. 
the most important fix was the addition of the yacybot user-agent for robots.txt loading,
because web masters look for that access to see if the crawler behaves correctly.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4968 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-07-04 11:03:03 +00:00
..
crawler added many missing user-agent declarations for yacy http client connections. 2008-07-04 11:03:03 +00:00
data added many missing user-agent declarations for yacy http client connections. 2008-07-04 11:03:03 +00:00
htmlFilter added many missing user-agent declarations for yacy http client connections. 2008-07-04 11:03:03 +00:00
http added many missing user-agent declarations for yacy http client connections. 2008-07-04 11:03:03 +00:00
icap - organize imports 2008-06-06 16:01:27 +00:00
index - modified and enhanced the crawl balancer: better list export, fixing of damaged crawl queue at start-up, re-sorting at start-up to enhance domain order 2008-07-03 13:08:37 +00:00
kelondro added options to configure the 'corporate identity'-icons, the home page link and the greeting line from 2008-07-03 23:37:04 +00:00
language/identification - organize imports 2008-06-06 16:01:27 +00:00
net added many missing user-agent declarations for yacy http client connections. 2008-07-04 11:03:03 +00:00
plasma added many missing user-agent declarations for yacy http client connections. 2008-07-04 11:03:03 +00:00
server undo reduced menu 2008-06-30 07:11:13 +00:00
tools added many missing user-agent declarations for yacy http client connections. 2008-07-04 11:03:03 +00:00
urlRedirector - added option to switch between yacy networks, especially between the two default networks (freeworld and intranet), 2008-05-14 21:36:02 +00:00
xml - organize imports 2008-06-06 16:01:27 +00:00
yacy added many missing user-agent declarations for yacy http client connections. 2008-07-04 11:03:03 +00:00
ymage revert last fix; was wrong 2008-06-21 21:07:28 +00:00