yacy_search_server/source/de/anomic/http
karlchenofhell 88245e44d8 - improved version of robots.txt (delete your old htroot/robots.txt before updating):
- robots.txt is a servlet now
  - no need to rewrite the whole file each time a section is added or removed
  - user-defined disallows, added manually, won't be overwritten anymore
- new config-setting: httpd.robots.txt, holding names of the disallowed sections

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3423 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-02 01:19:38 +00:00
..
httpc.java - some code-cleanup (no more syntax-warnings here) 2007-02-26 12:56:50 +00:00
httpChunkedInputStream.java * applied many compiler warning recommendations 2006-08-16 19:49:31 +00:00
httpChunkedOutputStream.java *) soap handler 2006-10-28 12:31:48 +00:00
httpContentLengthInputStream.java *) Bugfix: calling close must not close the wrapped input stream, otherwise 2006-10-24 06:09:38 +00:00
httpd.java more protection against mis-use of YaCyHop interface: 2007-02-09 15:25:10 +00:00
httpdAbstractHandler.java minor changes 2005-08-03 02:02:39 +00:00
httpdBoundedSizeOutputStream.java *) better handling of maximum file size limit in crawler 2006-09-11 08:26:39 +00:00
httpdByteCountInputStream.java *) adding httpContentLengthInputStream.java class to allow reading of http response bodies 2006-09-18 10:00:28 +00:00
httpdByteCountOutputStream.java *) better handling of maximum file size limit in crawler 2006-09-11 08:26:39 +00:00
httpdFileHandler.java - improved version of robots.txt (delete your old htroot/robots.txt before updating): 2007-03-02 01:19:38 +00:00
httpdHandler.java replaced indexing ram-queue by file-based stack-queue 2005-07-06 14:48:41 +00:00
httpdLimitExceededException.java added option to see ranking scores in surftipps 2006-09-30 23:28:03 +00:00
httpdProxyHandler.java replaced some toString() 2007-02-06 16:26:56 +00:00
httpdRobotsTxtConfig.java - improved version of robots.txt (delete your old htroot/robots.txt before updating): 2007-03-02 01:19:38 +00:00
httpdSwitchboard.java code cleanup 2005-12-06 23:51:29 +00:00
httpHeader.java - some code-cleanup (no more syntax-warnings here) 2007-02-26 12:56:50 +00:00
httpRemoteProxyConfig.java replaced some toString() 2007-02-06 16:26:56 +00:00
httpTemplate.java - fix for http://www.yacy-forum.de/viewtopic.php?t=3369 2007-01-12 00:52:38 +00:00