mirror of
https://github.com/yacy/yacy_search_server.git
synced 2024-09-19 00:01:41 +02:00
fcf6b16db4
New "Media Type detection" section in the advanced crawl start page allow to choose between : - not loading URLs with unknown or unsupported file extension without checking the actual Media Type (relying Content-Type header for now). This was the old default behavior, faster, but not really accurate. - always cross check URL file extension against the actual Media Type. This lets properly parse URLs ending with an apparently odd file extension, but which have actually a supported Media Type such as text/html. Sample URLs with misleading file extensions added as documentation in the crawl start page. fixes issue #244 |
||
|---|---|---|
| .. | ||
| contentcontrol | ||
| cora | ||
| crawler | ||
| data | ||
| document | ||
| gui | ||
| http | ||
| kelondro | ||
| peers | ||
| repository | ||
| search | ||
| server | ||
| utils | ||
| visualization | ||
| migration.java | ||
| yacy.java | ||