orbiter
|
f204076d25
|
removed usage of temporary files: causes too much IO
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6813 6c8d7289-2bf4-0310-a012-ef5d649a1542
|
2010-04-14 22:17:18 +00:00 |
|
orbiter
|
54af9e6b49
|
- added parsing of robots meta-tag in html headers to detect a noindexing request
- added evaluation and indexing prevention in case that a noindexing is given in a html file
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6709 6c8d7289-2bf4-0310-a012-ef5d649a1542
|
2010-03-03 23:32:56 +00:00 |
|
orbiter
|
08f1cbb125
|
another update to the pdf parser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6463 6c8d7289-2bf4-0310-a012-ef5d649a1542
|
2009-11-06 22:41:37 +00:00 |
|
orbiter
|
605e896d6c
|
more details for exception catching when parsing pdfs
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6461 6c8d7289-2bf4-0310-a012-ef5d649a1542
|
2009-11-06 19:47:24 +00:00 |
|
orbiter
|
4431b9767e
|
added about 450 replacements for printStackTrace() methods to pipe such traces into the log at DATA/LOG/
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6458 6c8d7289-2bf4-0310-a012-ef5d649a1542
|
2009-11-05 20:28:37 +00:00 |
|
orbiter
|
b79f4f062f
|
refactoring of yacy documents and parsers: they depend now only on the kelondro classes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6426 6c8d7289-2bf4-0310-a012-ef5d649a1542
|
2009-10-18 00:53:43 +00:00 |
|