Commit Graph

6110 Commits

Author SHA1 Message Date
orbiter
30f108f97d added stub of oai-pmh importer (not working yet)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6437 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-30 15:58:04 +00:00
orbiter
77c99e500f added more control over memory allocation
should avoid some of the OOMs

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6436 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-27 15:25:48 +00:00
orbiter
52470d0de4 - fix for xls parser
- fix for image parser
- temporary integration of images as document types in the crawler and indexer for testing of the image parser

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6435 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-22 22:38:04 +00:00
orbiter
5e8038ac4d - refactoring of blacklists
- refactoring of event origin encoding


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6434 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-21 20:14:30 +00:00
orbiter
26fafd85a5 - more refactoring
- fixed problem with parsers

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6433 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-21 15:12:34 +00:00
orbiter
e48f3dfb1e added documentation for new yacy package structure
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6432 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-20 12:05:36 +00:00
orbiter
3528b970d6 - refactoring
- added new experimental (not-yet-working) image parser
- added new test image

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6431 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-19 22:34:44 +00:00
lotus
6414ac9ecf fix for debian int script
http://forum.yacy-websuche.de/viewtopic.php?t=2418

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6430 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-19 17:05:15 +00:00
lotus
63e489c5f7 removed win9x scripts because the latest jre has v1.3 for these systems
http://www.java.com/en/download/help/win95.xml

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6429 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-18 09:45:34 +00:00
orbiter
cde1611919 updated junit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6428 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-18 02:52:09 +00:00
orbiter
a8ce192f63 - shifted main classes to new package net.yacy
- fixed some bugs in last commit

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6427 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-18 01:38:07 +00:00
orbiter
b79f4f062f refactoring of yacy documents and parsers: they depend now only on the kelondro classes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6426 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-18 00:53:43 +00:00
hermens
0fd9540866 Configuration of HTTPDProxyHandler logging
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6425 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-17 14:04:18 +00:00
low012
519c3619ff *) minor changes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6424 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-17 00:32:07 +00:00
low012
f5656b2ae1 *) Made sure that only files with appropriate file endings are listed as skin or language files.
*) Introduced protection against directory traversal attacks in configuration servlets for skin and language configuration. Files can only be deleted if they are contained in a list of files which has been read by the servlet first.


Until now it was possible to delete any data on a system YaCy is running on and which can be deleted by the user who's account has been used to start YaCy. Most of the times a user of YaCy is also the owner of the machine the peer is running on, but this might not always be the case and not even the owner of the machine should be able to use YaCy as a replacement for "rm" or "del".

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6423 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-17 00:26:14 +00:00
low012
3434ca381f *) grrr
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6422 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-16 22:17:21 +00:00
low012
ae42c51cf7 *) Skin names and language names are displayed in alphabetical order in dropdown menu now.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6421 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-16 22:16:36 +00:00
suessthomas
56a5bd090d Small fixes to header.template for more XHTML compatibility.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6420 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-16 20:31:06 +00:00
f1ori
34c71b22e8 fix and enable parser unit tests (tested with eclipse)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6419 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-16 09:33:18 +00:00
orbiter
99683f5f11 small changes to green color and round corners
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6418 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-15 19:39:11 +00:00
orbiter
76bca8cffd show interactive search without menu
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6417 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-15 13:26:14 +00:00
orbiter
3d5eeb842a new default skin 'pdblue'
The old default skin named 'default' is renamed to 'classic-blue'.
All users will keep their current default skin named default, but YaCy will copy the classic-blue also to the skin folder.
For all new peers, the new skin pdblue is used.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6416 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-15 12:59:44 +00:00
orbiter
cee7a05ff2 - de-serialized the pdf parser
- added fail callback for file indexer

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6415 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-15 10:47:29 +00:00
orbiter
9db928ce53 replaced fontbox 0.7.3 with fontbox 0.8.0
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6414 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-15 09:51:16 +00:00
orbiter
c2272785c7 - fix for xlsx and pptx parsing
- less exception logging for swf parser

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6413 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-14 19:15:38 +00:00
suessthomas
afae2a0bee Small changes to the Yacy Skins.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6412 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-14 19:11:52 +00:00
lotus
0975b1b493 update for apache poi library
possible solves http://forum.yacy-websuche.de/viewtopic.php?p=17736#p17736

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6411 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-14 15:24:53 +00:00
orbiter
c864901087 - moved httpd.mime to defaults path
- some documentation fixes
- adopted a default setting for the search window: moves css setting to base.css
- some enhancements for the DocumentIndex class

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6410 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-14 13:29:09 +00:00
low012
8829ec5f18 *) made sure that   is replaced with a space and not just deleted in CharacterCoding.java
*) added annotations and made minor changes to serverObjects.java
*) set subversion properties for several files

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6409 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-13 20:57:56 +00:00
orbiter
6c347a37eb more options for DocumentIndex
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6408 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-13 08:43:02 +00:00
orbiter
6192205533 more final modifier
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6407 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-12 21:59:39 +00:00
orbiter
0f6b011e1a fix for new index location and better way to use own classes by reflection
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6406 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-12 21:12:42 +00:00
orbiter
7a3bbd950f :-(
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6405 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-12 20:29:03 +00:00
orbiter
b953f04f90 one more reflection fix
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6404 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-12 17:45:42 +00:00
orbiter
77d6604856 fix for npe, see http://forum.yacy-websuche.de/viewtopic.php?p=17727#p17727
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6403 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-12 17:41:16 +00:00
orbiter
2a7fe35f92 performance tuning using more final modifiers in the kelondro core
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6402 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-12 17:37:12 +00:00
orbiter
cb4de9ceee fixed a bug in table iterator (did not recognize elements in write buffer)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6401 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-12 08:06:35 +00:00
orbiter
5841ee83d3 refactoring
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6400 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-11 21:29:18 +00:00
orbiter
e7f18ba24b refactoring
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6399 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-11 00:24:42 +00:00
orbiter
ce8dc575ca refactoring
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6398 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-11 00:12:19 +00:00
orbiter
bea3b99aff moved table and util classes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6397 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-10 01:14:19 +00:00
orbiter
bd876eb4b7 moved io classes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6396 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-10 01:00:49 +00:00
orbiter
c0e0e1f422 moved blob classes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6395 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-10 00:43:25 +00:00
orbiter
1e4f8b56ed accumulated classes from different packages into the new rwi package
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6394 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-10 00:39:15 +00:00
orbiter
194da25a2f moved kelondro index
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6393 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-09 23:32:08 +00:00
orbiter
4446acc8cd moved kelondro order
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6392 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-09 23:22:22 +00:00
orbiter
f677d534b1 start of a really extensive refactoring which will produce a hierarchical package structure with the domain yacy.net as package root
- moved here the logging classes as part of the new net.yacy.kelondro package

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6391 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-09 23:13:30 +00:00
orbiter
ea473e32b8 refactoring
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6390 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-09 22:27:50 +00:00
orbiter
735e2737e3 * added index segments
This is a major change in the organization of indexes.
Please consider a back-up of your data before you run this update.
All existing index files will be moved and renamed to a new position.
With this change, it will be possible to maintain different indexes for different purposes and it will be possible to have a distinction between DHT-in and DHT-out specific indexes. Tenants may also have their own index, and it may be possible to have histories and back-ups of indexes. This is just the beginning, many servlets must be adopted after this change, but all functions that had been there should still work.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6389 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-09 14:44:20 +00:00
orbiter
09de5da74a once again a performance hack
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6388 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-10-08 18:26:54 +00:00