Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
5,998 Commits 1 Branch 20 Tags 836 MiB
28d4b921b6
Commit Graph

3847 Commits

Author SHA1 Message Date
low012
93b2622503 *) repaired and added IM online status indicators 
*) added some missing SVN properties
*) removed unnecessary comment, added missing copyright notice

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6266 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-08-26 18:34:00 +00:00
orbiter
e7736d9c8d more refactoring: made all variables in SearchEvent private 
to prepare splitting of the class into two parts: local and remote search

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6265 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-08-26 15:59:55 +00:00
orbiter
4b92d0b9b7 patch for possible problems with normalization of '/' in urls. This applies in rare cases when '/' appear in post-properties 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6264 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-08-26 15:10:03 +00:00
orbiter
d8ca6e6bf1 more refactoring for search 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6263 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-08-25 21:27:01 +00:00
orbiter
fe4a4e3f6b added missing class 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6261 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-08-24 21:03:40 +00:00
orbiter
72ac5bd80f refactoring of search process. 
this is the beginning of some architecture changes that will hopefully bring some more stability, speed and transparency to the search process.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6260 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-08-24 15:24:02 +00:00
hermens
c4d0e22a77 Further speed upof concurrent DHT-receive 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6259 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-08-23 22:37:15 +00:00
hermens
2fbc0696bf Fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2334 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6258 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-08-21 13:10:59 +00:00
f1ori
d515bc11e2 added ooxmlparser 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6256 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-08-08 15:34:41 +00:00
orbiter
d9744b1b5d replaced old caching strategy control class with lightweight simplearc 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6254 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-08-07 23:01:33 +00:00
orbiter
8e56c2ace6 fix for fixes from this afternoon 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6253 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-08-07 22:53:49 +00:00
orbiter
cf739edc2e fix for possible deadlock, see 
http://forum.yacy-websuche.de/viewtopic.php?p=17017#p17017

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6252 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-08-07 12:11:22 +00:00
orbiter
6354b5e447 removed possible deadlock, see 
http://forum.yacy-websuche.de/viewtopic.php?p=17017#p17017

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6251 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-08-07 12:04:14 +00:00
orbiter
5cc17ccf8a a better caching with less overhead and more appropriate 
synchronisation use in more than 10 different data objects

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6250 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-08-07 11:55:32 +00:00
orbiter
92edd24e70 fixed problem with switching of networks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6247 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-30 15:49:23 +00:00
orbiter
0575f12838 fix for deadlock 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6246 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-30 09:08:44 +00:00
orbiter
fbfdaf063d - patch to omit IndexOutOfBoundsException when a b64-encoded key appears not to be well-formed. In that case the key is still accepted but rated higher than other regular keys to create a virtual ordering between well-formed and ill-formed keys 
- check routine at the beginning of the import of table keys that check that all imported keys are well-formed. All records that have a ill-formed key are deleted. This is a hack and is not tested since I don't have bad data here to test with. If the effect is seen in the wild, please report in the forum.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6245 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-29 19:43:11 +00:00
orbiter
c0e17de2fb - fixes for some problems with the new crawling/caching strategies 
- speed enhancements for the cache-only cache policy by using special no-delay rules in the balancer
- fixed some deadlock- and 100% CPU problems in the balancer

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6243 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-25 21:38:57 +00:00
orbiter
634a01a9a4 replaced wget-requests with caching requests 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6242 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-24 14:52:27 +00:00
orbiter
c6c97f23ad - added cache usage properties to crawl start 
- added special rule to balancer to omit forced delays if cache is used exclusively
- extended the htCache size by default to 32GB

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6241 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-24 11:54:04 +00:00
orbiter
c4ae2cd03f fixed bug that caused deletion of crawl profiles at every application startup 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6240 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-23 22:09:02 +00:00
orbiter
161d2fd2ef redesign of access to the HTCache (now http.client.Cache): 
- better control to the cache by using combined request-header and content access methods
- refactoring of many classes to comply to this new access method
- make shure that the cache is always written if something was loaded
- some redesign of the process how http response results are feeded into the new indexing queue
- introduction of a cache read policy:
 * never use the cache
 * use the cache if entry exist
 * use the cache if the proxy freshness rule confirmes
 * use only the cache and go never online
- added configuration options for the crawl profiles to use the new cache policies. There is not yet a input during crawl start to set the policy but this will be added in another step.
- set the default policies for the existing crawl profiles. If you want them to appear in your default profiles you must delete the crawl profiles database; othervise the policy is 'proxy freshness rule'
- enhanced some cache access methods in such a way that unnecessary retrievals are omitted (i.e. for size computation). That should reduce some IO but also a lot of CPU computation because sizes were computed after decompression of content after retrieval of the content from the disc.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6239 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-23 21:31:51 +00:00
f1ori
ba2e6de538 fix empty version string again 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6236 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-21 19:56:40 +00:00
orbiter
51534df0cb fix for possible synchronization problem 
see also: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2292&hilit=&p=16787#p16787

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6234 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-20 08:21:17 +00:00
orbiter
4da9042e8a code simplification 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6233 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-19 21:59:29 +00:00
orbiter
1d8d51075c refactoring: 
- removed the plasma package. The name of that package came from a very early pre-version of YaCy, even before YaCy was named AnomicHTTPProxy. The Proxy project introduced search for cache contents using class files that had been developed during the plasma project. Information from 2002 about plasma can be found here:
http://web.archive.org/web/20020802110827/http://anomic.de/AnomicPlasma/index.html
We stil have one class that comes mostly unchanged from the plasma project, the Condenser class. But this is now part of the document package and all other classes in the plasma package can be assigned to other packages.
- cleaned up the http package: better structure of that class and clean isolation of server and client classes. The old HTCache becomes part of the client sub-package of http.
- because the plasmaSwitchboard is now part of the search package all servlets had to be touched to declare a different package source.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6232 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-19 20:37:44 +00:00
f1ori
67da20647f * add new odf parser based on sax-xml-parser 
* remove odf_utils-jar
* test metadata in ParserTest


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6231 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-18 15:04:34 +00:00
f1ori
6d0e6d591b * ops, fix compiler error :( 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6227 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-17 20:02:56 +00:00
f1ori
3e5beb1654 * fix for empty version in seedlist 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6226 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-17 19:16:26 +00:00
orbiter
5bb8074150 removed the indexing queue. This queue was superfluous since the introduction of the blocking queues last year, where documents are parsed, analysed and stored in the index with concurrency. 
- The indexing queue was a historic data structure that was introduced at the very beginning at the project as a part of the switchboard organisation object structure. Without the indexing queue the switchboard queue becomes also superfluous. It has been removed as well.
- Removing the switchboard queue requires that all servlets are called without a opaque generic ('<?>'). That caused that all serlets had to be modified.
- Many servlets displayed the indexing queue or the size of that queue. In the past months the indexer was so fast that mostly the indexing queue appeared empty, so there was no use of it any more. Because the queue has been removed, the display in the servlets had also to be removed.
- The surrogate work task had been a part of the indexing queue control structure. Without the indexing queue the surrogates needed its own task management. That has been integrated here.
- Because the indexing queue had a special queue entry object and properties attached to this object, the propterties had to be moved to the queue entry object which is part of the new indexing queue withing the blocking queue, the Response Object. That object has now also the new properties of the removed indexing queue entry object.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6225 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-17 13:59:21 +00:00
orbiter
eea4c17ef2 removed rpm parser 
- no-one used that thing
- loading huge rpm files bay be causes for crashes


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6223 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-16 11:06:49 +00:00
orbiter
b332dfad67 - inserted request object into response object which carries this now instead generating new objects 
- fixed a problem with the crawler introduced in SVN 6216

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6222 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-15 23:08:35 +00:00
orbiter
ca72ed7526 -removed superfluous crawl cache 
-refactoring of crawler classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6221 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-15 21:07:46 +00:00
orbiter
13c63f4082 a set of small fixes to crawling behaviour 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6216 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-15 14:15:51 +00:00
orbiter
a564df3984 update to mime types in parsers and httpd.mime 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6215 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-15 14:10:29 +00:00
orbiter
43c8defd79 enhanced parser with more extension + mime attributes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6214 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-14 13:32:53 +00:00
orbiter
49bbb9bd45 replaced tar library with integrated apache ant tar lib 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6212 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-14 11:31:40 +00:00
orbiter
b2263bc720 enhanced document type recognition 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6209 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-14 11:01:05 +00:00
lotus
aa38eb5a20 * maxfilesize -1 for infinite filesize 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6208 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-14 08:39:39 +00:00
lotus
9cfe89c8fc * process content-length as soon as it is received 
* corrected indentation

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6206 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-13 19:55:13 +00:00
orbiter
50cf80056f removed jmimemagic library 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6203 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-13 10:58:37 +00:00
orbiter
3f113f38a8 removed unused imports 
removed unused libs from eclipse class path

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6201 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-13 10:19:10 +00:00
lotus
9f083bb6b2 check filetype before loading (no more mp4 loading) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6200 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-12 16:50:11 +00:00
f1ori
076ae02c44 * added pl and py to extensions excepted by htmlParser 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6198 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-12 16:35:35 +00:00
f1ori
d5e51cfd09 * workaround for non-working build property replacements 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6197 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-12 09:38:03 +00:00
f1ori
f814e0fa81 enable warnings and fix most of it 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6196 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-11 21:01:27 +00:00
f1ori
8931c8d6b4 improvments to debianpackage: 
* autoupdate completely disabled, display hint
* restart-button in interface works!

* moved all build-Variables to yacyBuildProperties
* fixed some warnings


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6195 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-11 17:03:22 +00:00
low012
fc1dc38b55 *) added spaces to make sure that no words are concatinated by accident 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6194 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-10 22:51:53 +00:00
low012
f242e7d7bc *) using Apache POI library to parse Word documents now 
*) removed tm-extractors library (can be found at http://www.textmining.org/ if necessary again)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6193 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-10 22:49:16 +00:00
orbiter
caedd72400 - enhanced logging and exception details for parsers 
- removed inconsistencies in mime type declaration (one mime type should only appear once in all parsers)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6192 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-10 16:15:33 +00:00
orbiter
4b74ad0a46 fixed setting of parser configuration servlets 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6191 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-10 15:02:34 +00:00
orbiter
57a88d435b redesign of parser mime type detection and parser steering 
There is now a mime-blacklist instead of a mime-whitelist

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6190 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-10 14:22:17 +00:00
lotus
e15d27bc63 avoiding double/wrong parser errors 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6189 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-10 09:31:10 +00:00
orbiter
21b8704fb4 refactoring of the ParserDispatcher and ParserConfig: resulted into Idiom, Parser and Classification classes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6188 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-09 22:25:31 +00:00
orbiter
8ca1f5d400 - some work to integrate the html parser the same way as the other parsers are integrated (not finished) 
- added migration of code of settings pages (hmm.. does not work correctly yet, sorry)
- more refactoring
- removed more unused code

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6187 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-09 20:56:30 +00:00
low012
1ee109761f *) added changes which were lost 
*) additional annotations
*) additional svn properties
*) _no_ functional changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6186 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-09 20:13:11 +00:00
orbiter
499723891d removed all non-http daemons; they had not been used and may be a potential security risk. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6185 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-08 22:24:34 +00:00
orbiter
0e8647d62f refactoring of search classes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6184 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-08 22:14:57 +00:00
orbiter
dafffd0153 refactoring of parsers and document processing 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6182 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-08 21:48:08 +00:00
low012
8041e91f56 *) Ooops! 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6181 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-08 20:58:47 +00:00
low012
69551ff3d9 *) added several MIME types (derived from http://filext.com/), some of them might be rather uncommon 
*) added an annotation forgotten in last commit

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6180 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-08 20:53:08 +00:00
low012
11dfb2d54f minor changes: 
*) added annotations
*) set svn properties and added keywords to comments of parser classes
*) made a variable final to prevent (theoretical case of) change of object instance in synchronized block

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6179 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-08 19:20:03 +00:00
orbiter
77d2a3782c removed strange debugging strings 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6177 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-06 15:21:43 +00:00
lotus
4320f69574 universal handling for crashed parsers 
reverting r6090/1

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6176 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-05 08:12:14 +00:00
orbiter
024744245c small refactoring to prepare for new queues 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6173 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-04 12:17:10 +00:00
orbiter
16efcd0366 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2252&hilit=&p=16389#p16389 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6172 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-04 06:24:58 +00:00
f1ori
0f3246e90a * fix debian package 
* add Class containing buildvariables


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6171 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-03 21:57:05 +00:00
f1ori
8544cfd5a6 * remove seperate build-files for parsers 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6170 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-03 20:06:06 +00:00
orbiter
24cb6d68bc - renamed Stack to RecordStack to avoid name confusion with new classes 
- added new Stack class that implements a stack on BLOB files
- added new Stacks class that can be used for a set of Stacks (a 'Stack Database')
- added methods to other classes to support the new stacks

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6169 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-03 16:35:34 +00:00
orbiter
995da28c73 all stack/heap files that had been stored in DATA/PLASMA are now stored in the network-specific QUEUES path 
There is no migration. All crawls must be restarted.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6167 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-02 17:01:23 +00:00
orbiter
aac89bf8ca trying to avoid "exceeding limit" message of server 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6166 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-02 15:47:42 +00:00
f1ori
48d78166ed * fix double copy of libraries 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6164 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-02 12:33:47 +00:00
lotus
7f868ca3c2 resource observer: support for yacyroot\DATA on an NTFS hardlink (Windows) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6162 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-07-02 10:02:20 +00:00
orbiter
409538e17a code cleanup and code simplifcation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6161 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-30 22:20:55 +00:00
orbiter
160031758d fix for problem with initializer 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6160 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-30 21:57:32 +00:00
orbiter
302a02cec8 moved all libraries from libx to lib 
removed libx directory
all libraries are now in lib, instead the test libraries in libt which are not part of releases

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6157 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-30 14:55:23 +00:00
orbiter
1f1399e5c5 extending visibility of objects and methods to avoid synthetic accessor methods and increase performance 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6156 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-30 13:25:46 +00:00
orbiter
154bbc3364 code cleanup: call of static methods directly to the class 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6155 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-30 13:01:35 +00:00
orbiter
222850414e simplification of the code: removed unused classes, methods and variables 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6154 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-30 09:27:46 +00:00
orbiter
93dfb51fd4 problems with code style 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6153 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-29 22:22:35 +00:00
orbiter
adf01c676e reduce lookup time when merging a large number of BLOBs 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6152 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-29 16:07:10 +00:00
orbiter
9a674d8047 - After the removal of the Tree class some code simplifications are possible. This affects mostly the Records class, which can be refactored and the result of the refactoring results in a reduced number of classes. 
- The EcoTable was renamed to Table.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6151 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-28 21:37:37 +00:00
orbiter
c5122d6836 completed migration of BLOBTree to BLOBHeaps: 
- removed migration code
- removed BLOBTree
after the removal of the BLOBTree, a lot of dead code appeared:
- removed dead code that was needed for BLOBTree
Some more classes may have not much use any more after the removal of BLOBTree, but still have some component that are needed elsewhere. Additional Refactoring steps are needed to clean up dependencies and then more code may appear that is unused and can be removed as well.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6150 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-28 21:02:56 +00:00
orbiter
d1083a6913 maybe we have less problems with open connections to the server if we don't do BF forced sleeps (just a test) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6149 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-28 10:35:12 +00:00
low012
ebe6c823ac *) changed svn properties agains (hopefully doing it right this time) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6147 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-27 09:56:01 +00:00
low012
a80ac3a415 *) fixed wrong parser descriptions 
*) changed svn properties

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6146 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-27 09:42:33 +00:00
low012
457b6c0d6d *) updated Apache POI library to be able to parse Visio files 
*) updated PPT and XLS parsers to use new Apache POI library
*) added new Visio (VSD) parser

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6145 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-27 09:33:09 +00:00
apfelmaennchen
a10c8022d1 DidYouMean: 
- limit the number of consumer threads to available CPUs
- added some javadoc

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6144 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-27 07:23:34 +00:00
f1ori
7eb3bff5b3 * workaround for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2220&hilit=#p16128 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6143 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-26 14:05:39 +00:00
orbiter
99fa265e1d fix for search bug caused by tenant patch 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6125 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-22 22:31:29 +00:00
orbiter
79875782af be a bit more lazy when removing domain navigation entries 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6120 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-22 12:30:22 +00:00
orbiter
57af311627 fix for wrong urls in navigator when a tenant is used 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6119 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-22 12:25:18 +00:00
lotus
76b96337e2 just some chatty code 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6118 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-22 11:45:21 +00:00
low012
91785d895c *) minor changes in comments 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6109 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-21 20:08:52 +00:00
orbiter
bdda140c02 fix for json output (no doubleqotes any more, doublequote quoting did not work) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6105 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-20 23:33:29 +00:00
orbiter
2f84736120 ignore signature files that cannot be downloaded because of failed encoding 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6103 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-20 20:46:58 +00:00
orbiter
041d9c253e some refactoring and more error-awareness in LogalizeHandler 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6102 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-20 20:29:57 +00:00
orbiter
6b307d6d59 more tolerance for corrupted index entries in exported row sets 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6099 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-19 21:35:44 +00:00
orbiter
33aafa9b4b better logging when writing merged dumps 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6098 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-19 17:02:50 +00:00
lotus
db70badcf0 possibility to set remote host on upnp device 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6097 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-19 15:32:59 +00:00