Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
2,362 Commits 1 Branch 20 Tags 836 MiB
9ded4e8d5a
Commit Graph

1556 Commits

Author SHA1 Message Date
theli
9ded4e8d5a *) Bugfix for name resolution in proxy mode 
See: http://www.yacy-forum.de/viewtopic.php?p=25241

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2478 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 11:26:53 +00:00
theli
1c8300fcec *) Bugfix for name resolution in proxy mode 
See: http://www.yacy-forum.de/viewtopic.php?p=25241

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2477 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 11:23:57 +00:00
theli
4e2a950ac9 *) next step of restructuring for new crawlers 
- avoid using the http crawler class directly. Using the interface class instead

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2476 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 09:24:24 +00:00
theli
09b106eb04 *) next step of restructuring for new crawlers 
- adding interface class (plasma/crawler/plasmaCrawlWorker.java) for protocol specific crawl-worker threads 
   - moving reusable code into abstract crawl-worker class AbstractCrawlWorker.java
   - the load method of the worker threads should not be called directly anymore (e.g. by the snippet fetcher)
     to crawl a page and wait for the result use function plasmaCrawlLoader.loadSync([...])

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2474 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 09:00:18 +00:00
theli
eb9b138986 *) next step of restructuring for new crawlers 
- conversion of the crawler pool into a keyed object pool
   - crawlers are now loaded based on the url protocol (of course works only for http now)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2473 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 06:52:55 +00:00
theli
1395aae742 *) starting restructuring which is needed to add crawlers for additional protocols 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2472 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 06:09:20 +00:00
theli
b4acbdaa97 *) better handling of server shutdown 
See: e.g. http://www.yacy-forum.de/viewtopic.php?p=25234

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2470 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 05:17:37 +00:00
theli
f3ac4dbbb9 *) better handling of server shutdown 
See: e.g. http://www.yacy-forum.de/viewtopic.php?t=2584

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2468 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-03 14:59:00 +00:00
theli
959b779aba *) avoid performance loss if log level is greater than 'fine' 
See: http://www.yacy-forum.de/viewtopic.php?p=25180

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2467 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-03 08:42:46 +00:00
auron_x
57dda1a92c *)again fixing for wrong version display, now totally working with double instead of float 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2464 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-28 17:54:07 +00:00
auron_x
479b74e1dd *) fix for stupid mistake in new ppm-calc which caused decimal digits beeing written to seedinfo 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2463 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-28 04:43:28 +00:00
auron_x
348258a557 *) changed PPM-calculation to be much more accurate 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2461 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-27 17:18:34 +00:00
orbiter
18b6876860 new cache flush configuration settings 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2460 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-25 22:31:21 +00:00
hermens
f0278b4092 Bugfix for / by zero when the AssortmentCluster is empty 
See: http://www.yacy-forum.de/viewtopic.php?t=2746



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2459 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-25 20:23:04 +00:00
orbiter
14e0bb0dcf allow more references per word for new db 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2458 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-25 12:06:23 +00:00
orbiter
985dcbde7f changed some parameters that may cause better memory usage and more indexing speed 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2457 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-24 23:39:52 +00:00
orbiter
b7f4a1521b added options to switch on or off the kelondroFlexTable for NURL, EURL and PreNURL 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2456 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-24 22:21:22 +00:00
orbiter
c26da4893b turned back NURL usage of kelondroTree, kelondroFlexTable has still problems with deleted entries 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2454 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-24 10:03:38 +00:00
orbiter
db1eae0227 * simplified initialization of database objects 
* replaced kelondroTree for NURLs by kelondroFlex
* replaced kelondroTree for EURLs by kelondroFlex
take care, may be very buggy
please finish crawls before updating. crawls will be lost.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2452 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-24 02:19:25 +00:00
hermens
0b73f2b132 Repair DNS prefetch during cacheScan 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2451 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-24 01:31:08 +00:00
orbiter
27a159b401 * documentation update 
* removed doc from release
* release information in doc/News.html
* release 0.46

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2442 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-23 11:36:09 +00:00
theli
f80f776b89 *) Trying to solve NullpointerException problem in function addURLtoErrorDB 
See: http://www.yacy-forum.de/viewtopic.php?t=2705

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2441 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-23 10:23:20 +00:00
orbiter
d78b824e85 fixed problem with default path after first start-up 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2440 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-22 13:35:51 +00:00
hydrox
1c99b5a484 *)fixed logging for urldbcleanup 
*)changed exception handling in urldbcleanup so that it shows NullPointerException correctly
*)added more Blacklisting to urlcleaner

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2436 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-21 06:42:42 +00:00
orbiter
135e019883 removed one superfluous line from last commit 
(hasnot is included in remove)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2435 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-21 01:59:44 +00:00
orbiter
1591a55963 added object cache miss-cache use for remove method 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2434 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-21 01:51:27 +00:00
orbiter
8f3f4ab0eb enhanced synchronisation in plasmaWordIndex 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2433 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-21 01:29:26 +00:00
orbiter
f933f00f09 another patch to URL protocol handling for 'news', 'nntp' etc: 
reject it! (the java.net.URL class rejects them too)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2432 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-21 01:04:04 +00:00
orbiter
4c6e00d80a more bugfixes for URL class, see: 
http://www.yacy-forum.de/viewtopic.php?p=24844#24844

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2431 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-21 00:23:39 +00:00
orbiter
23dd972608 fixed memory calculation in performanceMemory web page 
fixed also maximum cache size computation

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2429 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-20 01:20:34 +00:00
orbiter
b7dc251948 fixed bugs in url class: 
- correct backpath ('..') handling
- correct absolute path handling
- included https


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2428 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-19 22:27:01 +00:00
orbiter
1ce3c22761 better memory control: 
- added memory monitor for preNURL-db in performanceMemory
- changed default memory assignments

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2427 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-19 13:09:04 +00:00
orbiter
39b4c26bdc more memory control: 
- catchup of OutOfMemoryError in server threads
- automatic adoption of word cache size after a Short Mem Cycle

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2426 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-19 00:06:39 +00:00
orbiter
3e9d509c39 some small fixes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2425 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-18 22:50:05 +00:00
orbiter
276225d79e fix for URL class 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2423 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-18 21:33:00 +00:00
orbiter
eb633c0a4f server threads must now supply a method that can be called in case 
of short memory. This has been realized for the indexing thread.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2421 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-18 02:07:03 +00:00
orbiter
f5720cb2fa removed most synchronization in wordIndex (for testing) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2420 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-18 01:35:33 +00:00
orbiter
0187c60010 because of a bug in the JRE 1.4.2 there was no memory protection 
see http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4686462
this commit fixes the bug by using a memory-computation patch.
All uses of Runtime.maxMemory had been replaced by serverMemory.max
The bug is not present any more in Java 1.5

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2419 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-18 01:33:54 +00:00
auron_x
4eca0f8830 *) fixed PPM calculation for multiple indexer-threads 
*) fixed totalPPM calculation and added total PPM to Network.html

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2418 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-17 19:15:30 +00:00
orbiter
cfb51fdef1 less synchronization in plasmaWordIndex 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2416 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-17 00:10:50 +00:00
orbiter
d6a928c2da quickfix for http://www.yacy-forum.de/viewtopic.php?t=2705 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2415 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-16 23:20:21 +00:00
orbiter
6ad471ef96 * applied many compiler warning recommendations 
* cleaned up code
* added unit test code
* migrated ranking RCI computation to kelondroFlex and kelondroCollectionIndex


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2414 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-16 19:49:31 +00:00
allo
cf1186597b utf fix from theli 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2412 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-16 15:26:04 +00:00
hydrox
9da3aa74d3 silly me, fix for the fix as advised by theli 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2408 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-13 17:26:32 +00:00
hydrox
bb3d9a5582 *) e.getMessage().indexOf() can only be used if there is actually an ExceptionMessage. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2407 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-13 17:18:09 +00:00
hydrox
7a54010a9c *) Iterators can't be casted to IndexContainer 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2406 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-13 17:08:39 +00:00
theli
5e0b6f8f83 *) sorting peer name list on Blacklist_p.html 
*) restructuring of sharedBlacklist_p.java

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2405 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-13 13:29:50 +00:00
orbiter
cd5f7e137c fixed problem with NURL-generation upon first startup 
(a new kelondroFlexTable was generated, which should not)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2402 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-12 23:24:50 +00:00
orbiter
8418af141a added several consistency checks and small changes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2400 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-12 15:59:14 +00:00
theli
9d13aeca13 *) removing class. does not work so far 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2399 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-12 14:43:07 +00:00