Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
2,423 Commits 1 Branch 20 Tags 836 MiB
74d1dea30b
Commit Graph

1605 Commits

Author SHA1 Message Date
orbiter
74d1dea30b changes towards better join-search 
- added generation of a compressed index within remote peers during global search
- added selection of specific urls within remote peers during secondary global search


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2539 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-10 22:36:47 +00:00
orbiter
ae4e8ce03e - cut for 'probably last html-interface version': version number update 
- small enhancement to ranking

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2536 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-10 19:44:11 +00:00
orbiter
64bed59ee8 enhancements to ranking 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2535 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-09 23:44:54 +00:00
theli
63893003be *) Adding settings page for the crawler which allows to specify a file size limit and the timeout to use. 
*) adding first version of maximum filesize check for the crawler

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2534 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-09 15:06:49 +00:00
auron_x
06b1365066 *) fixed existing protection against divbyzero and removed the new one 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2530 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-08 23:43:30 +00:00
orbiter
94d7ced900 fix for last ranking commit 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2529 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-08 21:14:57 +00:00
orbiter
cc97a3e9c6 fixed possibly bug with indexOutOfBoundsException 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2528 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-08 20:27:45 +00:00
orbiter
03835c2ee8 enhanced search result computation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2527 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-08 20:26:44 +00:00
orbiter
809960ddc6 avoid division by zero 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2526 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-08 20:00:19 +00:00
orbiter
ac3419b65f better debugging for indexOutOfBoundException bug 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2525 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-08 19:41:52 +00:00
orbiter
75b03a4580 fix for new ArrayIndexOutOfBoundException 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2524 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-08 19:15:55 +00:00
orbiter
a8bc768206 enhancements to ranking evaluation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2523 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-08 16:04:50 +00:00
auron_x
a82e926c5d *) fix for wrong totalPPM-calculation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2522 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-08 15:19:16 +00:00
theli
33898ae7e9 *) ResourceInfoFactory.java: Bugfix for classNotFoundException 
See: http://www.yacy-forum.de/viewtopic.php?t=2797

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2521 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-08 13:43:23 +00:00
theli
406e170e25 *) more verbose error message 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2519 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-08 11:48:49 +00:00
theli
b298474e22 *) Bugfix needed because of changed plasmaCrawlLURL.load behavior 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2518 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-08 11:33:27 +00:00
orbiter
c2e6cc8c6b small part of Bosts patch 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2517 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-08 01:40:23 +00:00
orbiter
96c6e4e322 - enhancements to detailed search page 
- enhancements to search ranking computation process
- removed bugs in postranking

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2516 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-08 01:26:06 +00:00
orbiter
9340dbb501 fixed all possible problems with nullpointer exception for LURLs 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2513 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-07 18:24:39 +00:00
theli
a5ed86105b *) bugfix for handling of ResourceInfo object in proxy 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2512 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-07 15:50:45 +00:00
hermens
ff4362b02d some more fixes for new plasmaCrawlLURL.load behavior 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2511 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-07 14:32:46 +00:00
hermens
7aeadbe7cc another NullPointerException in http.ResourceInfo 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2510 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-07 13:13:19 +00:00
orbiter
141f9e5bb4 fix for new plasmaCrawlLURL.load behavior 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2509 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-07 12:27:32 +00:00
orbiter
1e7fd48afd added size method to ftpc 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2508 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-07 12:21:41 +00:00
hermens
087f7511f8 prevent NullPointerException in http.ResourceInfo 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2507 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-07 11:46:22 +00:00
orbiter
a2525072f2 bugfix for kelondroRow - property generation 
this bug affected ranking parameters :-(

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2506 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-07 10:55:34 +00:00
hydrox
59a5511dbb *) added missing static Strings as requested by theli 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2505 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-07 07:20:28 +00:00
theli
6578564c9a *) Ignore more hop by hop http headers 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2504 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-07 05:38:35 +00:00
theli
b44514242a *) crawler/ftp/CrawlWorker.java: better errorhandling 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2503 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-07 05:22:35 +00:00
theli
7d7f30139c *) crawler/ftp/CrawlWorker.java: delete old cache file 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2502 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-07 05:08:35 +00:00
theli
4ae0f122f8 *) ResourceInfo.java: License header added 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2501 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-07 05:01:07 +00:00
theli
043edfa4d8 *) ftp/ResourceInfo.java ResourceInfo object for ftp resources added 
*) ftp/CrawlWorker.java better errorhandling for ftp crawler
*) plasmaCrawlEURL.java: some errorcodes added

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2499 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-07 04:12:52 +00:00
orbiter
4866868c0e added write cache for LURLs 
This was necessary to speed up the index receive process during global search


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2498 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-07 01:13:03 +00:00
orbiter
8a0e35618b enhancements to search result preparation 
- added detailed count on remote search results
- enhanced search sequence during remote searches (doing local search in sequence)
- strict adherence to timout limits

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2497 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-06 17:51:28 +00:00
theli
5c1bb53d2a Missing description for last commit 
*) next step of restructuring for new crawlers
   > HTCaching should now work protocol independent
   -- introduction of new ResourceInfo objects containing protocolspecific metadata
      of a resource. 
   -- the ResourceInfo objects now implement old functions like shallIndexCacheForXXX, 
      shallStoreCacheForXXX in a protocol dependent manner   
   > Indexing should also work protocol independent now

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2496 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-06 14:35:45 +00:00
theli
dae763d8e3 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2495 6c8d7289-2bf4-0310-a012-ef5d649a1542 2006-09-06 14:31:17 +00:00
theli
4825bfaaf3 *) Bugfix for PrintWriter Problem 
See: http://www.yacy-forum.de/viewtopic.php?t=2792

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2494 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-05 18:55:45 +00:00
orbiter
d4c5e2af01 html-dirlist can now also be generated from existing connections 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2493 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-05 10:11:07 +00:00
theli
7930839594 *) URL.java: userinfo was not taken over when generating a new url from a base url and a rel. path 
*) CrawlWorker.java: using new dirhtml function of ftpc

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2492 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-05 05:17:57 +00:00
orbiter
17ba468165 added html dirlisting generation in ftpc.java: 
ftpc.dirhtml() generates a StringBuffer with a complete web page

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2491 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-05 00:11:59 +00:00
theli
7a35b8e237 *) direct access to responseheaders of sbQueue.Entry removed to make it more http independent 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2487 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 15:36:19 +00:00
theli
ffbf416e76 *) direct access to requestheader of htCache.Entry removed to make it more http independent 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2486 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 15:29:45 +00:00
theli
3870d615e3 *) setting htCache.Entry fields to private 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2485 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 15:06:58 +00:00
theli
393a7d10be *) setting htCache.Entry fields to private 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2484 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 15:03:54 +00:00
theli
ab5a9bee66 *) adding some copyright headers 
*) next step of restructuring for new crawlers
   - adding first testversion of ftp crawler class
   -- does not create a htCache entry yet

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2483 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 14:38:29 +00:00
theli
5847492537 *) next step of restructuring for new crawlers 
- IndexCreate_p.java: correcting problems with ftp urls
   - URL.java does not cutout the userinfo anymore 
    (needed to transport authentication info in ftp urls, e.g. ftp://username:pwd@ftp.irgendwas.de)
   - plasmaCrawlLoader.java: 
   -- hack to re enable https urls
   -- adding function getSupportedProtocols

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2482 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 13:17:11 +00:00
orbiter
6cce47e217 test of ftp-urls in URL class 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2481 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 13:10:40 +00:00
theli
fce9e7741b *) next step of restructuring for new crawlers 
- renaming of http specific crawler settings

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2480 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 11:56:47 +00:00
theli
e3f0136606 *) next step of restructuring for new crawlers 
- adding function isSupportedProcotol to plasmaCrawlLoader.java
   - disabling robots.txt check for protocols other than http(s)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2479 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 11:46:17 +00:00
theli
9ded4e8d5a *) Bugfix for name resolution in proxy mode 
See: http://www.yacy-forum.de/viewtopic.php?p=25241

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2478 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 11:26:53 +00:00