Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
2,475 Commits 1 Branch 20 Tags 836 MiB
d0a5a53789
Commit Graph

2475 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
orbiter
17ba468165 added html dirlisting generation in ftpc.java: 
ftpc.dirhtml() generates a StringBuffer with a complete web page

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2491 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-05 00:11:59 +00:00
(no author)
2dacf63dd9 Spelling correction of the language list "Slovenky" -> "Slovensky" 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2490 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 19:55:21 +00:00
theli
413e6b9855 *) direct access to responseheaders of sbQueue.Entry removed to make it more http independent 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2489 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 15:56:49 +00:00
theli
2126c51906 *) bugfix for ViewFile.java. Wrong http header were used 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2488 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 15:49:51 +00:00
theli
7a35b8e237 *) direct access to responseheaders of sbQueue.Entry removed to make it more http independent 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2487 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 15:36:19 +00:00
theli
ffbf416e76 *) direct access to requestheader of htCache.Entry removed to make it more http independent 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2486 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 15:29:45 +00:00
theli
3870d615e3 *) setting htCache.Entry fields to private 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2485 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 15:06:58 +00:00
theli
393a7d10be *) setting htCache.Entry fields to private 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2484 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 15:03:54 +00:00
theli
ab5a9bee66 *) adding some copyright headers 
*) next step of restructuring for new crawlers
   - adding first testversion of ftp crawler class
   -- does not create a htCache entry yet

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2483 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 14:38:29 +00:00
theli
5847492537 *) next step of restructuring for new crawlers 
- IndexCreate_p.java: correcting problems with ftp urls
   - URL.java does not cutout the userinfo anymore 
    (needed to transport authentication info in ftp urls, e.g. ftp://username:pwd@ftp.irgendwas.de)
   - plasmaCrawlLoader.java: 
   -- hack to re enable https urls
   -- adding function getSupportedProtocols

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2482 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 13:17:11 +00:00
orbiter
6cce47e217 test of ftp-urls in URL class 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2481 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 13:10:40 +00:00
theli
fce9e7741b *) next step of restructuring for new crawlers 
- renaming of http specific crawler settings

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2480 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 11:56:47 +00:00
theli
e3f0136606 *) next step of restructuring for new crawlers 
- adding function isSupportedProcotol to plasmaCrawlLoader.java
   - disabling robots.txt check for protocols other than http(s)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2479 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 11:46:17 +00:00
theli
9ded4e8d5a *) Bugfix for name resolution in proxy mode 
See: http://www.yacy-forum.de/viewtopic.php?p=25241

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2478 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 11:26:53 +00:00
theli
1c8300fcec *) Bugfix for name resolution in proxy mode 
See: http://www.yacy-forum.de/viewtopic.php?p=25241

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2477 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 11:23:57 +00:00
theli
4e2a950ac9 *) next step of restructuring for new crawlers 
- avoid using the http crawler class directly. Using the interface class instead

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2476 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 09:24:24 +00:00
theli
f94131c13d *) Bugfix for Blacklist_p.java 
- avoid nullpointerexception

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2475 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 09:02:23 +00:00
theli
09b106eb04 *) next step of restructuring for new crawlers 
- adding interface class (plasma/crawler/plasmaCrawlWorker.java) for protocol specific crawl-worker threads 
   - moving reusable code into abstract crawl-worker class AbstractCrawlWorker.java
   - the load method of the worker threads should not be called directly anymore (e.g. by the snippet fetcher)
     to crawl a page and wait for the result use function plasmaCrawlLoader.loadSync([...])

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2474 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 09:00:18 +00:00
theli
eb9b138986 *) next step of restructuring for new crawlers 
- conversion of the crawler pool into a keyed object pool
   - crawlers are now loaded based on the url protocol (of course works only for http now)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2473 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 06:52:55 +00:00
theli
1395aae742 *) starting restructuring which is needed to add crawlers for additional protocols 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2472 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 06:09:20 +00:00
theli
857a2d76a2 *) better handling of server shutdown 
See: e.g. http://www.yacy-forum.de/viewtopic.php?p=25234

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2471 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 05:47:17 +00:00
theli
b4acbdaa97 *) better handling of server shutdown 
See: e.g. http://www.yacy-forum.de/viewtopic.php?p=25234

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2470 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-04 05:17:37 +00:00
orbiter
7df572756a fist step+attempt so solve the snippet marking problem. 
See: http://www.yacy-forum.de/viewtopic.php?p=22855#22855

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2469 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-03 23:22:23 +00:00
theli
f3ac4dbbb9 *) better handling of server shutdown 
See: e.g. http://www.yacy-forum.de/viewtopic.php?t=2584

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2468 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-03 14:59:00 +00:00
theli
959b779aba *) avoid performance loss if log level is greater than 'fine' 
See: http://www.yacy-forum.de/viewtopic.php?p=25180

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2467 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-03 08:42:46 +00:00
auron_x
b515d49f87 *) fix for new combinedVersionString2PrettyString by bost 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2466 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-09-02 07:29:12 +00:00
auron_x
24316ba937 *) improved implementation of combinedVersionString2PrettyString by bost 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2465 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-30 16:04:40 +00:00
auron_x
57dda1a92c *)again fixing for wrong version display, now totally working with double instead of float 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2464 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-28 17:54:07 +00:00
auron_x
479b74e1dd *) fix for stupid mistake in new ppm-calc which caused decimal digits beeing written to seedinfo 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2463 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-28 04:43:28 +00:00
auron_x
5e558fbaae *) hopefully fixed the wrong display of yacy-version 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2462 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-27 21:52:58 +00:00
auron_x
348258a557 *) changed PPM-calculation to be much more accurate 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2461 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-27 17:18:34 +00:00
orbiter
18b6876860 new cache flush configuration settings 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2460 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-25 22:31:21 +00:00
hermens
f0278b4092 Bugfix for / by zero when the AssortmentCluster is empty 
See: http://www.yacy-forum.de/viewtopic.php?t=2746



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2459 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-25 20:23:04 +00:00
orbiter
14e0bb0dcf allow more references per word for new db 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2458 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-25 12:06:23 +00:00
orbiter
985dcbde7f changed some parameters that may cause better memory usage and more indexing speed 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2457 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-24 23:39:52 +00:00
orbiter
b7f4a1521b added options to switch on or off the kelondroFlexTable for NURL, EURL and PreNURL 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2456 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-24 22:21:22 +00:00
orbiter
c26da4893b turned back NURL usage of kelondroTree, kelondroFlexTable has still problems with deleted entries 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2454 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-24 10:03:38 +00:00
orbiter
db1eae0227 * simplified initialization of database objects 
* replaced kelondroTree for NURLs by kelondroFlex
* replaced kelondroTree for EURLs by kelondroFlex
take care, may be very buggy
please finish crawls before updating. crawls will be lost.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2452 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-24 02:19:25 +00:00
hermens
0b73f2b132 Repair DNS prefetch during cacheScan 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2451 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-24 01:31:08 +00:00
rramthun
e34e07e0a1 - Changed back to dev namescheme and new 0.461 
- Corrected some errors in News.html

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2450 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-23 19:39:48 +00:00
allo
2d9478d203 installer for 0.46 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2444 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-23 13:09:48 +00:00
orbiter
27a159b401 * documentation update 
* removed doc from release
* release information in doc/News.html
* release 0.46

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2442 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-23 11:36:09 +00:00
theli
f80f776b89 *) Trying to solve NullpointerException problem in function addURLtoErrorDB 
See: http://www.yacy-forum.de/viewtopic.php?t=2705

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2441 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-23 10:23:20 +00:00
orbiter
d78b824e85 fixed problem with default path after first start-up 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2440 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-22 13:35:51 +00:00
orbiter
57d50df858 documentation update 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2439 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-22 01:46:17 +00:00
low012
c2264962d1 *) Fixed a bug. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2438 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-21 21:34:45 +00:00
low012
b387bd47db *) Added a Wiki Code help page and links to it at every textbox Wiki Code can be used in. Maybe this will encourage people to play around with the Wiki and the Blog more. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2437 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-21 21:23:29 +00:00
hydrox
1c99b5a484 *)fixed logging for urldbcleanup 
*)changed exception handling in urldbcleanup so that it shows NullPointerException correctly
*)added more Blacklisting to urlcleaner

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2436 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-21 06:42:42 +00:00
orbiter
135e019883 removed one superfluous line from last commit 
(hasnot is included in remove)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2435 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-21 01:59:44 +00:00
orbiter
1591a55963 added object cache miss-cache use for remove method 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2434 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-08-21 01:51:27 +00:00