Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
3,417 Commits 1 Branch 20 Tags 836 MiB
81844e85b2
Commit Graph

2319 Commits

Author SHA1 Message Date
orbiter
c3e8c23f5d fix for 'CANNOT FETCH ENTRY: hash is null' bug 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3380 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-20 13:53:21 +00:00
orbiter
badab8d924 fixed some more bugs in new db handling 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3379 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-20 12:29:12 +00:00
orbiter
e72d253577 fixed problem with initial cache load 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3378 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-20 11:20:48 +00:00
orbiter
2d8e472cfd emergeny bugfix for last commit 
(kelondroTree should work again)
the cache prefill is broken and will be fixed later

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3377 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-20 10:25:17 +00:00
orbiter
dc0c06e43d PLEASE MAKE A BACK-UP OF YOUR COMPLETE DATA DIRECTORY BEFORE USING THIS 
redesign for better IO performance
enhanced database seek-time by avoiding write operations at distant
positions of a database file. until now, a USEDC counter was written
at the head-section of a kelondroRecords database file (which is the
basic data structure of all kelondro database files) to store the
actual number of records that are contained in the database. Now, this
value is computed from the database file size. This is either done
only once at start-time, or continuously when run in asserts enabled.
The counter is then updated only in RAM, and written at close of the
file. If the close fails, the correct number can be computed from the
file size, and if this is not equal to the stored number it is a strong
evidence that YaCY was not shut down properly.
To preserve consistency, the complete storage-routine had to be re-written.
Another change enhances read of nodes in some cases, where the data-tail
can be read together with the data-head. This saves another IO lookup during
each DB node fetch.
Includes also many small bugfixes.
IF ANYTHING GOES WRONG, ALL YOUR DATA IS LOST: PLEASE MAKE A BACK-UP

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3375 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-20 08:35:51 +00:00
karlchenofhell
c016fcb10f - added streaming-support to CrawlURLFetchStack_p servlet 
- bug for NPE in list.java
- use more constants

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3373 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-19 12:47:46 +00:00
karlchenofhell
d114a0136e - crawl profile: don't add null-values 
- added some settings and statistics for url-fetcher 'server'-mode
- added own stack for fetchable URLs
- added possibility to fill stack via shift from peer's queues, via POST (addurls=$count and url$num=$url) or via file-upload
- added "htroot" to classpath of linux start-script

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3370 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-17 19:16:53 +00:00
karlchenofhell
b2a9d32f29 why do I always forget some lines? sorry... 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3368 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-14 15:11:03 +00:00
theli
e1edb23689 *) Bugfix for IllegalMonitorStateException 
See: http://www.yacy-forum.de/viewtopic.php?t=3522

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3358 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-09 19:32:49 +00:00
orbiter
bf69a721cb more protection against mis-use of YaCyHop interface: 
- target must not be at port 80
- target access not more than every 3 seconds
- requester may not access more than every 10 seconds

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3357 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-09 15:25:10 +00:00
orbiter
a15963ff98 better balancing: if element from top would force a busy waiting, 
an element from the bottom of the stack is used instead.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3356 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-09 10:32:58 +00:00
orbiter
dda24fcb85 ups 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3355 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-09 09:55:21 +00:00
orbiter
8c1d2e0227 protection against crawl balancer failure: 
a minimum of 500 milliseconds distance between two acesses
to the same domain is now ensured

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3354 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-09 09:48:23 +00:00
orbiter
1f1f398bfa enhanced speed of RAM cache flush by factor 20 (twenty times faster) 
- the speed was doubled by avoiding read access during the dump
- the speed was dramatically increased at least by factor 10
   by using a temporary ram-file where the structures are flushed to
   before it is dumped then as a whole byte-chunk to the file system.
The speed enhancements also affects some other parts of the database.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3353 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-08 23:21:46 +00:00
orbiter
30d79d69a6 fix for wrong display of search statistics 
see http://www.yacy-forum.de/viewtopic.php?p=31242#31242

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3352 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-08 10:42:35 +00:00
orbiter
daf2e15f59 some storage process enhancements (write without preceding read) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3348 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-07 23:23:24 +00:00
orbiter
9c2101a852 small enhancement to cache dump 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3346 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-07 00:02:54 +00:00
orbiter
c464157a6e replaced some toString() 
see http://www.yacy-forum.de/viewtopic.php?p=31151#31151

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3345 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-06 16:26:56 +00:00
orbiter
7673f0869b minor enhancements 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3344 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-06 16:01:03 +00:00
orbiter
b4aa195c27 added user-agent check for yacy-hop proxy authentication 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3343 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-06 09:53:02 +00:00
orbiter
2d7f7da7ce fix for null pointer exception 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3342 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-06 09:50:24 +00:00
orbiter
d25caa07bf redesigned some parts of http authentication 
added another access check for peer hops

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3340 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-05 19:46:50 +00:00
hydrox
9184113284 *) fixed News deletion. News are now removed if they are no longer in a news-stack. This does not effect News-entires in the news-db that have no stack-entries. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3336 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-05 13:35:36 +00:00
(no author)
e218940293 The copyright sign "\u00A9" is already replaced by "©". String "(C)" is not a unicode sequence! 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3334 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-04 18:16:27 +00:00
low012
1bc4d8d470 *) If there is more than one pair of patterns in a line, all of them (and not only one pair) will be replaced. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3333 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-04 15:53:40 +00:00
low012
ea7a8cf7aa *) <hr> and <br> tags are XHTML compliant now. 
*) Avoid superflous trailing blank in non-proportional sections.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3332 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-04 15:03:13 +00:00
orbiter
d03cd41266 fix for http://www.yacy-forum.de/viewtopic.php?t=3411 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3331 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-04 04:39:47 +00:00
karlchenofhell
f2e6f19b90 - added versioning to Wiki 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3327 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-03 15:20:12 +00:00
orbiter
10ebcb5e1e fixed bug in disconnection time 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3326 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-03 11:22:23 +00:00
karlchenofhell
02a73dce87 - added Diff-class for wiki-versioning (forthcoming, first need suitable serverObjects.put() for it) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3325 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-03 05:24:44 +00:00
karlchenofhell
2401e748a3 - fixed wrong replacement of POST-parameters in httpd ('<' and '>' are still replaced, don't know why): http://www.yacy-forum.de/viewtopic.php?t=3466 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3324 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-03 01:26:05 +00:00
orbiter
b2f4087400 redesign of last-seen fieln inside seed: 
the field contains now a time in UDC-0 (instead relative to local UDC offset)
this fixes a bug in peer selection, where an iteration over all seeds
ordered by lastseen did not work correctly.
Problems may occur because the new meaning of this field may mix with
the different meaning of that field in older peers

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3322 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-02 23:54:27 +00:00
karlchenofhell
e68cdeeeb3 - reverted parseArg(String) to use a byte-array to handle correct UTF-8 parsing 
- arguments aren't passed html-escaped to the servlets anymore, bug-fix for http://www.yacy-forum.de/viewtopic.php?p=30573

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3321 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-02 21:20:53 +00:00
orbiter
e00e850a98 removed constants (no connection with yacySeed.dna identifier) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3320 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-02 14:52:54 +00:00
orbiter
fcc11391a8 some redesign attempts because sorting of lastseen does not work correctly 
not finished yet
target: better selection of peer-ping targets, which should enhance stabilization of the net

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3319 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-02 13:12:31 +00:00
orbiter
c2d6edf21d integrated number of remote targets as 'partitions' into remote search protocol 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3317 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-01 13:27:23 +00:00
orbiter
f696d3c1eb added double computation to kelondroMapObjects 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3316 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-01 09:48:31 +00:00
orbiter
819ff21c92 fixed QPM output 
QPM is temporarily called QPH (until more search requests are present?)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3313 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-01 00:17:35 +00:00
auron_x
89e7af037a *) used more switchboard-vars instead of config-vars 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3310 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-31 17:05:15 +00:00
orbiter
306c50ac40 QPM (queries per minute) statistic stub 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3308 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-31 15:39:11 +00:00
orbiter
7598e1243e removed unused variables/imports 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3306 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-31 09:28:47 +00:00
hydrox
27e4429645 *) fixed minimizeurldb-function 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3305 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-31 09:22:22 +00:00
orbiter
47ab83a7c0 added flag for YaCyHop - proxy access for all paths that start with /yacy/ 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3304 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-31 00:09:51 +00:00
karlchenofhell
9f74b128dd - added many more commented constants (please use constants rather than i.e. config-setting strings directly) 
- not all constants may be located correctly in the switchboard. Please relocate if you know the appropriate place for them

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3303 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-30 14:18:35 +00:00
orbiter
e4910f03d1 tag storage fix 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3302 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-30 11:52:15 +00:00
orbiter
991182b29b more space for bookmarks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3299 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-30 00:20:03 +00:00
orbiter
88fa764b64 implemented new kelondroObjects into bookmarkDB 
- Bookmark-Objects are stored inside the kelondroObjects cache
- removed superfluous classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3298 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-30 00:17:55 +00:00
orbiter
9c05e2a820 re-design ob kelondroMap 
- this class is replaced by an object that can hold any type of object
- this object must be defined as a class that implements kelondroObjectsEntry
- the kelodroMap is now implemented as kelondroMapObjects

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3297 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-29 23:51:10 +00:00
allo
669c21db05 first version of abstracted kelondroMap Cache. 
get returns a kelondroCachedObject(or in most cases a subclass of it),
or a map, which can be used to construct a kelondroCachedObject.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3295 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-29 19:10:55 +00:00
allo
98cb777e18 abstract wikiCode in putWiki 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3293 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-29 15:09:58 +00:00
orbiter
f25c0e98d1 - replaced String by StringBuffer in condenser 
- added CamelCase parser in condenser
- added option to switch on or off indexing for proxy

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3292 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-29 01:11:22 +00:00
allo
14f2068daf some more bookmark changes towards multiuser bookmarks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3291 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-28 17:38:43 +00:00
allo
25c7d4e25e fix for form (cookie) login 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3284 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-27 17:22:49 +00:00
allo
ff79c52fc0 bookmark users can now edit bookmarks. 
TO COME: tag bookmarks with username, list bookmarks of a special user, filter private bookmarks for users.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3274 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-23 10:24:26 +00:00
theli
49972b6193 *) SOAP: getPeerHash returns the proper value now 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3273 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-23 09:47:13 +00:00
karlchenofhell
15f0334cd3 - fixed IllegalThreadStateException in LogParser 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3265 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-21 14:45:52 +00:00
allo
f40169fcd7 preparing multiuser bookmarks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3256 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-19 19:42:50 +00:00
hydrox
814a09a0ed *) reversed r3250 and parts of r3252 (nanotime() is an java1.5 function) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3253 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-19 11:10:57 +00:00
karlchenofhell
d311e258f8 - adjusted LogStatistics to nano-seconds 
- removed patches of SVNs 3184/3185 preventing fast DHT

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3252 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-19 10:39:22 +00:00
hydrox
f7623f5d24 *) added missing measuring points for Parser-Runtime 
*) changed precision of Parser-Runtime from ms to ns

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3250 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-19 09:25:04 +00:00
orbiter
f3f99b19c6 extended search statistics 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3249 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-19 01:45:29 +00:00
orbiter
c0851ee943 refactoring: moved and renamed de.anomic.data.searchResults to plasma package 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3248 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-19 00:38:03 +00:00
allo
c39dda2374 finished refactoring of searchtemplates. 
now plasmaSwitchboard.searchFromLocal calculates a searchResults structure,
which is parsed in the yacysearch/detailedSearch Servlets.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3244 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-18 10:42:36 +00:00
allo
35039982da refactoring of search process: store results in a searchResults structure. At the moment, its just stored in it, and read from it again. 
Next step: return searchResults instead of serverObjects, and parse the results in the servlets.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3241 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-18 07:41:15 +00:00
orbiter
76fab83395 fixed bugs in seach statistics 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3240 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-18 00:26:16 +00:00
karlchenofhell
139810d504 - truncate attributes of new newsRecords if too long 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3239 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-17 23:45:53 +00:00
karlchenofhell
3c43e605ba - don't accept malformed bookmarks, fix for: http://www.yacy-forum.de/viewtopic.php?t=3414 (first report) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3238 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-17 23:39:03 +00:00
orbiter
d07b132a0d - fixed colors of network grafic 
- added option to activate write cache for seed-db
- did not activate write cache because it did not work

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3236 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-17 19:39:31 +00:00
karlchenofhell
5d540b219e - LogalizerHandler skips interfaces again 
- added LogParser stats to LogStatistics

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3234 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-17 17:01:20 +00:00
allo
e1fb3550ab fix for profile 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3231 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-17 14:52:51 +00:00
hydrox
6faf9b70b7 *) LogParserPLASMA now counts its total runtime. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3229 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-17 13:35:33 +00:00
hydrox
e5f854bc37 *) added LogalizerHandler-settings to yacy.logging. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3228 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-17 13:25:11 +00:00
karlchenofhell
7c40197e42 - fixed error pages and <label>s for index.html 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3226 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-17 04:20:19 +00:00
allo
29aa7031d3 workaround for the snippets 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3225 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-16 21:35:25 +00:00
allo
b4457763e5 fix for putSafeXML and supertemplates. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3223 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-16 21:06:31 +00:00
karlchenofhell
77b73aa7a8 - log-entries 'Indexed' are parsed correctly now 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3222 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-16 18:42:34 +00:00
karlchenofhell
aea199cb7b - IndexTransfer is working again 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3220 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-16 17:18:25 +00:00
karlchenofhell
71112b1fe6 - added LogStatistics_p.html servlet based on the logalizer (indexing values not functional yet due to charset/regex problems) 
add the following to DATA/LOG/yacy.logging:
---
# Properties for the LogalizerHandler
de.anomic.server.logging.LogalizerHandler.enabled = true
de.anomic.server.logging.LogalizerHandler.debug = false
de.anomic.server.logging.LogalizerHandler.parserPackage = de.anomic.server.logging.logParsers
---
and "de.anomic.server.logging.LogalizerHandler" to the list of global handlers

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3219 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-16 16:13:21 +00:00
allo
0c81bd39d4 XSS-safe put as default. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3217 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-16 14:07:54 +00:00
karlchenofhell
bdda9e802f - added some commented string constants to ease use of the result-table 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3215 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-16 05:51:39 +00:00
orbiter
5515571950 redesign of ymage classes 
- less memory usage
- better usage of awt classes
- drawing abstractions: preparations for movable objects for animation class
- test applet for animations
- known bugs: wrong colours for network picture

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3214 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-15 23:31:50 +00:00
orbiter
52c6461e6b some bugfix for statistics 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3211 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-15 16:03:00 +00:00
(no author)
fe72b772cf added a monitor page for search requests 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3206 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-15 01:50:57 +00:00
karlchenofhell
b3a2c79fec - fix for last fix 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3202 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-14 17:47:25 +00:00
karlchenofhell
4b23e79f51 - quick-fix for http://www.yacy-forum.de/viewtopic.php?t=3358 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3201 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-14 17:31:08 +00:00
karlchenofhell
4dce5ec261 - if mem is too low but former GCs helped, the word-cache limit is only decreased now, if a subsequent GC doesn't 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3197 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-13 06:30:17 +00:00
hydrox
c27e88104c *) getResults() should now work and compile properly with Java 1.4 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3191 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-12 11:12:09 +00:00
hydrox
a2fb54afff *) Quickfix for http://www.yacy-forum.de/viewtopic.php?p=29973#29973 getResults() used a java 1.5 function (Output is temporally disabled until a sulution with 1.4 functions is implemented) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3190 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-12 10:39:22 +00:00
karlchenofhell
b873ad51ab - fix for http://www.yacy-forum.de/viewtopic.php?t=3369 
- merged netBude's alternative for tables in yacysearch.html & search results valid
- added statistic info to index.html as proposed here: http://www.yacy-forum.de/viewtopic.php?p=29762#29762
- fixed error-log in httpTemplate

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3189 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-12 00:52:38 +00:00
auron_x
7fb30de8a4 *) made combined2prettyversion less restrictive and made it again easier to read 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3187 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-11 15:27:20 +00:00
hydrox
3acd90033c *) added functions to get results from log-parsers (not documented yet) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3186 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-11 09:18:10 +00:00
borg-0300
1aa74bbd2b update for last commit 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3185 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-09 23:57:04 +00:00
borg-0300
23e613b2ab CPU & IO reduce (Index Distribution) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3184 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-09 17:23:29 +00:00
(no author)
c67d22116e added exists-check based only on RAM index lookup: 
- faster double-check during crawling
- less IO

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3179 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-08 13:13:30 +00:00
(no author)
37e53b4a6a replaced tree database structure for seed db by flex data structure 
I don't know if this helps, we will find out...

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3177 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-07 23:34:13 +00:00
karlchenofhell
340dc52a9d - ConfigProfile_p.html now transmits usable encoding for other than 7-bit ASCII charset, see TODO in httpd.parseArg(String) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3174 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-07 02:07:27 +00:00
karlchenofhell
35fb671721 - updated DetailedSearch and ViewFile 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3173 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-06 12:21:13 +00:00
karlchenofhell
00aa9472d6 - added decode of HTML-entities in request lines 
- removed Bookmark symbol on search pages and surftips if not authenticated

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3172 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-06 11:05:50 +00:00
theli
d157201e08 *) IfesL for "Unexpected end of ZLIB" error message 
See: http://www.yacy-forum.de/viewtopic.php?t=3327

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3169 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-05 13:45:31 +00:00
karlchenofhell
0336480a3e - the maxMemory-fix for the Sun JVM 1.4.2 wrongly also applied to 1.6, thx to NN 
- added logging of reducing word-cache (log-level fine)
- disabled memprereq field in PerformanceQueues_p.html, because it is now set by the collections db
- minor changes to ConfigSkins / -Language

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3165 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-05 08:00:05 +00:00
borg-0300
735c1f87b1 restart re-activated (windows) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3164 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-04 22:03:32 +00:00
hydrox
2c01508ada *) fix for http://www.yacy-forum.de/viewtopic.php?p=29575#29575 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3162 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-04 11:36:38 +00:00
allo
6ff8359b98 possibility to use anonther bindPort than the externally reachable port. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3161 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-03 21:00:07 +00:00
borg-0300
d2be3c674d wrong cache values fixed 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3159 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-03 13:41:41 +00:00
karlchenofhell
df6281ba1f - removed JS from DetailedSearch => valid 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3151 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-02 21:43:21 +00:00
hydrox
fb1d8b91af *) changed Startpoints of IndexCleaner and IndexTransfer from ------------ to AAAAAAAAAAAA. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3150 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-01-01 12:27:05 +00:00
karlchenofhell
d6eb699e8e - fix for last commit (didn't know that the paragraph sign has an UTF-8-specific location) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3134 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-23 14:04:21 +00:00
karlchenofhell
41bc31d2c2 - ConfigAdvanced_p => XHTML (no invalid IDs) 
- removed unmappable characters from code

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3133 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-23 13:35:34 +00:00
orbiter
9b726ac366 release 0.50 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3132 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-23 04:26:05 +00:00
orbiter
036a0c828e fix for auto-configuration of crawler thread memory 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3131 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-23 03:04:58 +00:00
orbiter
16f88c49d9 applied patch from http://www.yacy-forum.de/viewtopic.php?p=27392#27392 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3130 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-23 02:34:57 +00:00
orbiter
2aa6f978c2 fix for http://www.yacy-forum.de/viewtopic.php?p=29125#29125 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3129 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-23 01:54:28 +00:00
orbiter
a4e90bc1dc fix + debug-code for http://www.yacy-forum.de/viewtopic.php?p=29126#29126 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3128 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-23 01:39:00 +00:00
borg-0300
6b5f28b746 answer for last commit: no 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3126 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-22 19:26:01 +00:00
borg-0300
d98ba7bc33 fix for memory limit computation ? 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3125 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-22 19:17:23 +00:00
orbiter
49ffedfd8b patch from Fuchs against deadlock, 
see http://www.yacy-forum.de/viewtopic.php?p=28804#28804


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3122 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-22 15:26:29 +00:00
borg-0300
a4f63d187d better map2string and NullPointer fix 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3119 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-22 13:13:41 +00:00
orbiter
c48374d14a new memory limit computation for indexing queue 
shall better prevent outofmemory errors

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3118 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-22 12:54:56 +00:00
theli
a9f1d3a065 *) SOAP: adding function to get the local peer hash 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3117 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-22 11:31:48 +00:00
borg-0300
9659c004c3 fix for wrong date (Lotus-1 Peer) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3111 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-21 13:14:13 +00:00
orbiter
08ac4c5ed0 bugfix for http://www.yacy-forum.de/viewtopic.php?p=29045#29045 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3110 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-21 10:43:18 +00:00
orbiter
8e3bd17554 adopted DetailedSearch page to new ranking options 
- fixed bug http://www.yacy-forum.de/viewtopic.php?t=3265
- more attributes on page
- attributes can be set as default for main search page
- option to re-set the attributes to built-in values

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3109 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-21 03:09:46 +00:00
orbiter
93a7e88245 more ranking parameter usage 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3108 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-20 23:51:55 +00:00
orbiter
2dbea612c9 fixed display bug for image search preview 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3107 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-20 23:31:57 +00:00
orbiter
0a050bc043 enhanced ranking 
- redesign of data storage in plasmaSearchRankingProfile
- profiles are extended by new ranking parameters
- new RWI ranking parameters are considered during ranking
- appearance attributes (i.e. emphasised text) is now considered
- faster ranking
- some attributes that had been checked during post-ranking can now be
  checked during pre-ranking phase
- removed old ranking parameter on index.html page (will be replaced by profiles in the future)
- ranking can now consider appearances of media content
- snippet-loading for media types now work correctly (fetches only from the wanted media)
- ranking-profiles can be handed over the remote peers and apply there also
- re-search of same query with different domain now also re-triggers remote search

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3105 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-20 15:44:29 +00:00
orbiter
d0c32c6aeb better protection against fraud peers 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3104 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-20 01:07:49 +00:00
orbiter
61798f0ae6 added option to distinguish between text crawl and media crawl 
- for each crawl start, there is now a flag for text and media
- the localCrawl flag is superfluous
- added new crawl profiles
- if an image search is done, only media links are crawled for the snippets


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3100 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-19 03:10:46 +00:00
karlchenofhell
be941c4475 - "javascript:"-URLs are recognized as well (as intended formerly I assume) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3097 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-18 20:19:02 +00:00
karlchenofhell
a619ba3f49 - fix for String index out of range during URL parsing 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3096 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-18 19:32:01 +00:00
orbiter
febe6b114a design update of crawler monitor 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3094 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-18 01:18:28 +00:00
orbiter
40049e0635 fixed media search snippet flow 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3092 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-17 22:52:58 +00:00
karlchenofhell
5763d20795 - added DATA/yacy.running file to indicate whether YaCy is running or not. Helpful for i.e. the updater and linux init-scripts 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3089 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-17 18:06:39 +00:00
borg-0300
05d0464377 only do 16 checks if "address" starts with "172."; 
better readably;


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3087 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-17 13:54:11 +00:00
rramthun
1a525710c1 *) cursor jumps now to searchbox on searchpages again 
*) added missing private IP-ranges for APIPA/Zeroconf and 172.16.0.0–172.31.255.255
*) Changed some seed-download-errors to warnings

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3086 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-17 13:21:17 +00:00
karlchenofhell
e17591acc3 - parse HTML arguments as UTF-8 strings 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3085 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-16 21:40:59 +00:00
karlchenofhell
d30932c7d8 - fix for fix... sry 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3084 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-16 16:43:52 +00:00
karlchenofhell
6118fb73ec - added decode of UTF-16 escapes in url-arguments (%u0123), bugfix for http://www.yacy-forum.de/viewtopic.php?t=2762 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3083 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-16 16:40:40 +00:00
allo
782db9099d version independent name for commons-pool lib 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3082 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-15 07:12:33 +00:00
orbiter
7ff86d6ba6 - image search now shows thumbnails (in bad order, but it works) 
- repaired DHT selection

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3081 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-14 02:48:37 +00:00
orbiter
ee3d91cb6b print-out of links that result from contraint-filtering 
in search result

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3078 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-13 01:39:34 +00:00
borg-0300
f4737eebd6 Properties 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3076 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-12 12:51:31 +00:00
borg-0300
1fc75c0f67 better logging 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3075 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-12 12:46:51 +00:00
orbiter
e4570bffaf -implemented a specialized snippet-fetch for media content 
-changed search result preparation for media search presentation

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3073 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-12 02:09:25 +00:00
low012
694a6e4f44 *) better text snipptes: any possible searchword (welt, linux, tag) in welt-linux-tag will be marked correctly now 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3072 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-11 15:19:35 +00:00
orbiter
bddc197453 reverted by-mistake removed change from low012/SVN 3068 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3070 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-11 11:07:36 +00:00
orbiter
1377c53aa3 extraction of media links from search results 
these links are mixed to the snippets for testing purpose
(a final version will handle this differently)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3069 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-11 01:31:23 +00:00
low012
586add4c6c *) Better snippets: words like GNU/Linux will not prevent Linux or GNU from being marked if they are searchword (see http://www.yacy-forum.de/viewtopic.php?t=2891) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3068 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-10 23:13:53 +00:00
borg-0300
29922d5dd2 changed writeMap 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3066 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-10 14:06:58 +00:00
borg-0300
5b00a04359 changed writeMap 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3065 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-10 13:56:32 +00:00
karlchenofhell
52abbd4131 - fix for wrong public IP if no hostname was set and IP was from range 192.168.*.* 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3063 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-09 22:00:34 +00:00
borg-0300
8b7c543885 NullPointer fix 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3061 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-09 15:51:34 +00:00
orbiter
937ccd4e76 fix for snippet-generation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3060 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-09 02:13:43 +00:00
orbiter
559f41a001 fix for http://www.yacy-forum.de/viewtopic.php?p=28607#28607 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3059 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-08 23:35:32 +00:00
auron_x
c086c71f17 *) fixed ArrayIndexOutOfBoundsException 
--> http://www.yacy-forum.de/viewtopic.php?t=3210

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3058 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-08 14:56:18 +00:00
orbiter
c93cfdc23a fix for http://www.yacy-forum.de/viewtopic.php?p=28564#28564 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3057 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-08 13:03:03 +00:00
orbiter
93a5ace330 fix for http://www.yacy-forum.de/viewtopic.php?p=28544#28544 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3056 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-08 12:57:17 +00:00
orbiter
bf0d820659 - added correct flagging of word properties 
- added self-healing to database in case that wrong free-pointers exist
- added presentation of media links in snippets (does not yet work correctly)
- code cleanup

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3055 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-08 02:14:56 +00:00
orbiter
10d888e70c - added a media search for images, audio, video and applications 
- new search options on search page
- new option in ViewInfo to display all links of a file
- enhanced collection data structure

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3054 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-07 02:40:57 +00:00
orbiter
a603c4d5e8 more code simplifications 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3052 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-06 13:13:55 +00:00
orbiter
9a85f5abc3 cleanup 
- removed 'deleteComplete' flag; this was used especially for WORDS indexes
- shifted methods from plasmaSwitchboard to plasmaWordIndex

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3051 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-06 12:51:46 +00:00
borg-0300
fbe1ee402b plasmaCrawlLURL$kiter cleanup 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3050 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-06 12:49:13 +00:00
orbiter
773ba1e91a - generalized object order handling 
- controlled object order for all database tables
- migrated DHT position computation to correct base64-decoded values
  this also closed the 'gaps' in the dht positions

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3049 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-06 03:02:57 +00:00
borg-0300
15381cbf73 other bugfix 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3048 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-05 23:45:51 +00:00
borg-0300
ad65cc9d2f NullPointer fixes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3047 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-05 18:43:05 +00:00
borg-0300
d33745a7ea NullPointer 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3046 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-05 17:48:01 +00:00
orbiter
3a4933b63c bugfix for 
http://www.yacy-forum.de/viewtopic.php?p=28493#28493

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3045 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-05 12:32:19 +00:00
orbiter
109ed0a0bb - cleaned up code; removed methods to write the old data structures 
- added an assortment importer. the old database structures can
  be imported with
  java -classpath classes yacy -migrateassortments
- modified wordmigration. The indexes from WORDS are now imported
  to the collection database. The call is
  java -classpath classes yacy -migratewords
  (as it was)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3044 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-05 02:47:51 +00:00
orbiter
052f28312a removed assortments from indexing data structures 
removed options to switch on assortments

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3041 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-02 19:34:59 +00:00
orbiter
2372b4fe0c release 0.49 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3040 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-02 01:57:49 +00:00
orbiter
f8efb3c948 fixed a null pointer exception problem reported in the forum. 
I cant find the forum entry any more because my girlfriend switched
off the power while the forum window was open.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3039 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-01 22:36:32 +00:00
orbiter
ad1e4aa88e added selection of audio, video, image and application resources 
to search procedure. This function can currently not used through the
search interface, but only through remote search.

added accumulation of search attributes to enable the audio, video,
image and application selection.

fixed a problem with external URL representation generation


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3036 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-01 16:21:17 +00:00
orbiter
fb7902aa68 fix for http://www.yacy-forum.de/viewtopic.php?p=26142#26142 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3033 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-01 11:01:56 +00:00
orbiter
7cc4cec9c9 bugfix for assertion bugs documented in 
http://www.yacy-forum.de/viewtopic.php?p=28261#28261

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3030 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-12-01 01:30:05 +00:00
orbiter
97596af843 patch for wrong old RWI entries 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3028 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-30 14:24:41 +00:00
hydrox
2c69cc969a *) more special chars removed 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3027 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-30 07:39:09 +00:00
hydrox
ebb42906f8 *) removed special characters 
*) added Copyright comments

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3026 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-30 07:33:54 +00:00
theli
954db729db *) Bugfix for ArrayIndexOutOfBoundsException during SSL detection 
See: http://www.yacy-forum.de/viewtopic.php?p=28247#28247

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3025 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-30 06:38:53 +00:00
orbiter
5afd2f283c fix for http://www.yacy-forum.de/viewtopic.php?p=27633#27633 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3024 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-30 01:17:28 +00:00
orbiter
9dfae90da8 better control of sortBound; should help to avoid an indexOutOfBound Exception 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3023 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-30 00:24:12 +00:00
orbiter
4ce590622f - more asserts 
- better memory usage during remove in kelondroRowSet

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3022 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-30 00:23:07 +00:00
orbiter
7dbcd358b4 fix for http://www.yacy-forum.de/viewtopic.php?p=28231#28231 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3021 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-29 23:09:56 +00:00
orbiter
86394e7a56 fix for cache-delete problem: 
- better synchronization
- files are only deleted if they have been in the cache for 5 minutes
- hash-path for the HTCACHE is now default

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3018 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-29 00:34:25 +00:00
orbiter
ceb9e3aa17 - enhanced parser: collection of audio, video, image and application links 
- enhanced condenser: better handling of utf-8 and pre-formatted texts


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3017 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-28 15:00:15 +00:00
orbiter
984285bdd6 better organisation of dns hit/miss cache flush 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3016 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-27 15:53:42 +00:00
orbiter
73c63578ad - activated the dns miss cache 
- added a cache-control for cache miss flush to the dns miss cache
- better naming of cache variables to distinguish hit- and miss- cache

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3015 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-27 15:27:43 +00:00
theli
eb20ec3837 *) soap-service: adding function to check if a specific url is blacklisted 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3014 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-26 08:53:43 +00:00
orbiter
0b9370a9dc fix for http://www.yacy-forum.de/viewtopic.php?p=28108#28108 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3013 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-25 23:33:28 +00:00
orbiter
b5a29e9651 - fix for snippets that are too short 
- added keyword to snippet fetch to suppres removal of not-found snippet words (for debugging)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3009 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-25 00:38:09 +00:00
orbiter
13868ebb3e reverted IP-patch in yacySeed 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3007 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-24 16:23:08 +00:00
allo
4cb688018d wikiAdmin Recht 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3006 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-24 11:32:00 +00:00
low012
29fa17bd40 *) simplified some code in wikiCode.java 
*) deleted outdated text in Settings_p-html (see http://www.yacy-forum.de/viewtopic.php?p=28027)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3005 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-24 02:52:38 +00:00
orbiter
f1528672b1 filtering of non-index pages during index-of search 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3004 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-24 02:46:02 +00:00
orbiter
0a0c3edeb6 fixed a bug in index transfer 
- the encoding within the new entry format for binary data was wrong
- the string parser of RWI receive had to be enhanced

added some mor debugging tools
- a target peer for index transfer can now be selected by typing in the peer name
- the RWI result list has an entry counter

enhanced routing
- if communication is between two peers that have the same IP address,
  the loopback address 127.0.0.1 is used instead the public IP
  to contact the peer

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3003 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-24 01:12:14 +00:00
orbiter
8e7215475b - extended ViewFile to use is as debugging-tool: you can now use the 
post-parameter url to submit an url directly
- fixed some bugs in text parser (not all parts had been analysed)
- fixed a bug in remote search interface (could not handle constraints)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3001 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-23 15:47:19 +00:00
orbiter
30888e7a2f implementation of search constraints 
Such constraints may formulate specific restrictions to web searches
This is implemented by scraping information for constraints from a web
page during parsing, and storing flags to the pages within the web index.

In this first step, only information for index pages ("index of", directory listings)
are scraped and stored in flags
- added new flag class kelondroBitfield
- added scraper method in condenser
- added bitfield structure for all scrape types (see also condenser)
- added bitfield structure for appearance locations (see RWIEntry)
- added handover protocol for remote search and index distribution
- extended kelondroColumn class to hold bitfield types
- added another search attribute on search page (index.html)
- extended search-filter to enable filtering of non-matching constraints
- set all new database types to be default
- refactoring: moved word hash generation to condenser class

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2999 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-23 02:16:30 +00:00
orbiter
49a83f99d9 - fix for wrong DHT ordering in DHT selection 
- fix for http://www.yacy-forum.de/viewtopic.php?t=3112&highlight=

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2995 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-21 00:36:41 +00:00
orbiter
e55ef0df28 - automatic migration of old RWI entries to new format during remote search 
if new collections are activated
- one more assert in RowSet, control of removeMarker

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2993 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-20 22:55:27 +00:00
orbiter
f4b547dc13 limited index transfer to peer with version 0.486 
this protects peers with version below 0.486 from new RWI objects
(which they cannot handle)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2988 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-20 02:46:53 +00:00
orbiter
10a4ab5195 disabled some (more) write caches 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2987 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-20 00:27:02 +00:00
orbiter
09bcc10344 bugfix for some problems of last change with assortments 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2986 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-11-19 23:10:58 +00:00