daburna
f4c13b422c
*updated translation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3421 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-03-01 09:36:59 +00:00
theli
9b33562ed1
*) adding mimetype application/x-rar
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3420 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-28 13:11:59 +00:00
orbiter
51e12049fa
third generation of R/W head path optimization
...
- data from collection arrays are read in order
- merged data is written in order
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3419 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-28 11:13:23 +00:00
karlchenofhell
1fe505f0b0
- adapted User_p to general web-interface style (and removed status-only page on changes)
...
- beautified WikiHelp.html + typos
- IP hasn't been set correctly in Blog.xml
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3418 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-28 09:22:31 +00:00
karlchenofhell
92b6bc0ad2
- fixed wrongly applied replacement of "<" and ">" in Blog and simplified the code a bit
...
- added check, whether active blacklist engine is supported by blacklist cleaner
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3417 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-28 00:04:32 +00:00
karlchenofhell
a1d68fe092
- use .class rather than Class.forName for classes in class-path
...
- added Bost's patch for Diff.findDiagonale() from: http://www.yacy-forum.de//files/patch_685.txt
- fixed minor bugs in Blog
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3416 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-27 22:52:22 +00:00
orbiter
10a3c20b8d
some more enhancements to R/W Head path optimization
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3415 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-27 15:54:02 +00:00
orbiter
f4cfd19835
second Generation of collection R/W head path optimization:
...
- permanent cache flush is switched off. The optimized cache flush
works better if it is a large number of collections that is flushed
together
- the flush size can be configured instead the flush divisor. There is
only one size for all flushes
- collection records that shall be removed during collection transition
(jump from one collection file to another) are now not really removed
but only marked in RAM. add-operations to the collection use these
marked collection spaces
- index bulk write operations are now separated for each file of a kelondroFlex
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3414 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-27 13:01:22 +00:00
hydrox
e92e8b2ae3
*) added RSS-Feed for blog
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3413 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-27 10:05:46 +00:00
hydrox
a107961099
*) fixed blog-comment-deletion without admin-rights is no longer possible
...
*) fixed no empty blog-comments anymore
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3412 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-27 08:37:47 +00:00
daburna
ea2dbcb034
*updated translation for
...
-blacklistcleaner
-blogcomments
-header.template
*small changes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3411 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-27 07:52:14 +00:00
(no author)
cf47075855
CSS corrects
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3410 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-26 23:03:10 +00:00
orbiter
1fda50fd3c
correct R/W head positioning in kelondroFlex
...
and some enhancements
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3409 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-26 22:25:39 +00:00
hydrox
116fc016d0
*) fix for Blogcomment-Preview
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3408 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-26 16:18:43 +00:00
orbiter
304412a049
first generation of collection index R/W head path optimization
...
- collections are now hand-over as collection lists to collection index for merge opertations
- collection index lists are separated into 'new' and 'extend' lists
- lists are written separately
- write operations are done into array sets and array indexes. These are now serialized
- write operations into index files are sorted by index;
that means that a R/W head does not need to go forward
and backward, only forward
More enhancements are possible
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3407 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-26 15:49:23 +00:00
hydrox
54fef3574f
*) missing files for last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3406 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-26 14:38:34 +00:00
hydrox
cb89c74d52
*) added blog-comments
...
*) removed debug-output when deleting news
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3405 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-26 14:36:01 +00:00
karlchenofhell
6fbe31425a
- some code-cleanup (no more syntax-warnings here)
...
- added deletion from loadedURLs of URLs to be blacklisted in IndexControl_p
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3404 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-26 12:56:50 +00:00
orbiter
32867580ee
update to kelondroRecords needed fo last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3403 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-26 11:55:36 +00:00
orbiter
e3480d4ad3
fix for warning in crawl balancer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3402 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-26 11:54:43 +00:00
daburna
ed021a3f70
bugfix, see http://www.yacy-forum.de/viewtopic.php?t=3573
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3401 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-26 10:05:28 +00:00
karlchenofhell
31ad42535a
- added buttons to add complete domain or single URL to blacklist to IndexControl_p
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3400 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-25 23:14:45 +00:00
orbiter
8668ac5d91
preparations for collection index cache flush optimization
...
(hand-over commit, no functional change to current code)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3399 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-25 21:06:26 +00:00
allo
42e9747650
fixed /path/forwarding. uncomment, if you want to use it.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3398 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-25 20:07:35 +00:00
karlchenofhell
e0decf4653
- added support for changing invalid entries in blacklist cleaner
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3397 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-25 19:36:05 +00:00
karlchenofhell
c58ef48e1c
- increased size of subject text-field
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3396 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-25 18:48:25 +00:00
karlchenofhell
1d31ebbeec
- added experimental PHP script which redirects from a vhost to a peer, using a public seed-file
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3395 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-25 15:18:22 +00:00
auron_x
9cbf94222f
*) added seedurl to network.xml as requested by lulabad
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3394 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-25 10:24:42 +00:00
karlchenofhell
39a2000d8b
- added support for [[Bookmark:$bookmarkTag|description]]-link-listings (requested by theli) to wiki-parser
...
- added support for <pre>-tags to wiki-parser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3393 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-24 21:26:48 +00:00
karlchenofhell
619653c054
- fix for last commit
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3392 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-24 15:40:56 +00:00
karlchenofhell
26f5757b40
- added support for multiple paths per domain to default-blacklist
...
warning: an interface-change had been neccessary:
- remove(String, String) has been renamed to removeAll(String, String), because it removes all path-entries for the specified host
- remove(String, String, String) has been added to delete only a path-entry
- geBlacklistType(String) has been renamed to getBlacklistType(String)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3391 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-24 13:56:32 +00:00
karlchenofhell
3d6ab19f7e
- remove double entries in blacklist as well
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3390 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-23 18:27:56 +00:00
karlchenofhell
bf7a69197d
- fix for possible NPE in queues_p
...
- WatchCrawler_p:
- display crawler traffic
- pause/resume local- and global crawler
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3389 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-22 22:26:11 +00:00
allo
9702d3abba
further supertemplate test
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3388 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-22 11:01:16 +00:00
karlchenofhell
a5a36d9252
- hopefully last fix fo 1.5 methods (sorry for that, eclipse isn't that helpful in identifying those methods)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3387 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-22 08:04:09 +00:00
karlchenofhell
e97b6f0458
- we still use Java 1.4 ...
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3386 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-21 22:43:31 +00:00
karlchenofhell
0c7b8cf632
- added first version of new wiki-parser
...
- added blacklist support to manual URLFetcher stack fill
- fix for NPE: http://www.yacy-forum.de/viewtopic.php?t=3559
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3385 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-21 22:31:36 +00:00
orbiter
f7803a6ce4
enhanced crawl balancer
...
- new domains now get a chance to get crawled early
- less IO operations
- new balancing method
- better dump order at shutdown time
- bugfixes regarding not found url hashes (no more superfluous cache kill)
- domain access time is now shared over all balancer stacks
- viewing the stack does no more disturbish the balancing algorithm that much
- intelligent selection of best next domain using domain access times
- extra double-check (to double-check the double-check)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3384 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-21 16:23:31 +00:00
low012
801eea8849
*) Fixed bug where pairReplace() got caught in infinite recursion. ( http://www.yacy-forum.de/viewtopic.php?t=3466 )
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3383 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-20 22:07:59 +00:00
theli
c8862e47fb
*) adding mimetype for svg
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3382 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-20 15:25:46 +00:00
orbiter
39b0658839
Redesign of Webinterface menu structure
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3381 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-20 14:35:29 +00:00
orbiter
c3e8c23f5d
fix for 'CANNOT FETCH ENTRY: hash is null' bug
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3380 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-20 13:53:21 +00:00
orbiter
badab8d924
fixed some more bugs in new db handling
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3379 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-20 12:29:12 +00:00
orbiter
e72d253577
fixed problem with initial cache load
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3378 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-20 11:20:48 +00:00
orbiter
2d8e472cfd
emergeny bugfix for last commit
...
(kelondroTree should work again)
the cache prefill is broken and will be fixed later
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3377 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-20 10:25:17 +00:00
orbiter
868aaabf88
documentation update
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3376 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-20 08:37:09 +00:00
orbiter
dc0c06e43d
PLEASE MAKE A BACK-UP OF YOUR COMPLETE DATA DIRECTORY BEFORE USING THIS
...
redesign for better IO performance
enhanced database seek-time by avoiding write operations at distant
positions of a database file. until now, a USEDC counter was written
at the head-section of a kelondroRecords database file (which is the
basic data structure of all kelondro database files) to store the
actual number of records that are contained in the database. Now, this
value is computed from the database file size. This is either done
only once at start-time, or continuously when run in asserts enabled.
The counter is then updated only in RAM, and written at close of the
file. If the close fails, the correct number can be computed from the
file size, and if this is not equal to the stored number it is a strong
evidence that YaCY was not shut down properly.
To preserve consistency, the complete storage-routine had to be re-written.
Another change enhances read of nodes in some cases, where the data-tail
can be read together with the data-head. This saves another IO lookup during
each DB node fetch.
Includes also many small bugfixes.
IF ANYTHING GOES WRONG, ALL YOUR DATA IS LOST: PLEASE MAKE A BACK-UP
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3375 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-20 08:35:51 +00:00
hydrox
5af76fccd7
*) peer-search on Network.html now is case-insensitive
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3374 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-19 13:00:41 +00:00
karlchenofhell
c016fcb10f
- added streaming-support to CrawlURLFetchStack_p servlet
...
- bug for NPE in list.java
- use more constants
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3373 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-19 12:47:46 +00:00
karlchenofhell
65af9d3215
- continue shifting even in the case the stacked URL could not be found
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3372 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-02-18 01:50:41 +00:00