Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
3,182 Commits 1 Branch 20 Tags 836 MiB
868aaabf88
Commit Graph

3182 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
orbiter
868aaabf88 documentation update 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3376 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-20 08:37:09 +00:00
orbiter
dc0c06e43d PLEASE MAKE A BACK-UP OF YOUR COMPLETE DATA DIRECTORY BEFORE USING THIS 
redesign for better IO performance
enhanced database seek-time by avoiding write operations at distant
positions of a database file. until now, a USEDC counter was written
at the head-section of a kelondroRecords database file (which is the
basic data structure of all kelondro database files) to store the
actual number of records that are contained in the database. Now, this
value is computed from the database file size. This is either done
only once at start-time, or continuously when run in asserts enabled.
The counter is then updated only in RAM, and written at close of the
file. If the close fails, the correct number can be computed from the
file size, and if this is not equal to the stored number it is a strong
evidence that YaCY was not shut down properly.
To preserve consistency, the complete storage-routine had to be re-written.
Another change enhances read of nodes in some cases, where the data-tail
can be read together with the data-head. This saves another IO lookup during
each DB node fetch.
Includes also many small bugfixes.
IF ANYTHING GOES WRONG, ALL YOUR DATA IS LOST: PLEASE MAKE A BACK-UP

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3375 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-20 08:35:51 +00:00
hydrox
5af76fccd7 *) peer-search on Network.html now is case-insensitive 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3374 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-19 13:00:41 +00:00
karlchenofhell
c016fcb10f - added streaming-support to CrawlURLFetchStack_p servlet 
- bug for NPE in list.java
- use more constants

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3373 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-19 12:47:46 +00:00
karlchenofhell
65af9d3215 - continue shifting even in the case the stacked URL could not be found 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3372 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-18 01:50:41 +00:00
rramthun
fdd1180ac5 Adding two icon files, both containing different sizes from 16x16 to 128x128 pixels in one file. 
The .icns is for Macintosh

Both made by Philipp Redeker

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3371 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-17 22:21:11 +00:00
karlchenofhell
d114a0136e - crawl profile: don't add null-values 
- added some settings and statistics for url-fetcher 'server'-mode
- added own stack for fetchable URLs
- added possibility to fill stack via shift from peer's queues, via POST (addurls=$count and url$num=$url) or via file-upload
- added "htroot" to classpath of linux start-script

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3370 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-17 19:16:53 +00:00
karlchenofhell
a46dc43f45 - added lock symbol for restart- and stutdown-buttons on Status-page (see http://www.yacy-forum.de/viewtopic.php?p=31444#31444) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3369 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-16 00:25:45 +00:00
karlchenofhell
b2a9d32f29 why do I always forget some lines? sorry... 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3368 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-14 15:11:03 +00:00
karlchenofhell
e6ddf135bb - enabled fetching new crawls via /yacy/list.html?list=queueUrls for testing purposes 
- sent URLs are taken off the limit-stack (of the global crawl trigger) (may be moved somewhere else in future versions)
- added option to set the requested chunk-size

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3367 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-14 14:50:55 +00:00
karlchenofhell
67d96249b4 - fix for last commit 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3366 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-13 21:17:43 +00:00
karlchenofhell
c5a2ba3a23 - prepared URL fetch from other peers 
- more feedback for user

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3365 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-13 20:18:12 +00:00
daburna
661a7bb702 *updated translation for 
-network
-wiki

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3364 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-12 20:53:47 +00:00
auron_x
5ba531a722 *) higher precision for QPH also on status-page 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3363 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-11 09:33:39 +00:00
karlchenofhell
4e5eda6ef9 huch... 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3362 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-10 20:25:45 +00:00
karlchenofhell
50b59e312f - added experimental CrawlURLFetch_p-Servlet to fetch new URLs from a specified location (\n-seperated list). Requested by Theli. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3361 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-10 20:20:00 +00:00
karlchenofhell
6c6375577e - fix for http://www.yacy-forum.de/viewtopic.php?t=3523 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3360 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-10 11:50:18 +00:00
karlchenofhell
ea20d8d7c5 - return to edited wiki-page after submit 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3359 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-09 19:47:21 +00:00
theli
e1edb23689 *) Bugfix for IllegalMonitorStateException 
See: http://www.yacy-forum.de/viewtopic.php?t=3522

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3358 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-09 19:32:49 +00:00
orbiter
bf69a721cb more protection against mis-use of YaCyHop interface: 
- target must not be at port 80
- target access not more than every 3 seconds
- requester may not access more than every 10 seconds

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3357 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-09 15:25:10 +00:00
orbiter
a15963ff98 better balancing: if element from top would force a busy waiting, 
an element from the bottom of the stack is used instead.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3356 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-09 10:32:58 +00:00
orbiter
dda24fcb85 ups 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3355 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-09 09:55:21 +00:00
orbiter
8c1d2e0227 protection against crawl balancer failure: 
a minimum of 500 milliseconds distance between two acesses
to the same domain is now ensured

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3354 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-09 09:48:23 +00:00
orbiter
1f1f398bfa enhanced speed of RAM cache flush by factor 20 (twenty times faster) 
- the speed was doubled by avoiding read access during the dump
- the speed was dramatically increased at least by factor 10
   by using a temporary ram-file where the structures are flushed to
   before it is dumped then as a whole byte-chunk to the file system.
The speed enhancements also affects some other parts of the database.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3353 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-08 23:21:46 +00:00
orbiter
30d79d69a6 fix for wrong display of search statistics 
see http://www.yacy-forum.de/viewtopic.php?p=31242#31242

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3352 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-08 10:42:35 +00:00
theli
ac376662cc *) changing alternate link to relative link 
*) fix for wrong date

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3351 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-08 06:19:45 +00:00
theli
b4981187c5 *) adding alternate link to rss 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3350 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-08 06:16:52 +00:00
theli
24e3dd4734 *) first version of yacy changelog RSS Feed 
See: http://www.yacy-forum.de/viewtopic.php?t=3462

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3349 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-08 06:13:30 +00:00
orbiter
daf2e15f59 some storage process enhancements (write without preceding read) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3348 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-07 23:23:24 +00:00
hydrox
faad869865 *) added peer-search to Network.html 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3347 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-07 11:20:31 +00:00
orbiter
9c2101a852 small enhancement to cache dump 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3346 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-07 00:02:54 +00:00
orbiter
c464157a6e replaced some toString() 
see http://www.yacy-forum.de/viewtopic.php?p=31151#31151

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3345 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-06 16:26:56 +00:00
orbiter
7673f0869b minor enhancements 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3344 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-06 16:01:03 +00:00
orbiter
b4aa195c27 added user-agent check for yacy-hop proxy authentication 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3343 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-06 09:53:02 +00:00
orbiter
2d7f7da7ce fix for null pointer exception 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3342 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-06 09:50:24 +00:00
theli
6256d89883 *) bugfix for reg.exp to determine svn rev. nr 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3341 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-06 07:47:56 +00:00
orbiter
d25caa07bf redesigned some parts of http authentication 
added another access check for peer hops

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3340 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-05 19:46:50 +00:00
low012
588e48ce0b *) Part II of last commit. Note to myself: check svn commandline syntax :-( 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3339 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-05 18:21:11 +00:00
low012
0d2431d6f7 *) removed printed out '<br />' in row Hit-Size Miss-Size by moving <br /> from Java file to HTML file. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3338 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-05 18:16:01 +00:00
hydrox
ff829e97f8 *) fixed headlines in blog (see: http://www.yacy-forum.de/viewtopic.php?t=3442 ) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3337 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-05 14:40:02 +00:00
hydrox
9184113284 *) fixed News deletion. News are now removed if they are no longer in a news-stack. This does not effect News-entires in the news-db that have no stack-entries. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3336 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-05 13:35:36 +00:00
karlchenofhell
a647a7ca8d - <tt>-tags look like <span class="tt">-tags now, fix for EDIT 3 of http://www.yacy-forum.de/viewtopic.php?t=3485 
- Typo: crawl depth 0 indexes the given url, 1 indexes all links on it

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3335 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-04 18:22:53 +00:00
(no author)
e218940293 The copyright sign "\u00A9" is already replaced by "&copy;". String "(C)" is not a unicode sequence! 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3334 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-04 18:16:27 +00:00
low012
1bc4d8d470 *) If there is more than one pair of patterns in a line, all of them (and not only one pair) will be replaced. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3333 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-04 15:53:40 +00:00
low012
ea7a8cf7aa *) <hr> and <br> tags are XHTML compliant now. 
*) Avoid superflous trailing blank in non-proportional sections.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3332 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-04 15:03:13 +00:00
orbiter
d03cd41266 fix for http://www.yacy-forum.de/viewtopic.php?t=3411 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3331 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-04 04:39:47 +00:00
karlchenofhell
3bafd643c0 - fix for http://www.yacy-forum.de/viewtopic.php?t=3483 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3330 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-03 20:22:39 +00:00
karlchenofhell
6667930352 - old versions may be reviewed and restored 
- removed explicit replacement of '<' and '>', fix for first bug in: http://www.yacy-forum.de/viewtopic.php?t=3485

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3329 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-03 19:48:18 +00:00
karlchenofhell
bf8f120340 - reduced margin of headlines in wiki (someone has to create a nice base.css urgently) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3328 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-03 16:04:43 +00:00
karlchenofhell
f2e6f19b90 - added versioning to Wiki 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3327 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-02-03 15:20:12 +00:00