Commit Graph

5289 Commits

Author SHA1 Message Date
orbiter
e04a0e05c3 fix for last commit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5614 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-16 16:21:12 +00:00
orbiter
a9ad863686 second part of 'doubles' fix - better handling of doubles in RAMIndex. More logging.
still missing: deletion of double entries in collections

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5613 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-16 16:13:48 +00:00
orbiter
59427064fb first part of 'doubles' fix (not fully ready yet)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5612 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-16 00:47:48 +00:00
orbiter
26978b2a25 - better memory protection in kelondro caches: computation of needed memory for cache grow
- removed excessive gc calls
- step to 16 vertical DHT partitions

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5611 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-15 23:35:59 +00:00
lotus
e9e2fff47a better scaling on performance graph
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5610 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-15 17:36:13 +00:00
lotus
4aad461100 added UPnP support
YaCy can now automatically forward ports on home routers
off by default

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5609 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-14 13:12:08 +00:00
orbiter
62762ba81a fix for thread dump
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5608 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-14 01:08:20 +00:00
orbiter
99b9788e54 fix for possible 100% CPU caused by concurrent access of HashMap
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5607 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-14 00:39:53 +00:00
orbiter
be0c492ae5 fix for memory leak bug in new dht transmissions
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5606 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-14 00:01:05 +00:00
hermens
2173865f92 Prevent race condition when switching timezones.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5605 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-13 11:59:50 +00:00
apfelmaennchen
40ab13c886 YaCy-Ui: small bug fixes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5604 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-13 11:00:34 +00:00
orbiter
40d9849aa4 - better control of chunk size in dht selection
- more restrict values in selection
- step to 4 vertical partitions

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5603 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-12 14:47:32 +00:00
daburna
5d3233ca6c some updates for german translation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5602 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-12 14:24:14 +00:00
apfelmaennchen
e4597bd780 YaCy-UI: small bugfix
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5601 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-11 17:28:51 +00:00
orbiter
30a1de41b3 disabled the BufferedIOChunks, because I consider it as broken.
I will try to fix that, but it is better to not use a buffer than using a broken buffer.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5600 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-11 15:21:48 +00:00
orbiter
411f2212f2 more memory leak fixing hacks
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5599 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-11 13:31:10 +00:00
apfelmaennchen
8aafbe234b YaCy-UI:
- some themes to get started

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5598 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-11 12:58:04 +00:00
apfelmaennchen
222c97b798 YaCy-UI:
- updated to jquery 1.3.1
- updated to jquery-ui 1.6rc6
- added support for themes (not yet persistent but otherwise working well)
- deleted some obsolete files

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5597 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-11 12:50:42 +00:00
orbiter
985d421f91 found and fixed some memory leaks
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5596 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-11 11:24:15 +00:00
orbiter
333489420b - fix for NPE when loading the cytag image
- some hacks for less memory usage:
-- less usage of buffer and cache memory in EcoFS
-- buffer allocation on-demand in BufferedIOChunks
-- removed largest ybr idx

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5595 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-11 10:52:56 +00:00
low012
4c11843eba *) fixed small bug with large effect (OOM exception hazard when "show all" was selected for a blacklist)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5594 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-10 23:46:51 +00:00
orbiter
6a32193916 - refactoring of cache naming in web index cache (no more dht semantics there)
- activating a feature in the thread dump that cuts off dumping of a trance of inside-java-core events

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5593 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-10 23:28:13 +00:00
orbiter
e52c3eb79d added option to create a custom number of thread dumps in multiple thread dumps
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5592 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-10 20:57:04 +00:00
orbiter
6c627dbdff update to the server core
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5591 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-10 13:26:26 +00:00
apfelmaennchen
bc1e7da83f small bugfix for Safari to work with YaCy-UI
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5590 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-10 12:35:09 +00:00
orbiter
5393f356aa fix for termination problem
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5589 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-10 01:08:06 +00:00
orbiter
6a876ecb88 first fixes to the DHT transmission process
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5588 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-10 00:48:54 +00:00
orbiter
de610b99ec replaced invisible pixel image with 'real' invisible pixel ;-)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5587 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-10 00:13:00 +00:00
orbiter
c25c334b75 replaced old DHT transmission method with new method. Many things have changed! some of them:
- after a index selection is made, the index is splitted into its vertical components
- from differrent index selctions the splitted components can be accumulated before they are placed into the transmission queue
- each splitted chunk gets its own transmission thread
- multiple transmission threads are started concurrently
- the process can be monitored with the blocking queue servlet
To implement that, a new package de.anomic.yacy.dht was created. Some old files have been removed.
The new index distribution model using a vertical DHT was implemented. An abstraction of this model
is implemented in the new dht package as interface. The freeworld network has now a configuration
of two vertial partitions; sixteen partitions are planned and will be configured if the process is bug-free.
This modification has three main targets:
- enhance the DHT transmission speed
- with a vertical DHT, a search will speed up. With two partitions, two times. With sixteen, sixteen times.
- the vertical DHT will apply a semi-dht for URLs, and peers will receive a fraction of the overall URLs they received before.
  with two partitions, the fractions will be halve. With sixteen partitions, a 1/16 of the previous number of URLs.
BE CAREFULL, THIS IS A MAJOR CODE CHANGE, POSSIBLY FULL OF BUGS AND HARMFUL THINGS.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5586 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-10 00:06:59 +00:00
orbiter
c6c57b7b3d svn 5580 placed a bug into yacyinteractive and it was not possible
to search any more. The cause was the replacement of '&' with '&amp'
in the call of the json source.
Please TEST _every_ code change.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5585 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-09 09:21:31 +00:00
orbiter
e9a4182e6a using a concurrent hash map for the template cache
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5584 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-08 21:48:37 +00:00
lotus
e8ae2599fd * some refactoring/moves to consoleInterface
* added possibility to find maximum possible heap size
you can get it via getWin32MaxHeap.bat
this may cause high system load
moreover the found limit is no guarantee for stable startups since it depends on system configuration

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5583 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-07 11:53:48 +00:00
auron_x
49d95fb3d9 - cytag-xhtml-fix
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5582 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-06 15:31:28 +00:00
orbiter
01b97ef3f8 added new cybertag-tracking feature that was inspired by itgrl
from the forum discussion in
http://forum.yacy-websuche.de/viewtopic.php?p=12612#p12612

The feature will provide two basic entities:
- you can integrate image links which point to your yacy installation anywhere in the web.
  the image can be loaded with
  <img src="http://<yourpeer>:<yourport>/cytag.png?icon=invisible&nick=<yournickname_or_community_id>&tag=<anything>">
  This will place a invisible 1-pixel image. If you change the icon=invisible to icon=redpill, you will see a red pill
  Use this, to track your activity in the web.
- you can view your tracks at
  http://localhost:8080/Tracks.html
- There is a public api to your tracks at
  http://localhost:8080/api/tracks_p.json
  which needs authentication


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5581 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-06 15:06:19 +00:00
auron_x
03a16f6c20 - more XHTML-validation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5580 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-06 14:45:56 +00:00
auron_x
46af1e06dc - XHTML-Validation of some pages (Status.html still needs work)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5579 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-06 08:51:03 +00:00
borg-0300
b19bc611b0 gc: better logging
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5578 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-05 19:42:32 +00:00
orbiter
b1f9c00118 fix for bug in merge operator initialization
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5577 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-05 15:26:16 +00:00
orbiter
b57c9da1f8 - fixes to doc, ppt, xls parser: better title
- fixes to httpd server response header generation
- fixes to a server date computation bug
- new Button in indexControl to view content of url in ViewFile


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5576 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-05 15:15:13 +00:00
f1ori
7936e58fe7 * sorry,previous version didn't compile
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5575 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-05 12:15:21 +00:00
f1ori
76cdc59789 * added some convertions to and from UTF-8
* this might fix problems on windows systems
  (like http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1824)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5574 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-05 12:12:07 +00:00
orbiter
65a1de6c05 longer timeout for remote crawl queries
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5573 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-04 16:55:09 +00:00
orbiter
072dd01ac8 more logging for RSS parser (to fix the remote crawl problem)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5572 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-04 16:50:10 +00:00
orbiter
86763c42c4 enhanced interactive search
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5571 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-04 15:59:22 +00:00
low012
5b79221037 *) bugfix for my last commit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5570 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-03 21:45:41 +00:00
orbiter
9d282d2c16 - renamed interactivesearch to yacyinteractive
- added a configuration option to set the pop up page in Config Appearance
- added a minimized header option to yacyinteractive
- fixed a bug in yacysearch: default values when no query is done


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5569 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-03 13:04:02 +00:00
low012
910c2aaed6 *) blacklists will only displayed partly now with 50 entries per page by default, this should make it easier to administrate large lists
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5568 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-02 18:24:04 +00:00
orbiter
f6da5f8f00 more mime types
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5567 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-02 16:07:31 +00:00
orbiter
75bef03ac6 fix for bad encoding in yacydoc.html and yacydoc.xml
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5566 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-02 15:55:45 +00:00
orbiter
180fe81ef7 quick hack to copy new log configuration over old one
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5565 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-02 14:49:46 +00:00