Commit Graph

5301 Commits

Author SHA1 Message Date
orbiter
9559bc23fd automatic clean-up of dead connections
(hope that works well..)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5626 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-20 22:27:02 +00:00
hermens
02dfd6183b Fix logging in serverCore
Prevent NPEs from keeping stopped Sessions in the pool and blocking slots



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5625 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-20 18:54:01 +00:00
hermens
d30456e2c8 Fix logging in serverCore
Prevent NPE:
I 2009/02/20 15:15:56 PLASMA check for Session_77.37.19.225:38812#0: 86515 ms alive, stopping thread
I 2009/02/20 15:15:56 PLASMA Closing main socket of thread 'Session_77.37.19.225:38812#0'
E 2009/02/20 15:15:56 SERVER receive interrupted - exception 2 = Socket closed
Exception in thread "Session_77.37.19.225:38812#0" java.lang.NullPointerException
        at de.anomic.server.serverCore$Session.run(serverCore.java:623)



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5624 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-20 15:12:00 +00:00
orbiter
4f9dae2571 remove reference in crawl entries
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5623 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-19 22:58:00 +00:00
orbiter
1ba4301920 automated interruption of dead incoming connections, if they are there for more than one minute
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5622 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-19 22:27:24 +00:00
orbiter
c12bb8a6d0 - refactoring of the http client
- added a protection against memory leaks for the access tracker

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5621 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-19 16:24:46 +00:00
orbiter
5d3983faae the soLinger parameter was wrong.
With soLinger=true the httpd looses connections
The effect can be seen when crawling the internal repository:
lost connections filled the client process queue until it was full
and no more connections were possible.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5620 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-17 16:22:15 +00:00
orbiter
62505bb3cb more bugfixes as recommendet by findbugs
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5619 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-17 09:12:47 +00:00
orbiter
6b450d09ca some fixes recommended by findbugs
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5618 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-16 23:31:54 +00:00
orbiter
4db80065ac select more
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5617 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-16 21:53:37 +00:00
orbiter
94c42691d8 - reject less transmissions as transmission receiver
- do not flag too much receiver when something goes wrong during transmission as sender

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5616 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-16 21:28:48 +00:00
orbiter
f887fc159f try to reduce the large number of unclosed incoming connections
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5615 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-16 16:26:57 +00:00
orbiter
e04a0e05c3 fix for last commit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5614 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-16 16:21:12 +00:00
orbiter
a9ad863686 second part of 'doubles' fix - better handling of doubles in RAMIndex. More logging.
still missing: deletion of double entries in collections

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5613 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-16 16:13:48 +00:00
orbiter
59427064fb first part of 'doubles' fix (not fully ready yet)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5612 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-16 00:47:48 +00:00
orbiter
26978b2a25 - better memory protection in kelondro caches: computation of needed memory for cache grow
- removed excessive gc calls
- step to 16 vertical DHT partitions

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5611 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-15 23:35:59 +00:00
lotus
e9e2fff47a better scaling on performance graph
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5610 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-15 17:36:13 +00:00
lotus
4aad461100 added UPnP support
YaCy can now automatically forward ports on home routers
off by default

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5609 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-14 13:12:08 +00:00
orbiter
62762ba81a fix for thread dump
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5608 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-14 01:08:20 +00:00
orbiter
99b9788e54 fix for possible 100% CPU caused by concurrent access of HashMap
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5607 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-14 00:39:53 +00:00
orbiter
be0c492ae5 fix for memory leak bug in new dht transmissions
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5606 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-14 00:01:05 +00:00
hermens
2173865f92 Prevent race condition when switching timezones.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5605 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-13 11:59:50 +00:00
apfelmaennchen
40ab13c886 YaCy-Ui: small bug fixes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5604 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-13 11:00:34 +00:00
orbiter
40d9849aa4 - better control of chunk size in dht selection
- more restrict values in selection
- step to 4 vertical partitions

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5603 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-12 14:47:32 +00:00
daburna
5d3233ca6c some updates for german translation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5602 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-12 14:24:14 +00:00
apfelmaennchen
e4597bd780 YaCy-UI: small bugfix
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5601 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-11 17:28:51 +00:00
orbiter
30a1de41b3 disabled the BufferedIOChunks, because I consider it as broken.
I will try to fix that, but it is better to not use a buffer than using a broken buffer.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5600 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-11 15:21:48 +00:00
orbiter
411f2212f2 more memory leak fixing hacks
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5599 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-11 13:31:10 +00:00
apfelmaennchen
8aafbe234b YaCy-UI:
- some themes to get started

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5598 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-11 12:58:04 +00:00
apfelmaennchen
222c97b798 YaCy-UI:
- updated to jquery 1.3.1
- updated to jquery-ui 1.6rc6
- added support for themes (not yet persistent but otherwise working well)
- deleted some obsolete files

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5597 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-11 12:50:42 +00:00
orbiter
985d421f91 found and fixed some memory leaks
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5596 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-11 11:24:15 +00:00
orbiter
333489420b - fix for NPE when loading the cytag image
- some hacks for less memory usage:
-- less usage of buffer and cache memory in EcoFS
-- buffer allocation on-demand in BufferedIOChunks
-- removed largest ybr idx

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5595 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-11 10:52:56 +00:00
low012
4c11843eba *) fixed small bug with large effect (OOM exception hazard when "show all" was selected for a blacklist)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5594 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-10 23:46:51 +00:00
orbiter
6a32193916 - refactoring of cache naming in web index cache (no more dht semantics there)
- activating a feature in the thread dump that cuts off dumping of a trance of inside-java-core events

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5593 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-10 23:28:13 +00:00
orbiter
e52c3eb79d added option to create a custom number of thread dumps in multiple thread dumps
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5592 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-10 20:57:04 +00:00
orbiter
6c627dbdff update to the server core
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5591 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-10 13:26:26 +00:00
apfelmaennchen
bc1e7da83f small bugfix for Safari to work with YaCy-UI
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5590 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-10 12:35:09 +00:00
orbiter
5393f356aa fix for termination problem
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5589 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-10 01:08:06 +00:00
orbiter
6a876ecb88 first fixes to the DHT transmission process
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5588 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-10 00:48:54 +00:00
orbiter
de610b99ec replaced invisible pixel image with 'real' invisible pixel ;-)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5587 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-10 00:13:00 +00:00
orbiter
c25c334b75 replaced old DHT transmission method with new method. Many things have changed! some of them:
- after a index selection is made, the index is splitted into its vertical components
- from differrent index selctions the splitted components can be accumulated before they are placed into the transmission queue
- each splitted chunk gets its own transmission thread
- multiple transmission threads are started concurrently
- the process can be monitored with the blocking queue servlet
To implement that, a new package de.anomic.yacy.dht was created. Some old files have been removed.
The new index distribution model using a vertical DHT was implemented. An abstraction of this model
is implemented in the new dht package as interface. The freeworld network has now a configuration
of two vertial partitions; sixteen partitions are planned and will be configured if the process is bug-free.
This modification has three main targets:
- enhance the DHT transmission speed
- with a vertical DHT, a search will speed up. With two partitions, two times. With sixteen, sixteen times.
- the vertical DHT will apply a semi-dht for URLs, and peers will receive a fraction of the overall URLs they received before.
  with two partitions, the fractions will be halve. With sixteen partitions, a 1/16 of the previous number of URLs.
BE CAREFULL, THIS IS A MAJOR CODE CHANGE, POSSIBLY FULL OF BUGS AND HARMFUL THINGS.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5586 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-10 00:06:59 +00:00
orbiter
c6c57b7b3d svn 5580 placed a bug into yacyinteractive and it was not possible
to search any more. The cause was the replacement of '&' with '&amp'
in the call of the json source.
Please TEST _every_ code change.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5585 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-09 09:21:31 +00:00
orbiter
e9a4182e6a using a concurrent hash map for the template cache
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5584 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-08 21:48:37 +00:00
lotus
e8ae2599fd * some refactoring/moves to consoleInterface
* added possibility to find maximum possible heap size
you can get it via getWin32MaxHeap.bat
this may cause high system load
moreover the found limit is no guarantee for stable startups since it depends on system configuration

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5583 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-07 11:53:48 +00:00
auron_x
49d95fb3d9 - cytag-xhtml-fix
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5582 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-06 15:31:28 +00:00
orbiter
01b97ef3f8 added new cybertag-tracking feature that was inspired by itgrl
from the forum discussion in
http://forum.yacy-websuche.de/viewtopic.php?p=12612#p12612

The feature will provide two basic entities:
- you can integrate image links which point to your yacy installation anywhere in the web.
  the image can be loaded with
  <img src="http://<yourpeer>:<yourport>/cytag.png?icon=invisible&nick=<yournickname_or_community_id>&tag=<anything>">
  This will place a invisible 1-pixel image. If you change the icon=invisible to icon=redpill, you will see a red pill
  Use this, to track your activity in the web.
- you can view your tracks at
  http://localhost:8080/Tracks.html
- There is a public api to your tracks at
  http://localhost:8080/api/tracks_p.json
  which needs authentication


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5581 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-06 15:06:19 +00:00
auron_x
03a16f6c20 - more XHTML-validation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5580 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-06 14:45:56 +00:00
auron_x
46af1e06dc - XHTML-Validation of some pages (Status.html still needs work)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5579 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-06 08:51:03 +00:00
borg-0300
b19bc611b0 gc: better logging
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5578 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-05 19:42:32 +00:00
orbiter
b1f9c00118 fix for bug in merge operator initialization
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5577 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-02-05 15:26:16 +00:00