Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
5,326 Commits 1 Branch 20 Tags 836 MiB
46632f4385
Commit Graph

5326 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
orbiter
46632f4385 performance update to yacyURL 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5651 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-25 00:17:34 +00:00
orbiter
cf9b74e6e3 added another method to process url lists: extract hosts only 
This can be used like
java -Xmx2000m -cp classes de.anomic.data.URLAnalysis -host DATA/EXPORT/20090224213823.txt

changed als the call method to generate statistics, please use now
java -Xmx2000m -cp classes de.anomic.data.URLAnalysis -stat DATA/EXPORT/20090224213823.txt


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5650 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-24 22:51:07 +00:00
orbiter
89d8e824ed memory protection for URLAnalysis 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5649 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-24 22:05:09 +00:00
orbiter
0f6fa804ff performance update to URLAnalysis 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5648 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-24 21:35:33 +00:00
orbiter
8444357291 added new row interator in kelondro tables files that enumerates rows 
without an order by the primary key. The result is a very fast enumeration of the Eco table data structure. Other table data types are not affected.
The new enumerator is used for the url export function that can be accessed from the online interface (Index Administration -> URL References -> Export). This export should now be much faster, if all url database files are from type Eco
The new enumeration is also used at other functions in YaCy, i.e. the initialization of the crawl balancer and the initialization of YaCy News.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5647 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-24 10:40:20 +00:00
orbiter
e8f5f2f612 added tool to analyse url strings 
and to generate statistics about words occurring in urls

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5646 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-24 10:00:35 +00:00
apfelmaennchen
72cc1d3a6c YaCy-UI: fix for query string handling in yacyuisearch.html: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1869#p12878 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5645 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-24 07:48:23 +00:00
apfelmaennchen
784a7bb7fd YaCy-UI: small fix 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5644 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-24 07:03:55 +00:00
apfelmaennchen
aa13d03b99 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1874#p12914 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5643 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-23 23:08:58 +00:00
lotus
6117e083e5 option to customize tray label (tooltip) with tray.label 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5642 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-23 21:07:08 +00:00
orbiter
b8c3803bfc don't panic when canceling server sessions 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5641 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-23 17:15:52 +00:00
daburna
1df1fd91b0 small update 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5640 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-23 17:02:15 +00:00
orbiter
de714783b1 - added host, path, filename to search result 
- modified yacyinteractive, shows now also date
- added size attribut to export file in xml format

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5639 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-23 11:39:20 +00:00
orbiter
7cfab2ee9a added a size field to the search results. The file size is now shown in the html search page and the yacyinteractive. The field is presen in the rss and json 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5638 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-23 07:59:11 +00:00
orbiter
4693495bbe added sru-compatibility to yacyuisearch ('query' instead of 'search') 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5637 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-22 23:43:05 +00:00
lotus
9519d84372 changed "dooble" variable to "browserintegration" to be less specific 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5636 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-22 17:32:17 +00:00
apfelmaennchen
54c7698219 YaCy-UI: some clean up of base.css 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5635 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-22 16:00:24 +00:00
apfelmaennchen
5d806623d5 YaCy-UI: 
- fix for folders
- first experiments with topwords
- first experiments with bookmark tag folder

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5634 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-22 15:59:24 +00:00
lotus
8429083972 adjusted tray for dooble: 
you can now set dooble=true in yacy.init to disable the menu and browser popups by default

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5633 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-22 15:11:44 +00:00
apfelmaennchen
bd6a3083c5 YaCy-UI Widget: added a target='_blank' to search results 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5632 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-21 22:35:41 +00:00
apfelmaennchen
b6b39e9556 YaCy-UI Widget: 
- added parameters fontsize=[small,normal,big], height and width
- e.g. http://localhost:8080/yacy/ui/yacyui-widget.html?style=darkness&fontsize=small&height=400&width=300

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5631 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-21 22:25:07 +00:00
apfelmaennchen
be29ece9d0 - added a new YaCy-UI Widget: /yacy/ui/yacyui-widget.html?style=xyz 
- http://forum.yacy-websuche.de/viewtopic.php?f=9&t=1127#p12781

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5630 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-21 13:30:47 +00:00
orbiter
ef62ec635e removed overwriting of logging config 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5629 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-21 00:43:38 +00:00
orbiter
c852d2d70e - reject too old seeds 
- do not store the complete seed in the reverse name cache, only the hash of the peer

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5628 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-21 00:39:44 +00:00
orbiter
aca973e2d9 catch more exceptions 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5627 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-20 23:24:49 +00:00
orbiter
9559bc23fd automatic clean-up of dead connections 
(hope that works well..)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5626 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-20 22:27:02 +00:00
hermens
02dfd6183b Fix logging in serverCore 
Prevent NPEs from keeping stopped Sessions in the pool and blocking slots



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5625 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-20 18:54:01 +00:00
hermens
d30456e2c8 Fix logging in serverCore 
Prevent NPE:
I 2009/02/20 15:15:56 PLASMA check for Session_77.37.19.225:38812#0: 86515 ms alive, stopping thread
I 2009/02/20 15:15:56 PLASMA Closing main socket of thread 'Session_77.37.19.225:38812#0'
E 2009/02/20 15:15:56 SERVER receive interrupted - exception 2 = Socket closed
Exception in thread "Session_77.37.19.225:38812#0" java.lang.NullPointerException
        at de.anomic.server.serverCore$Session.run(serverCore.java:623)



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5624 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-20 15:12:00 +00:00
orbiter
4f9dae2571 remove reference in crawl entries 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5623 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-19 22:58:00 +00:00
orbiter
1ba4301920 automated interruption of dead incoming connections, if they are there for more than one minute 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5622 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-19 22:27:24 +00:00
orbiter
c12bb8a6d0 - refactoring of the http client 
- added a protection against memory leaks for the access tracker

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5621 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-19 16:24:46 +00:00
orbiter
5d3983faae the soLinger parameter was wrong. 
With soLinger=true the httpd looses connections
The effect can be seen when crawling the internal repository:
lost connections filled the client process queue until it was full
and no more connections were possible.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5620 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-17 16:22:15 +00:00
orbiter
62505bb3cb more bugfixes as recommendet by findbugs 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5619 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-17 09:12:47 +00:00
orbiter
6b450d09ca some fixes recommended by findbugs 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5618 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-16 23:31:54 +00:00
orbiter
4db80065ac select more 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5617 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-16 21:53:37 +00:00
orbiter
94c42691d8 - reject less transmissions as transmission receiver 
- do not flag too much receiver when something goes wrong during transmission as sender

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5616 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-16 21:28:48 +00:00
orbiter
f887fc159f try to reduce the large number of unclosed incoming connections 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5615 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-16 16:26:57 +00:00
orbiter
e04a0e05c3 fix for last commit 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5614 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-16 16:21:12 +00:00
orbiter
a9ad863686 second part of 'doubles' fix - better handling of doubles in RAMIndex. More logging. 
still missing: deletion of double entries in collections

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5613 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-16 16:13:48 +00:00
orbiter
59427064fb first part of 'doubles' fix (not fully ready yet) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5612 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-16 00:47:48 +00:00
orbiter
26978b2a25 - better memory protection in kelondro caches: computation of needed memory for cache grow 
- removed excessive gc calls
- step to 16 vertical DHT partitions

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5611 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-15 23:35:59 +00:00
lotus
e9e2fff47a better scaling on performance graph 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5610 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-15 17:36:13 +00:00
lotus
4aad461100 added UPnP support 
YaCy can now automatically forward ports on home routers
off by default

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5609 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-14 13:12:08 +00:00
orbiter
62762ba81a fix for thread dump 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5608 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-14 01:08:20 +00:00
orbiter
99b9788e54 fix for possible 100% CPU caused by concurrent access of HashMap 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5607 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-14 00:39:53 +00:00
orbiter
be0c492ae5 fix for memory leak bug in new dht transmissions 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5606 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-14 00:01:05 +00:00
hermens
2173865f92 Prevent race condition when switching timezones. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5605 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-13 11:59:50 +00:00
apfelmaennchen
40ab13c886 YaCy-Ui: small bug fixes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5604 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-13 11:00:34 +00:00
orbiter
40d9849aa4 - better control of chunk size in dht selection 
- more restrict values in selection
- step to 4 vertical partitions

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5603 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-12 14:47:32 +00:00
daburna
5d3233ca6c some updates for german translation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5602 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-02-12 14:24:14 +00:00