Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
6,808 Commits 1 Branch 20 Tags 836 MiB
906c572621
Commit Graph

6808 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
orbiter
906c572621 - enhanced index create menu structure 
- clear search log caches each time a search is done

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7142 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-14 09:06:27 +00:00
orbiter
fc924f024e import of oai sources from a list using a command line interface: 
if you have a list of oai servers you can import them all using the linux command:
bin/importOAIList.sh <name-of-oai-list-file>


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7141 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-13 10:13:34 +00:00
orbiter
64860dc1bb enhanced search event logging (to be used for further improvements) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7140 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-13 09:33:04 +00:00
orbiter
7dbc357593 patch to identify corrupted database files 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7139 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-13 07:20:53 +00:00
sixcooler
17eebd4ef8 counting crawler traffic again: 
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2808

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7138 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-11 15:58:15 +00:00
lotus
547d5226ae fix banner reload parameters (were no html errors) 
adapted default colours

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7137 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-11 11:05:07 +00:00
lotus
d2a3d08c44 avoid div. by zero 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7136 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-11 10:58:33 +00:00
orbiter
2c7edea35e - better shutdown behavior for the GUI (waits until data is written if GUI is killed) 
- release 0.97

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7135 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-10 12:47:24 +00:00
orbiter
34a25856a5 - added navigation to next/prev search page using arrow keys (left/right) 
- better information text for YaCy GUI application

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7134 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-10 10:42:01 +00:00
orbiter
32f73d1aaa added copy for Info.plist for Mac application release updates (this file contains class paths and start parameters) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7133 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-10 09:48:09 +00:00
lotus
5ce679a053 focus search field on load, no click necessary anymore 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7132 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-10 08:45:26 +00:00
orbiter
013926f01c added 'francais' as language option for default configuration 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7131 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-10 08:35:47 +00:00
orbiter
4c21d8dc9d - changed default values for online caution (the pausing may not be necessary any more) 
- fixed bug in WeakPriorityBlockingQueue
- show favicon faster using pre-loading (same technique as used for fast image search)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7130 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-09 23:25:19 +00:00
orbiter
570ca577c6 performance hacks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7129 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-09 22:42:54 +00:00
orbiter
348dece62f redesign of the SortStack and SortStore classes: 
created a WeakPriorityBlockingQueue as special implementation
of a PriorityBlockingQueue with a weak object binding.
- better abstraction of ordering technique
- fixed some bugs according to result numbering (distinguish different counters in Queue)
- fixed a ordering bug in post-ranking (ordering was decreased instead of increased)
- reversed ordering numbering using a reversed ordering. The higher the ranking number the better (now).

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7128 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-09 15:30:25 +00:00
hermens
03eb021568 Fix for byte[] Objects as keys 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7127 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-09 14:41:20 +00:00
orbiter
114bdd8ba7 fixed old sitemap importer which was not able to parse urls containing post elements 
- removed old parser
- removed old importer framework (was only used by removed old parser)
- added a new sitemap parser in parser framework
- linked new parser with parser access in old sitemap processing routines

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7126 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-08 14:13:15 +00:00
lotus
b73ea6581d fix json in case of query includes " 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7125 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-08 11:54:25 +00:00
orbiter
c0b08ac59b slighlty changed way of pdf parser integration 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7124 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-08 07:32:47 +00:00
orbiter
6d83c7cb62 removed unnecessary Override statements (produces errors in strict validation) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7123 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-08 07:15:41 +00:00
lotus
6a09f1f7e5 fix dedicated upnp testing 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7122 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-07 18:17:23 +00:00
orbiter
848ef6937e licenses for pdfbox 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7121 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-07 17:17:08 +00:00
orbiter
5fe828fa06 - replaced pdfbox and fontbox version 1.1.0 with 1.2.1 
- added some clear statements that shall clear static cache size within the pdfbox library
- the pdfbox library contains a memory leak; it is unsafe to run a peer with pdf parser permanently on.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7120 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-07 17:13:47 +00:00
orbiter
c757a4aa9f - corrected lifetime computation for search events 
- made search event cache cleanup concurrent because cleanup may cause index modifications

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7119 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-06 16:05:19 +00:00
lotus
ec8f1c0446 adapted _debug.bat to regular starter script 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7118 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-06 13:36:57 +00:00
lotus
5dff8f62c4 fix tray information display for non-windows 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7117 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-06 13:30:40 +00:00
orbiter
24502fe3de performance hacks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7116 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-06 12:59:33 +00:00
f1ori
a6c2e9ef0c add file.encoding=UTF8 to debian init script 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7115 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-06 12:53:03 +00:00
orbiter
ffaa9a1c51 avoiding double-loading of the same resource from the web in case that a seond attempt to load the resource is started while the first attempt is still loading the content from the web. This will delay the second attempt to the time when the first attempt has finished with the possible result that the second attempt reads only from the web cache, not from the web. 
This will also enhance the process of image result display from SVN 7105

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7114 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-06 10:23:30 +00:00
orbiter
fb828f3767 - performance enhancements in search response time using faster query ID computation and an ID cache 
- code cleanup

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7113 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-06 10:00:07 +00:00
orbiter
0ab6a462ee - added a missing entry in YaCy interface robots.txt for bookmarks 
- changed default robots.txt deny list to include some more interface pages because the loading of such pages are a peer load issue for YaCy when crawlers come by and information on these pages are not useful for public search. 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7112 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-06 09:58:54 +00:00
orbiter
d865ef77a8 removed re-read of index in case of a bad index. This may not solve the problem but it applies a 100% CPU problem on the peer. I'm afraid bad index files must be abandoned, and cannot be fixed this way. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7111 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-06 09:55:04 +00:00
orbiter
b2c9db48ea Performance enhancement 
- introduced byte[] - based ARC method for MapHeap which avoids a String generation each time the cache is accessed
- bugfixing in required class ComparableARC

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7110 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-06 09:53:33 +00:00
orbiter
ae07e11bc5 enhanced image search result display: concurrent loading of images before they are displayed 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7109 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-05 23:02:46 +00:00
orbiter
72a096fccb using -XX:-UseGCOverheadLimit when starting java. 
see also: http://forum.yacy-websuche.de/viewtopic.php?p=20709#p20709

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7108 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-05 14:19:02 +00:00
orbiter
22047ffad5 enhanced computation speed of many replaceAll string operations 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7107 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-05 13:19:42 +00:00
orbiter
e8228fba09 less locking in time format computation, caching and during secondary (remote) search evaluation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7106 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-05 11:13:12 +00:00
orbiter
9c0c94683c because of a bug in search result caching count search results had not been generated as fast as possible. 
with this fix search results are (even) faster.
Also enhanced: image search. This is now speeded up using a image search result look-ahead

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7105 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-04 22:57:12 +00:00
orbiter
fa2eb9676e removed unused class 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7104 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-04 21:45:33 +00:00
low012
5f391fcfa9 *) cleaned up in wikiCode parser (more to be done) 
*) HTML fixes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7103 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-04 14:01:34 +00:00
orbiter
b3f0d06444 fixed a problem with restarts in YaCy mac applications: the DATA directory path was not submitted when doing a restart. This solves the problem by: 
- storing the startup properties when yacy is started
- using the properties in the restart-script again. this transports also the DATA directory location as parameter of the -gui option that is used when the Mac version of YaCy is started

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7102 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-03 23:08:43 +00:00
orbiter
d4e4967e19 cleaned up code in yacyRelease (there will be work to do there) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7101 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-03 22:35:48 +00:00
orbiter
7be988768d simple selection of views in ViewFile.html (omit usage of button) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7100 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-03 22:35:07 +00:00
orbiter
2cb8cf5b41 added overall utf-8 default support for mac releases 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7099 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-03 22:34:08 +00:00
orbiter
698730c4eb better utf-8 support everywhere using a default utf-8 encoding for all string conversions. 
todo: add this also for debian start script (how do I do that?)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7098 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-03 22:18:10 +00:00
sixcooler
ca0a03e9ea ... migrating to HttpComponents-Client-4.x ... 
ssl-stuff: accept almost everything

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7097 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-03 16:02:52 +00:00
orbiter
d8f52c5b9c added a changelog url to download 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7096 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-03 12:55:36 +00:00
orbiter
1da5241c2d do not block server session if maximum number of sessions is reached, just try to clean up once 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7095 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-03 12:05:37 +00:00
orbiter
3988a95fb5 added ability in rss reader to parse atom feeds 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7094 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-03 08:53:24 +00:00
orbiter
5de70c3d7c changed way of storage for search requests: 
- the search request cache can now get as large as 1000 entries
- if more entries arrive, unused are deleted
- the elements may stay in the cache up to 10 minutes and longer if they are used
- the elements are deleted earlier that 10 minutes if the memory gets low
This commit was mainly done for metager-feeding peers that have a query load of 50000 queries each day. Also added:
- a monitor for cache hit/cache miss in PerformanceMemory_p.html (see at bottom of page)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7093 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-09-02 21:52:45 +00:00