Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
6,021 Commits 1 Branch 20 Tags 836 MiB
5e4f267a36
Commit Graph

6021 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
lotus
db70badcf0 possibility to set remote host on upnp device 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6097 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-19 15:32:59 +00:00
orbiter
4d29e90708 uaeh 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6096 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-19 14:54:39 +00:00
orbiter
3c3e6499ae added more logging for merge operation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6095 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-19 14:51:35 +00:00
orbiter
a119860b82 moved IndexImportWikimedia into different menu position 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6094 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-19 14:03:28 +00:00
orbiter
15180fc95e - patch for future computation in SplitTable 
- added same concurrent process for has() from SPlitTable in ArrayStack

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6093 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-18 23:24:23 +00:00
orbiter
9a5ec20b3c avoid merge during startup 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6092 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-18 21:18:56 +00:00
lotus
bf6b92343c try to avoid stuck pdf parser 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6091 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-18 06:31:55 +00:00
lotus
c695c7f512 try to remove hung swf parser from queue 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6090 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-17 17:48:02 +00:00
orbiter
fc69a76197 update to web structure picture: 
- allow bigger size
- better instructions for api usage

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6089 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-17 12:37:31 +00:00
orbiter
ae015e8e98 refactoring of blob package classes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6088 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-17 09:58:15 +00:00
orbiter
8b8877c233 moved image collector 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6087 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-16 21:48:09 +00:00
orbiter
be1c7ddc64 refactoring of search classes -- moved Ranking Profile to search package 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6086 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-16 21:45:40 +00:00
orbiter
1457bfce16 added updateYACY.sh to release build 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6085 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-16 21:02:41 +00:00
orbiter
fd31a3616a - more logging in server process 
- fix for bas ascii in comment

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6084 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-16 15:10:59 +00:00
orbiter
b5bc399cea added necessary synchronization for logging statistics (causes deadlock) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6083 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-16 10:37:13 +00:00
lulabad
e377a1e9a1 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1969 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6082 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-16 08:16:59 +00:00
orbiter
5a7fd6b4c8 just some comment lines 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6081 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-16 06:38:26 +00:00
orbiter
31f60a3b3e when doing searches, also apply a online caution to DHT transmission and stop transmissions while heavy load caused by searching. This omits the many requests to the URL database that are needed for DHT transfer and it avoids collisions with URL retrieval needed for search results. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6080 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-15 22:03:27 +00:00
orbiter
17dc6d4be5 small fix for new Logger 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6079 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-15 21:39:24 +00:00
orbiter
ce1adf9955 serialized all logging using concurrency: 
high-performance search query situations as seen in yacy-metager integration showed deadlock situation caused by synchronization effects inside of sun.java code. It appears that the logger is not completely safe against deadlock situations in concurrent calls of the logger. One possible solution would be a outside-synchronization with 'synchronized' statements, but that would further apply blocking on all high-efficient methods that call the logger. It is much better to do a non-blocking hand-over of logging lines and work off log entries with a concurrent log writer. This also disconnects IO operations from logging, which can also cause IO operation when a log is written to a file. This commit not only moves the logger from kelondro to yacy.logging, it also inserts the concurrency methods to realize non-blocking logging.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6078 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-15 21:19:54 +00:00
lotus
aec3e7995a autoconfig.pac can be used to browse .yacy-domains only 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6077 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-15 19:48:11 +00:00
orbiter
4e825852d2 added stub for phpBB3 search integration guide 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6076 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-15 11:52:57 +00:00
orbiter
bc6dd8194b refactoring: moved search query class to new search package 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6075 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-15 11:49:00 +00:00
orbiter
a4805defdd added stub for new search process 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6074 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-15 11:46:23 +00:00
orbiter
b8e738a7be a collection of 
- small bug fixes
- better/more comments
- more asserts
- fixed synchronization
- test case enhancements
- code cleanup
- performance hacks

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6073 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-14 22:09:08 +00:00
apfelmaennchen
39779e4796 DidYouMean: as I moved to only 8 consumer and 4 producer threads, I removed poison pills as it does not make sense anymore - threads are interrupted directly. Having a consumer thread per test case just didn't make sense either (see svn 6070) due to the massive overhead. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6072 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-14 16:31:31 +00:00
apfelmaennchen
c3c4dd0933 DidYouMean - changed to much simpler LinkedBlockingQueue 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6071 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-14 15:25:57 +00:00
apfelmaennchen
01ac1b5d7e - blocking queue implementation of DidYouMean 
- timeout ist set to 500ms

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6070 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-14 11:53:09 +00:00
orbiter
b8bb1bb364 join with a timeout does not cause that the corresponding thread is stopped after the time-out. It does only cause that the waiting is stopped. Here we need additionally a signal to the thread to stop after we finished waiting. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6069 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-13 23:54:52 +00:00
orbiter
b69f22e9ca mistake in last commit: computation of loops in ReversingTwoConsecutiveLetters 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6068 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-13 23:37:51 +00:00
orbiter
3130334932 - start first with threads that run more loops 
- join first with threads that run less loops

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6067 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-13 23:34:16 +00:00
apfelmaennchen
6cde7ebf16 DidYouMean 
- without I/O intensive sorting by count
- but with multiple threads

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6066 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-13 23:16:14 +00:00
orbiter
f348190566 tried to insert a database dump import method to the phpBB3 import function. Reason: imports or large database dumps are cannot be handled with phpMyAdmin and this should be an easy way to the database dumps into a mySQL database where it can be exported again with the phpBB3 content integration adapter. Completion or removal of this function stub will follow before next main release. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6065 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-13 23:03:40 +00:00
orbiter
945777aa80 replaced rwi term counting method by one that computes the maximum of the blobs that contibute to the RWI. An addition of the blob sizes is wrong/incorrect and does not reflect the real size. Truncation the size operation to the maximum of all blobs is also incorrect, but not as wrong as the sum of all blob sizes wich double-counts many rwi entries. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6064 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-13 22:59:54 +00:00
apfelmaennchen
303ccda69f small fix for "did you mean" 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6063 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-13 11:11:30 +00:00
orbiter
7c4d1d471c hand-over of more specific object 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6062 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-13 10:22:25 +00:00
apfelmaennchen
9150bc0f7d - don't show empty "did you mean" 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6061 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-13 07:02:50 +00:00
apfelmaennchen
6c116be536 - set default &meanCount=5 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6060 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-13 06:49:17 +00:00
apfelmaennchen
09acfa66d1 - improved "did you mean" 
- added &meanCount= to query string
- &meanCount=0 ==> no suggestion, no performance loss
- sorting suggestions by sb.indexSegment.termIndex().count()

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6059 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-13 06:20:05 +00:00
apfelmaennchen
da6ce37f7b - fixed encoding problem 
- added limit to 10 suggestions

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6058 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-12 21:36:26 +00:00
apfelmaennchen
54a48b4184 - added "did you mean" to search page 
- currently works for single word queries only!

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6057 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-12 20:36:03 +00:00
apfelmaennchen
31360ba40c - Updated ConfigLiveSearch.html 
- added documentation for load_js and load_css

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6056 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-12 05:57:08 +00:00
apfelmaennchen
ab09d8ebb3 - small noscript fix 
- noscript is now functionall but ugly

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6055 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-11 22:10:02 +00:00
apfelmaennchen
55ef9ae12a small fix for last post 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6054 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-11 21:42:34 +00:00
apfelmaennchen
36dc9b09ac - partial update to jquery-1.3.2 
- partial update to jquery-ui-1.7.2
- yacyportalsearch fixed sidebar for navigators


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6053 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-11 21:34:39 +00:00
orbiter
550312ac85 added new command script to do a auto-Update from command line. this will make it easy to do mass-auto-updates in private yacy clusters 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6052 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-11 11:31:26 +00:00
orbiter
0fc1168554 - reduced time-out for socket-connection communication from 20 seconds to 5 seconds. This is a test to find out if the time-out was a cause for problems in metager environments 
- turned a fine log entry in case of rejected connections on the server socket into a warning. (look for 'exceeding limit')


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6051 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-11 10:20:31 +00:00
orbiter
28b86385cd patch for bad behaving swf parser 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6050 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-11 09:54:48 +00:00
orbiter
d58b395993 fix for http://forum.yacy-websuche.de/viewtopic.php?p=15693#p15693 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6049 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-11 09:38:25 +00:00
orbiter
cffef67dc5 added a short info line about the latency monitor 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6048 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-10 23:03:29 +00:00