f1ori
f814e0fa81
enable warnings and fix most of it
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6196 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-07-11 21:01:27 +00:00
orbiter
dafffd0153
refactoring of parsers and document processing
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6182 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-07-08 21:48:08 +00:00
orbiter
1f1399e5c5
extending visibility of objects and methods to avoid synthetic accessor methods and increase performance
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6156 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-06-30 13:25:46 +00:00
apfelmaennchen
a10c8022d1
DidYouMean:
...
- limit the number of consumer threads to available CPUs
- added some javadoc
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6144 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-06-27 07:23:34 +00:00
orbiter
ce1adf9955
serialized all logging using concurrency:
...
high-performance search query situations as seen in yacy-metager integration showed deadlock situation caused by synchronization effects inside of sun.java code. It appears that the logger is not completely safe against deadlock situations in concurrent calls of the logger. One possible solution would be a outside-synchronization with 'synchronized' statements, but that would further apply blocking on all high-efficient methods that call the logger. It is much better to do a non-blocking hand-over of logging lines and work off log entries with a concurrent log writer. This also disconnects IO operations from logging, which can also cause IO operation when a log is written to a file. This commit not only moves the logger from kelondro to yacy.logging, it also inserts the concurrency methods to realize non-blocking logging.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6078 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-06-15 21:19:54 +00:00
apfelmaennchen
39779e4796
DidYouMean: as I moved to only 8 consumer and 4 producer threads, I removed poison pills as it does not make sense anymore - threads are interrupted directly. Having a consumer thread per test case just didn't make sense either (see svn 6070) due to the massive overhead.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6072 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-06-14 16:31:31 +00:00
apfelmaennchen
c3c4dd0933
DidYouMean - changed to much simpler LinkedBlockingQueue
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6071 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-06-14 15:25:57 +00:00
apfelmaennchen
01ac1b5d7e
- blocking queue implementation of DidYouMean
...
- timeout ist set to 500ms
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6070 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-06-14 11:53:09 +00:00
orbiter
b8bb1bb364
join with a timeout does not cause that the corresponding thread is stopped after the time-out. It does only cause that the waiting is stopped. Here we need additionally a signal to the thread to stop after we finished waiting.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6069 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-06-13 23:54:52 +00:00
orbiter
b69f22e9ca
mistake in last commit: computation of loops in ReversingTwoConsecutiveLetters
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6068 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-06-13 23:37:51 +00:00
orbiter
3130334932
- start first with threads that run more loops
...
- join first with threads that run less loops
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6067 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-06-13 23:34:16 +00:00
apfelmaennchen
6cde7ebf16
DidYouMean
...
- without I/O intensive sorting by count
- but with multiple threads
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6066 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-06-13 23:16:14 +00:00
orbiter
7c4d1d471c
hand-over of more specific object
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6062 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-06-13 10:22:25 +00:00
apfelmaennchen
09acfa66d1
- improved "did you mean"
...
- added &meanCount= to query string
- &meanCount=0 ==> no suggestion, no performance loss
- sorting suggestions by sb.indexSegment.termIndex().count()
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6059 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-06-13 06:20:05 +00:00
apfelmaennchen
da6ce37f7b
- fixed encoding problem
...
- added limit to 10 suggestions
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6058 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-06-12 21:36:26 +00:00
apfelmaennchen
54a48b4184
- added "did you mean" to search page
...
- currently works for single word queries only!
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6057 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-06-12 20:36:03 +00:00
