Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
6,582 Commits 1 Branch 20 Tags 836 MiB
353a924760
Commit Graph

6582 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
orbiter
3a50b5aa04 enhanced object hash computation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6816 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-15 14:19:29 +00:00
orbiter
1a8a134e0c continuing String-hash - to - byte[]-hash redesign that was started in SVN 6775 and continued in SVN 6790 
The result should be a less usage of new String() and less memory usage (since a String-encapsulated byte[] has 40 bytes overhead)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6815 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-15 13:22:59 +00:00
orbiter
dde394a977 - shifted some computation out of synchronization to allow more concurrency 
- removed synchronization where not necessary

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6814 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-14 23:22:06 +00:00
orbiter
f204076d25 removed usage of temporary files: causes too much IO 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6813 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-14 22:17:18 +00:00
orbiter
48b9371735 changed balancer re-load counter. causes less blocking here doing intranet indexing. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6812 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-14 20:57:26 +00:00
orbiter
650be3599f added a time-out to the RWI cache to flush the cache if it has not been written for ten minutes. This additional dump criteria is necessary because some data sources repeat their vocabulary and may cause that the number of words in a RWI does not increase while the number of references in the RWI set increases. Now the RWI Buffer is flushed every 10 minutes or later if at that time already a dump is ongoing. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6811 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-14 20:30:34 +00:00
orbiter
ff6cf24b80 replaced RowSetArray in ObjectIndexCache with RowSet to reduce complexity in MergeIterator. This complexity caused too much computing overhead when the RowSetArray had become very large. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6810 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-14 19:26:51 +00:00
orbiter
0d04ab1422 new access tracking data type strategy; previous data types may have caused deadlocks of httpd when performing statistic cleanups 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6809 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-14 16:18:04 +00:00
low012
fc43f3028e *) hopefully fixing NPE issue introduced in r6797 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6808 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-13 23:33:50 +00:00
orbiter
55d8e686ea performance hacks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6807 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-13 23:29:55 +00:00
orbiter
2f181d0027 introduced concurrency in HTCACHE storage compression 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6806 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-13 16:22:09 +00:00
orbiter
2e26744f4e more concurrency when normalizing RWI entries + cleanup 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6805 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-13 14:47:57 +00:00
orbiter
555b333041 fix for wrong count of server processes. may fix non-access problems in some cases 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6804 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-13 14:34:16 +00:00
orbiter
5d8238680f better thread dump statistic evaluation in threaddump servlet when creating multiple dumps for profiling: no distinguishing between thread states 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6803 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-13 12:19:09 +00:00
orbiter
aa083fc45c try to get a fix for OOM problem in case that there is no real problem with missing memory. 
See also http://forum.yacy-websuche.de/viewtopic.php?p=19835#p19835

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6802 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-13 11:39:54 +00:00
orbiter
70e6222978 more concurrency during search requests 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6801 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-13 11:12:36 +00:00
orbiter
4917f96729 fixes for some changes in SVN 6797 that caused NPEs when the bookmarks initialized 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6800 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-13 10:14:08 +00:00
low012
dff660441a *) changes for better code readability 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6799 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-13 01:31:16 +00:00
low012
15d9ea8375 *) changes for better code readability 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6798 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-13 01:25:15 +00:00
low012
2bc459252e *) changes for better code readability 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6797 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-13 01:16:09 +00:00
low012
dc93cec3a8 *) Java 1.5 compatibility (see http://forum.yacy-websuche.de/viewtopic.php?f=8&t=2764) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6796 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-13 00:25:46 +00:00
orbiter
67ec58d8e7 search performance enhancement 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6795 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-12 07:31:43 +00:00
hermens
4ec0092677 more null == proxy fixes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6794 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-10 18:31:12 +00:00
hermens
2f90f0ad56 Remove asserts blocking proxy use cases 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6793 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-10 15:12:39 +00:00
hermens
ef467a0303 Another workaround for the second part of http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2770 
This should prevent URLs with bad referrer entries from being dropped by transferURL or even crashing the whole Transmission$Chunk


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6792 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-10 13:57:46 +00:00
sixcooler
eb2a4bb555 workaround(?) for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2770&start=0&st=0&sk=t&sd=a&hilit=DefaultCharsetStringPart 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6791 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-10 00:21:07 +00:00
orbiter
25aef069a6 continuing String-hash - to - byte[]-hash redesign that was started in SVN 6775 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6790 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-08 00:11:32 +00:00
mikeworks
aad0a3045a Fixed CSS file yacyui-portalsearch.cs 
border-width: small; --> border-width: thin;
margin-botton --> margin-bottom
Forum Thread: http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2767

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6789 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-06 00:04:01 +00:00
mikeworks
6ef4900ec5 Changed function name status() -> statuscheck() in yacyui-portalsearch.js fixing the "status is not a function" bug visible in Firefox error console: 
http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2625

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6788 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-05 12:44:35 +00:00
low012
b97ad0f380 *) some minor changes for better code readability 
*) added more SVN properties

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6787 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-05 12:37:33 +00:00
mikeworks
7a3c19846f Updated German translation de.lng: added new Table_RobotsTxt_p.html and some other changes 
Changed 'Sprache' -> 'Language' in yacydoc.html and added translation in de.lng

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6783 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-29 00:56:51 +00:00
orbiter
ba51d140e1 added more info in assert in balancer 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6782 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-28 22:59:19 +00:00
low012
000dad034b *) better usage of JS timers (intervals) 
*) added messages when status is checked

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6781 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-28 22:17:36 +00:00
orbiter
a85c5bb8a7 added support for multiple (fail-over) network definition locations when http-locations are given. multiple locations can be given with a comma-separated list of urls pointing to the network definition file 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6780 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-27 23:15:15 +00:00
orbiter
f83b1b91b9 increased dht busy sleep time to 10 seconds to reduce TCP/IP traffic for default settings. 2 seconds had been too much traffic for home-use routers. 
Please try to set your dht busy sleep time in existing installations also to 10 seconds.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6779 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-27 23:04:00 +00:00
orbiter
9b3840cb66 performance hacks for the template engine + cleanup 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6778 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-27 22:52:48 +00:00
orbiter
5c10f8bc5f enhanced latest hack 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6777 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-27 07:19:49 +00:00
orbiter
b3238bec83 performance hack for httpd 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6776 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-27 07:09:55 +00:00
orbiter
1e8e79b9ef redesign of reference hash (URL-hash) parameter hand-over: 
pass value as byte[], not as String. This should cause that less
byte[] <-> String conversions are made during time-critical tasks.
This redesign is not yet complete, more to come ..

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6775 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-26 18:33:20 +00:00
orbiter
72d8e9897b removed unnecessary cache flush call in backend of BufferedRecords 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6774 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-26 12:44:13 +00:00
orbiter
749ffbd642 - added another catch case for the index dump and index merge process that should cause non-blocking behavior in case that index dump and/or index merge caused any unexpected exception. 
- reverted SVN 6766, this is too dangerous (may cause unexpected memory usage) and should not be necessary

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6773 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-26 10:46:40 +00:00
orbiter
9ddb8e4a43 set an option for the java-internal image parser that prevents that the image is cached using the file-system in a temporary file. This should speed up image parsing during image indexing dramatically and should also cause better performance when showing the yacy banner and OSM tiles. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6772 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-26 10:43:31 +00:00
orbiter
312ca5d917 removed flush at end of every rwi entry since this reduces the write performance. 
This should speed up RWI cache dump and RWI merge operations and should cause less blocking time during these processes for the indexer.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6771 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-26 10:41:20 +00:00
orbiter
0018163c07 moved table row/column matching method from front-end to back-end 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6770 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-26 10:01:27 +00:00
orbiter
e12f1fd821 - added setting of access rights for executable scripts after auto-installation 
The correct access right was missing expecially for bin/apicall.sh

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6769 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-25 09:51:01 +00:00
orbiter
21fcbcc35f added sorting function in network table, reverting SVN 6736 (not removing new sorttable) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6768 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-25 07:47:41 +00:00
orbiter
31e29a8831 - removed synchronization during index dump and index cleaning 
- added semaphores to synchronize index dump and index cleaning for each process separately

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6767 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-25 07:09:53 +00:00
orbiter
95f31da8da increase dump cache queue length from 1 to 2 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6766 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-24 20:36:35 +00:00
orbiter
fad3abb524 Tables_p.html servlet can now show tables with selected rows using a search field 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6765 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-24 10:52:37 +00:00
low012
4c6dc396d8 *) more beautyful (IMO) code, no functional changes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6764 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-23 21:05:26 +00:00