Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
5,611 Commits 1 Branch 20 Tags 836 MiB
4338dcf936
Commit Graph

3588 Commits

Author SHA1 Message Date
orbiter
4338dcf936 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2093&hilit= 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5937 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-09 19:07:34 +00:00
lotus
bad7ce9286 experimental option trayIcon.force for unsupported platforms. java 1.6 needed 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5936 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-09 18:35:02 +00:00
low012
ea27853c59 *) some refactoring 
*) added one assertion
*) no functional changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5935 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-09 13:43:38 +00:00
low012
d164b42604 *) cosmetics 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5934 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-08 19:26:36 +00:00
orbiter
17150b2950 fixed bug in snippet computation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5932 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-08 15:26:32 +00:00
orbiter
89aeb318d3 enhanced the wikimedia dump import process 
enhanced the wiki parser and condenser speed

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5931 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-08 10:36:13 +00:00
orbiter
5fb77116c6 added a submenu to index administration to import a wikimedia dump (i.e. a dump from wikipedia) into the YaCy index: see 
http://localhost:8080/IndexImportWikimedia_p.html

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5930 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-08 07:54:10 +00:00
hermens
df733af4fa Try not to loose content from ram during IndexCell.delete by moving ram.delete after the dangerous operations on the array (array.get and array.delete) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5929 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-06 12:53:17 +00:00
hermens
ac72005f2f Let IndexCell.remove remove entries from the ram portion of the DB as well. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5928 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-06 12:32:34 +00:00
orbiter
8ba7ff5353 a fix and another speed enhancement for the RWI cache 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5927 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-05 22:40:40 +00:00
orbiter
05f077e85f added stack trace output to solve problem in 
http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2076&hilit=&p=14612#p14612

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5926 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-05 20:24:20 +00:00
orbiter
71a4cadf31 better and more performant synchronization in SimpleARC, the caching object for word hashes. Speeds up indexing. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5925 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-05 20:19:51 +00:00
orbiter
e6773cbb33 better handling of RWI cache for concurrency and less overhead when writing new entries -> even more indexing speed 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5924 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-05 20:08:23 +00:00
orbiter
c097531e3d added a catch Exception to all thread to check if any of them silently dies without any other notification 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5922 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-05 06:31:35 +00:00
orbiter
083533e5ec fix for bugs in IODispatcher 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5921 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-04 21:37:59 +00:00
orbiter
21fbca0410 better scaling of HEAP dump writer for small memory configurations; 
should prevent OOMs during cache dumps

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5920 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-04 08:29:44 +00:00
orbiter
6e0b57284d better care for states of the IODispatcher 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5919 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-03 22:54:47 +00:00
orbiter
1db9cdd4e4 fixed bug in writing of robots.txt entries in case that host names exceeded 64 characters and some other problems 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5918 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-03 19:35:10 +00:00
f1ori
bde88b684a * splitt off yacyRelease from yacyVersion 
* added some gui infos about signatures


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5916 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-02 12:12:22 +00:00
orbiter
057ce14c8e more fixes (character encoding, parser exceptions, http client failure, blob writing) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5914 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-02 07:43:03 +00:00
orbiter
d2ac0aa682 - fixed possible bugs in Stack (may affect Crawler reset) and RandomAccess handling 
- increased default memory size to 180MB
- fixed possible bug in http client reset (there was a deadlock)
- bug in BOBHeap marked, but not solved, cause is still unknown.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5912 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-02 01:40:03 +00:00
lotus
1351d903a1 don't follow links like mailto: 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5909 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-01 08:53:50 +00:00
orbiter
e88a66bcae temporary disabling computation of all sublinks (check needed) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5908 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-01 07:30:53 +00:00
low012
ff5f82d780 *) removed description of removed commands from wikiHelp ([= =]) 
*) used format function of Netbeans for wikiCode to make it more readable, no functional changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5907 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-01 07:28:59 +00:00
orbiter
eacf95213a fix for crawling of mailto-links 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5906 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-01 07:25:55 +00:00
orbiter
9c6ac43f66 fixes for wiki parser 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5905 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-30 22:03:35 +00:00
orbiter
3a64c9d02f - fix for problem with concurrency when computing word hashes 
- fix for search in case that a urlfilter was used and zero results were returned

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5904 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-29 22:14:12 +00:00
orbiter
d3f8aa5a2a set of small fixes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5903 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-29 21:36:20 +00:00
low012
78ffb61297 *) got rid of unnecessary variable which might also fix IndexOutOfBoundsException 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5902 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-29 21:08:44 +00:00
orbiter
d31e6f9c14 fix for http://forum.yacy-websuche.de/viewtopic.php?p=14457#p14457 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5899 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-29 11:18:17 +00:00
orbiter
8d6212233b fix for IODispatcher 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5896 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-28 07:24:28 +00:00
orbiter
f678472f46 fix for quote problem in json output 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5895 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-27 22:27:02 +00:00
orbiter
d079d6dfdb small changes in surrogate reader, wiki code and portal test 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5894 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-27 20:30:43 +00:00
orbiter
07f09742bb set of small fixes and comments 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5893 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-27 15:29:50 +00:00
borg-0300
06ed4ef7b3 * better picture handling 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5891 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-27 11:19:15 +00:00
orbiter
5a634cab23 removed generation of anchor link sets in document types that describe container formats. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5890 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-27 08:46:11 +00:00
low012
f1244264b8 *) hopefully fixed bug reported in http://forum.yacy-websuche.de/viewtopic.php?t=2057 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5882 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-26 16:18:14 +00:00
orbiter
2e3186189b fix for mediawikiIndex surrogate producer + added concurrency 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5880 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-25 21:52:21 +00:00
apfelmaennchen
6f5ea7b1a8 small fix for previous post 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5879 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-25 21:28:08 +00:00
apfelmaennchen
138a0747e3 added serverObjects.putJSON as JSON has very particulare encoding requirements 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5877 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-25 20:56:29 +00:00
orbiter
d977dd9a96 fix for surrogate loader 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5870 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-24 22:54:40 +00:00
orbiter
9cb68353da fix for bug in ProfilingGraph for ppm >> 10000 ppm (!) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5868 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-24 13:18:20 +00:00
orbiter
9e4db75aac reduced internal logging and reduced memory that internal logging can use 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5867 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-24 12:09:04 +00:00
orbiter
c10c257255 attempt to fix a deadlock situation where the IODispatcher did not work. 
I suspect the dispatcher thread has crashed and queues filled so no indexing process was able to write data.
This fix tries to heal the problem, but I am unsure if it helps. To get a better view of the problem, some more log outputs had been inserted.
Added also a new attribut indexer.threads to get a control over the number of default threads for the indexer (default is 1)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5866 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-24 11:55:39 +00:00
orbiter
09987e93fd fixed some more bad handling of byte[] 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5865 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-23 22:02:12 +00:00
orbiter
1bcc1450cb more explaining error message in case of IOExceptions during html parsing 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5864 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-23 21:18:01 +00:00
orbiter
fe51f4d668 less synchronization may help to prevent deadlocks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5863 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-23 20:54:13 +00:00
orbiter
58802e4201 added missing success test in storeDocumentIndex, 
see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1922&hilit=

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5862 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-23 20:38:56 +00:00
orbiter
171e62bee5 addition to the fix from last commit (which did not work) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5860 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-23 16:36:21 +00:00
orbiter
059949a0d1 tried to fix problem with snippet fetch for second search page when verify=false 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5859 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-04-23 15:29:30 +00:00