Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
7,625 Commits 1 Branch 20 Tags 836 MiB
6fba6e7cee
Commit Graph

7625 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
lotus
6fba6e7cee fix: follow link target setting on image search 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7962 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-18 16:59:01 +00:00
orbiter
ce2a76d603 performance hack for search process 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7961 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-16 10:00:51 +00:00
orbiter
a6bb0f9af4 fixed missing menu entries in access tracker 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7960 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-15 23:26:09 +00:00
orbiter
aaf7a0feaa yet another cache strategy 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7959 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-15 22:40:01 +00:00
orbiter
8a428d3e77 ensure termination of pdf parser to avoid deadlocking of other processes during search result preparation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7958 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-15 11:17:38 +00:00
orbiter
2c4a672fe2 bugfixes and performance hacks for tabe index 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7957 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-15 11:17:02 +00:00
orbiter
dad5b586a4 added a concurrent warmin-up of Table data structures. that should speed-up the start-up process but may also cause stronger CPU load at that time. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7956 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-15 10:01:21 +00:00
orbiter
734059d33e performance hacks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7955 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-14 23:34:05 +00:00
orbiter
23e81b28b2 synchronization enhancements 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7954 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-14 21:19:02 +00:00
orbiter
dd4635e323 patches 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7953 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-14 20:11:27 +00:00
orbiter
65ab067491 migration to solrj 3.4.0 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7952 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-14 20:08:59 +00:00
orbiter
ffd848c7a9 moved the log, memory, processes and the messages into a new computation monitor main menu item 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7951 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-14 09:59:30 +00:00
orbiter
ef72fdac79 added keyboard-based search result page navigation: 
- page-up or tab switches to next search result page
- page-down switches to previous search result page

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7950 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-14 09:15:09 +00:00
orbiter
e48ce5d80e - style change for search box: larger font, selected by default 
- style change for search results: by default no parser, size, image info

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7949 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-14 09:05:06 +00:00
orbiter
5905392ca3 redesign and simplification of main menu; bundling of some sub-menues 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7948 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-14 01:24:56 +00:00
orbiter
e5a93a1742 fix for image name 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7947 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-14 00:33:08 +00:00
orbiter
5fd4f3fef8 fresh look for yacy icons 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7946 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-14 00:03:33 +00:00
orbiter
95790b82d9 replaced old-style favicon 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7945 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-13 23:34:36 +00:00
orbiter
bb0c045036 fix for problem with relocation of network 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7944 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-13 18:46:11 +00:00
orbiter
85a5487d6d YaCy can now use the solr index to compute text snippets. This makes search result preparation MUCH faster because no document fetching and parsing is necessary any more. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7943 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-13 14:39:41 +00:00
orbiter
0819e1d397 protection against OOM cases in image parser. See also bugs.yacy.net/view.php?id=54 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7942 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-09 23:00:45 +00:00
orbiter
52a2b3f110 try to fix bug http://bugs.yacy.net/view.php?id=26 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7941 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-08 19:13:19 +00:00
orbiter
b0b4886618 try to avoid the unresolved pattern in search result 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7940 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-08 18:47:00 +00:00
orbiter
2e8c8ebf54 fix for clear commands 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7939 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-08 12:54:10 +00:00
orbiter
2cba860693 - fix for wrong entries in NOLOAD indexing queue (that caused that urls had been only indexed based on their url and not loaded) 
- patch for better urls to solr admin interface

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7938 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-08 12:23:55 +00:00
orbiter
2842ce30d6 added synchronization in ReferenceContainer and logging for shrinking 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7937 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-07 22:15:01 +00:00
orbiter
cec3836e73 added reference limitation to IndexControlRWIs_p.html servlet 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7936 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-07 21:47:54 +00:00
sixcooler
ecb4986b38 refactored stuff from last commit to ReferenceContainer 
see: http://forum.yacy-websuche.de/viewtopic.php?f=5&t=3353&p=23163#p23163
the limiting of references is disabled per default
to enable this set yacy.conf - index.maxReferences to a value of e.g. 100000

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7935 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-07 18:55:16 +00:00
sixcooler
f7c4abfdd7 limit references per blob & term to the 100.000 youngest 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7934 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-07 13:08:06 +00:00
orbiter
28f5b79deb added a fast mass-deletion method 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7933 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-07 11:42:06 +00:00
orbiter
a70dbce41c added another file tool class to yacy-cora 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7932 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-07 10:09:35 +00:00
orbiter
49e5ca579f added new configuration property "crawler.embedLinksAsDocuments". If this is switched on (this is default now), the all embedded image, audio and video links from all parsed documents are added to the search index as individual document. This will increase the search index size dramatically but will also enable us to create a much faster image, audio and video search. If the flag is switched on, the index entries are also stored to a solr index, if this is also enabled. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7931 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-07 10:08:57 +00:00
orbiter
e02bfbde56 fix for solr url 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7930 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-07 09:07:40 +00:00
f1ori
41e146116a fixes size of document in case the server doesn't give the size in the header 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7929 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-05 12:21:25 +00:00
orbiter
580beb12a5 reverting SVN 7863; the synchronization was needed and no synchronization causes repeated DNS lookup for the same hosts 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7928 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-05 00:26:27 +00:00
orbiter
44d6416e2d ensure termination of shrink() 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7927 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-05 00:22:21 +00:00
orbiter
52230a6864 replaced catching of Exception with Throwable, which catches also Errors 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7926 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-05 00:09:48 +00:00
orbiter
877eaf6bcb switched off logging of org.apache.http which was suddenly switched on by default (??) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7925 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-04 23:28:15 +00:00
orbiter
e1a3d609aa moved merger object from Segment to IndexCell to enable a correct shutdown sequence. This solves a bug where yacy cannot be shut down during an index merge that appears during the shutdown phase. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7924 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-04 23:27:12 +00:00
orbiter
656286347e fix for javascript error during search (not ready yet) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7923 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-03 07:10:47 +00:00
orbiter
dc25c48fc9 added more libraries that are needed by solrj 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7922 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-02 23:43:04 +00:00
sixcooler
2cf61a40ce fixed a bug from 7856, where Snippet returned an error by mistake when Metadata was found 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7921 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-02 16:50:05 +00:00
orbiter
0229029dcf a bit protection against search result bugs in interactive search 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7920 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-01 16:08:33 +00:00
orbiter
610b01e1c3 - added a 'add every media object linked in a html document as a new document' to the html parser. This causes that all image, app, video or audio file that is linked in a html file is added as document. In fact that means that parsing a single html document may cause that a number of documents is inserted into the search index. 
- some refactoring for mime type discovery

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7919 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-01 16:05:00 +00:00
orbiter
9a8937f8b6 be more liberal when evaluating search results. This may cause that it is possible to fraud content on fresh peers, but that is better than looong waiting times for the evaluation of every link which causes that everybody rejects YaCy as 'too slow'. But this is only because of the high standards that YaCy sets to itself. If we are able to gain more users by lowering the standard, then that is useful. The option to set that flag to verify each link is still there. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7918 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-01 16:02:15 +00:00
orbiter
3da21c4266 protection against starting of a (second) yacy peer while another one is already running on the same port 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7917 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-01 13:13:21 +00:00
orbiter
2d03dc1804 removed unnecessary warning 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7916 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-01 10:37:14 +00:00
orbiter
cf8e3b0df8 small fix for count: overXX includes the count 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7915 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-01 10:25:27 +00:00
orbiter
6db8921a0f enhanced termlist 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7914 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-01 10:23:22 +00:00
orbiter
b5252ef91f added new word recommendation library in DictionaryLoader_p.html 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7913 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-09-01 10:14:17 +00:00