Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
8,615 Commits 1 Branch 20 Tags 836 MiB
ef0d09f103
Commit Graph

3871 Commits

Author SHA1 Message Date
sixcooler
9b6e4e46ca fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=4430 2012-07-05 14:06:00 +02:00
Michael Peter Christen
b0c408788b made class methods static where possible 2012-07-05 12:38:41 +02:00
Michael Peter Christen
5bd3c90907 - removed unnecessary semicolons 
- added default case for switch
 2012-07-05 11:18:31 +02:00
Michael Peter Christen
7c1ba99755 removed more unused method parameters 2012-07-05 10:44:30 +02:00
Michael Peter Christen
0301aba1e9 removed unused method parameters 2012-07-05 10:23:07 +02:00
Michael Peter Christen
241dd8410a removed snippet pattern filter - it was not used 2012-07-05 09:21:27 +02:00
Michael Peter Christen
d3964253ae - added @SuppressWarnings to unused servlet method parameters 
- removed unnecessary casts
- removed unnecessary throw statements
 2012-07-05 09:14:04 +02:00
Michael Peter Christen
ea10766bfd cleaned unnecessary nested code 2012-07-05 08:44:39 +02:00
orbiter
78fc3cf8f8 refactoring and new usage of SentenceReader: this class appeared as one 
of the major CPU users during snippet verification. The class was not
efficient for two reasons:
- it used a too complex input stream; generated from sources and UTF8
byte-conversions. The BufferedReader applied a strong overhead.
- to feed data into the SentenceReader, multiple toString/getBytes had
been applied until a buffered Reader from an input stream was possible.
These superfluous conversions had been removed.
- the best source for the Sentence Reader is a String. Therefore the
production of Strings had been forced inside the Document class.
 2012-07-04 21:15:10 +02:00
Michael Peter Christen
276a66a793 Adding a limit of 1000 links that a parser shall store during indexing. 
A limit was necessary because some web pages have such huge numbers of
links that it can easily cause a OOM just by the number of links.
The quesion if the number of 1000 links is sufficient or too weak must
be answered with the result of testing this feature.
 2012-07-03 17:06:20 +02:00
Michael Peter Christen
1825f165b8 better integration of blacklist according to use case 2012-07-02 13:57:29 +02:00
Michael Peter Christen
c18fa9fa75 Merge branch 'master' of git://gitorious.org/~reger/yacy/bbyacy-rc1 2012-07-02 12:20:57 +02:00
Michael Peter Christen
ce8d4b87d9 fixes for new eclipse 'Juno' warning 'Resource leak'. 2012-07-02 10:27:46 +02:00
reger
067728bccc add search result heuristic. adding a crawl job with depth-1 for every displayed search result (crawling every external linked page of displayed search result pages) 2012-07-01 00:12:20 +02:00
Michael Peter Christen
03280fb161 removed segments-concept and the Segments class: 
the segments had been there to create a tenant-infrastructure but were
never be used since that was all much too complex. There will be a
replacement using a solr navigation using a segment field in the search
index.
 2012-06-28 14:27:29 +02:00
Michael Peter Christen
9116013c64 - allow lazy initialization of solr value (if using 'lazy', then no 
0-values and no empty strings are written). This may save a lot of
memory (in ram and on disc) if excessive 0-values or empty strings
appear)
- do not allow default boolean values for checkboxes because that does
not make sense: browsers may omit the checkbox attribute name if the box
is not checked. A default value 'true' would not comply with the
semantic of the browsers response.
- add a checkbox in IndexFederated_p for the lazy initialization of solr
fields.
 2012-06-27 12:17:58 +02:00
Michael Peter Christen
96aeb127e3 generalized localhost naming. 
this is also a preparation for a better IPv6 implementation.
 2012-06-26 00:08:25 +02:00
Michael Peter Christen
77f795756c fixing redirects and status codes: storing of status code in 
ResponseHeader to make it available for late evaluations, like storage
in solr.
 2012-06-25 18:17:31 +02:00
Michael Peter Christen
8dd469b9dd added option to configure the autocommit delay time of solr on-the-fly 2012-06-25 14:59:46 +02:00
Michael Peter Christen
b9dfca4b0a - fixed IndexFederated Servlet / a embedded Solr can now be selected 
- added code stub for an embedded Solr but generation of Solr store is
still commented out (it works but is not yet ready for usage)
 2012-06-25 11:34:38 +02:00
Michael Peter Christen
fad3b14813 added jetty libraries, needed for future use as web server and as 
application server for the solr search interface
 2012-06-22 15:31:17 +02:00
Michael Peter Christen
b9d42fd9c8 using com.google.common.io.Files instead of homebrew methods 2012-06-22 11:39:17 +02:00
Michael Peter Christen
a5eb91fa60 refactoring 2012-06-22 00:49:32 +02:00
cominch
c63c3a4495 Show additional interaction elements in footer section on each page, if 
activated in ConfigPortal.html.
This footer is also visible in augmented browsing proxy mode.
 2012-06-20 18:04:23 +02:00
cominch
fa98657bb3 Augmented Browsing: changed the settings page 2012-06-20 09:10:39 +02:00
cominch
751eeade0d Merge remote-tracking branch 'original yacy/master' 2012-06-20 07:58:27 +02:00
cominch
84a11ec48c Corrected loading of default page settings on ConfigPortal.html 2012-06-20 07:55:28 +02:00
sixcooler
bea002dc15 correct table in new look of Crawler_p 2012-06-19 13:13:00 +02:00
Michael Peter Christen
8738336408 set Xms lower than Xmx 2012-06-19 08:45:49 +02:00
cominch
6b4545d6b0 Only load tag information if necessary 2012-06-19 01:40:22 +02:00
cominch
011f8a5818 Auto Tagging: Add hyperlinks to tags (provisional) 2012-06-19 01:24:06 +02:00
Michael Peter Christen
1d4e206b2b bugfix in vocabulary generation 2012-06-18 18:10:40 +02:00
cominch
2c89975378 Merge remote-tracking branch 'original yacy/master' 2012-06-18 16:16:46 +02:00
cominch
71047fe63a Augmented browsing: CSS fix 2012-06-18 16:16:31 +02:00
Michael Peter Christen
52f5d40043 better abstraction of document model generation 2012-06-18 15:55:18 +02:00
Michael Peter Christen
8b7c4d3144 produce a rdf output containing the triplestore with yacydoc; ie: 
http://localhost:8090/api/yacydoc.rdf?urlhash=yOiCM7Fh1hyQ
 2012-06-18 15:47:54 +02:00
cominch
f7160dae5c Merge remote-tracking branch 'original yacy/master' 2012-06-18 15:44:50 +02:00
cominch
e4555cbee3 Augmented browsing: Pass on additional action parameter 2012-06-18 15:44:01 +02:00
Michael Peter Christen
24bbe359ca integrate also geonames library files for less cities. these are more 
useful for tagging since less normal words are false-identified as
location
 2012-06-18 15:19:57 +02:00
Michael Peter Christen
5a41e739b4 better apilink description 2012-06-18 13:04:20 +02:00
Michael Peter Christen
e16e4bd2ba added ontology extraction in xml as api call for vocabularies 2012-06-18 13:02:12 +02:00
cominch
8cf47a8335 Merge remote-tracking branch 'original yacy/master' 2012-06-18 12:11:07 +02:00
cominch
b85f01a14e Augmented browsing: small UI fix 2012-06-18 12:01:03 +02:00
Michael Peter Christen
26cb1c65c2 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 
Conflicts:
	source/net/yacy/document/importer/OAIPMHLoader.java
 2012-06-17 23:50:44 +02:00
Michael Peter Christen
963f92ed9a - merged files 
- changed behaviour of delete button in vocabulary edit
- fixed size numbe in vocabulary listing
 2012-06-17 23:48:33 +02:00
cominch
d8815db877 Merge remote-tracking branch 'original yacy/master' 2012-06-17 23:07:00 +02:00
cominch
e4dab19045 Augmented Browsing: added template for document info bar 2012-06-17 23:05:53 +02:00
Michael Peter Christen
743b0ec89f - added size of vocabulary to vocabulary view 
- fixed bad terms in vocabulary-from-titles autogeneration
 2012-06-17 17:32:52 +02:00
Michael Peter Christen
22d5e33c5e added more methods to vocabulary generation: scrape document title and 
document author to vocabulary
 2012-06-17 14:53:16 +02:00
Michael Peter Christen
b2d1c25ebb removed warnings/unused entities 2012-06-17 11:22:08 +02:00