Commit Graph

8839 Commits

Author SHA1 Message Date
Michael Peter Christen
4d29f59a27 removed warnings 2012-09-10 07:15:52 +02:00
Michael Peter Christen
8c099d2106 Merge remote-tracking branch 'origin/master'
Conflicts:
	htroot/api/ymarks/import_ymark.java
	source/de/anomic/data/ymark/YMarkEntry.java
	source/de/anomic/data/ymark/YMarkTables.java
2012-09-10 07:05:20 +02:00
apfelmaennchen
59bd478ed1 Added more sophisticated RDF output for YMarks, including the folder
structure (b:Topic) and support for multiple tags (dc:subject) and
folders (b:hasTopic) via rdf:Bag container.
2012-09-09 22:56:24 +02:00
apfelmaennchen
d31a632951 - added dmoz RDF dump importer
- added indexing to Tables columns to support larger bookmark
collections
- added RDF output (HTTP) for public bookmarks at /YMarks.rdf
- YMarkRDF also provides a Jena RDF Model as "internal" API
- various other changes/fixes for YMarks (mainly backend)
2012-09-09 09:53:58 +02:00
Michael Peter Christen
10b911eed4 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-09-07 22:07:02 +02:00
Michael Peter Christen
be67c70a47 added Solr fields:
inboundlinks_text_chars_val
inboundlinks_text_words_val
inboundlinks_alttag_txt
outboundlinks_text_chars_val
outboundlinks_text_words_val
outboundlinks_alttag_txt
2012-09-07 22:06:51 +02:00
orbiter
d73fff0e0e added solr field images_withalt_i 2012-09-07 21:33:45 +02:00
orbiter
66ac4076c2 added disjunction '|' option to site parameter in GSA API 2012-09-06 22:35:55 +02:00
sixcooler
a975bcffcb clear fulltext-cache and stop crawling if running out of memory 2012-09-06 22:10:03 +02:00
sixcooler
e78fe3f477 also do a clearcache on the solr-connector-caches 2012-09-06 22:07:07 +02:00
sixcooler
9ee2e09983 statistics for solr-cache 2012-09-06 22:02:29 +02:00
Michael Peter Christen
d8425e6809 added collections to crawl monitor 2012-09-04 14:47:53 +02:00
Michael Peter Christen
ee23fc7a32 added h1..h6 counter fields 2012-09-04 14:11:11 +02:00
Michael Peter Christen
4b36a2c3b4 small style changes 2012-09-04 11:23:41 +02:00
Michael Peter Christen
8ca842b137 added new button design to more buttons 2012-09-03 16:04:57 +02:00
Michael Peter Christen
04709e91d7 add nice submit buttons to pdblue skin 2012-09-03 15:27:47 +02:00
Michael Peter Christen
ef6de52ab5 dependency is java6 only 2012-09-03 15:26:47 +02:00
Michael Peter Christen
b2b516cc3e added a collection attribute to crawls and searches:
- a solr field collection_sxt can be used to store a set of crawl tags
- when this field is activated, a crawl tag can be assigned when crawls
are started
- the content of the collection field can be comma-separated, all of
them are assigned to the documents when they are indexed as result of
such a crawl start
- a search result can be drilled down to a specific collection; this is
currently only available in the solr interface and also in the gsa
interface using the 'site' option
- this adds a mandatory field for gsa queries (the google api demands
that field all the time)
2012-09-03 15:26:08 +02:00
Michael Peter Christen
174530a9e0 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-09-03 00:46:17 +02:00
apfelmaennchen
43f3a932fd removed jquery.slider as it is already included as part of jquery-ui
package
2012-09-01 14:17:20 +02:00
apfelmaennchen
a01eb1b7fe removed unused jquery plugin slider as it is part of jquery-ui package 2012-09-01 10:25:22 +02:00
Michael Peter Christen
4815713ec7 added synchronization to solr server requests since lucene is not
thread-safe. We experienced problems as described in
http://stackoverflow.com/questions/5327978/lockobtainfailedexception-updating-lucene-search-index-using-solr
2012-08-31 15:16:33 +02:00
Michael Peter Christen
f75b3f8a47 added more patches to work without RWI data structure 2012-08-31 14:35:56 +02:00
Michael Peter Christen
a427a68bac removed many warnings 2012-08-31 14:07:33 +02:00
Michael Peter Christen
c72c435517 - moved the gsa search interface from /gsa/searchresult? to /gsa/search?
- fixed the NB field data
2012-08-31 14:00:53 +02:00
Michael Peter Christen
31d4d38804 - extended the solr interface by a references-by-word-count method
- reduced danger that a non-existing RWI database causes NPEs
- added Solr queries to did-you-mean: this makes it possible that our
did-you-mean algorithm works together with only Solr and without RWIs
2012-08-31 13:03:00 +02:00
Michael Peter Christen
528d6763fa - added new solr fields:
title_count_i, title_chars_val, title_words_val
description_count_i, description_chars_val, description_words_val
- added many asserts to ensure data type correctness from YaCy to Solr
and vice versa
- made many fixes according to new findings from these asserts (!)
2012-08-31 10:30:43 +02:00
Michael Peter Christen
3142e675e8 fixed problems with GSA api:
- better FS attribute
- highlightning of searched words in title
2012-08-29 16:48:53 +02:00
Michael Peter Christen
3b19fe7b52 - fixed num parameter in GSA api
- changed FS attribute in GSA api
2012-08-29 16:28:32 +02:00
Michael Peter Christen
2ddc33646a added new field for solr:
url_paths_sxt
url_parameter_i
url_parameter_key_sxt
url_parameter_value_sxt
url_chars_i
2012-08-29 16:11:23 +02:00
Michael Peter Christen
75d5e3475d Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-08-29 10:13:51 +02:00
cominch
a2841261bd content control: apply filter if enabled to crawls 2012-08-29 09:52:14 +02:00
cominch
dc468dad01 add content control features for custom filter lists 2012-08-29 09:04:28 +02:00
Michael Peter Christen
316b5fe116 - added a solr type definition verifier
- fixed type definition found by the verifier
- added multivalue-string fields for solr with extension 'sxt'
- added multivalue-integer fields for solr with extension 'val'
- renamed some solr attributes from txt to sxt
- changed solr query line to an explicit AND/OR structure
- added a country code second level domain list to Domains class; with
parser
- added a host string parser to get domain class name, country-code
second-level domain and subdomain out of it
- removed old coordinate attributes
2012-08-28 16:58:06 +02:00
orbiter
a3d5959981 Merge commit '65d49df865f60511d22d86fb15c33a082176e7ab' 2012-08-27 16:56:22 +02:00
Michael Peter Christen
4521d63c92 added boosts to solr search queries 2012-08-27 15:25:25 +02:00
Michael Peter Christen
4c79ddb91e switched off some solr logging 2012-08-27 14:41:47 +02:00
Michael Peter Christen
e8acd542b5 - added faceted drill-down for host and geolocation to solr queries
- added a new geolocation field to index schema, the old values are
migrated if possible
2012-08-27 14:41:33 +02:00
Michael Peter Christen
f00168ecc5 added gsa result attribute 'has' 2012-08-27 12:15:42 +02:00
reger
65d49df865 security fix: clear automtic password only if adminAccountForLocalhost=false to prevent remote access to protected pages after restart.
if adminAccountForLocalhost=true leave automatic password unchanged so access from local host is granted but remote access is preventet from the 1st second.
2012-08-26 22:28:14 +02:00
reger
52a62af184 Merge branch 'master' of git://gitorious.org/yacy/rc1.git 2012-08-26 22:12:59 +02:00
orbiter
2094df2e4e - correct length computation for BStringObject (bugfix suggested by
apfelmaennchen)
- using ASCII for string conversion for Strings generated from Integer
2012-08-26 17:46:40 +02:00
reger
2d2be546fe fix path to env/grafics to display api icon on meta data page 2012-08-26 04:36:52 +02:00
orbiter
6d03433cda - added hack to prevent that stream servlet paths are not parsed wrongly
if the path contains a dot.
- added also warnings if documents are requests which do not exist.
2012-08-25 19:08:42 +02:00
orbiter
a1227879a9 release 1.1 2012-08-24 23:59:10 +02:00
orbiter
7ac259477f added a direct access to solr search api to enhance the visibility if
the embedded solr
2012-08-24 23:04:19 +02:00
orbiter
67f2866cd0 small fixes 2012-08-24 21:44:22 +02:00
orbiter
ce156a01ba Merge commit 'c2341a175fdd755a34965ff63c7ea437b380352d' 2012-08-24 18:24:24 +02:00
David Rubio
c2341a175f Fixed a bug that prevented Yacy from indexing files with non ASCII filenames in FTP servers.
Previously Yacy could read file listings in UTF-8, but couldn't send commands to the FTP server in UTF-8 (the second byte of every multi-byte character was ignored), which caused a lot of errors on the server side.
Now it handles UTF-8 correctly.
2012-08-24 17:45:14 +02:00
orbiter
3ebc4264c5 fixed concurrent query 2012-08-24 14:15:40 +02:00