Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
9,573 Commits 1 Branch 20 Tags 836 MiB
a5e328d7c5
Commit Graph

9573 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Michael Peter Christen
6fafed2180 fix for solr cache when a delete buffer is filled and a document, which 
is the delete queue, is replaced with a new one.
 2013-05-03 02:03:30 +02:00
Michael Peter Christen
20b767f35e preventing score computation in solr where applicable 2013-05-03 02:02:35 +02:00
orbiter
7de5b9cfa0 fix for http://bugs.yacy.net/view.php?id=233 
- check geolocation coordinates and accept only those, which are
well-formed
- the solr push process does not stop crawling any more if after 20
requests to Solr Solr does not accept the record. Instead, a severe log
entry asks the user to create a bug request
 2013-05-03 00:24:39 +02:00
sixcooler
e145afb8d6 fix for PerformanceMemory showing UNRESOLVED_PATTERN by removing 
solr-cache-stuff, which is not available anymore
 2013-05-02 15:47:21 +02:00
Michael Peter Christen
ee217dbdee remove sort order in all cases where not needed 2013-04-30 11:44:56 +02:00
Michael Peter Christen
70e981b333 prevent that long-running deletion tasks block a hard commit. 2013-04-30 11:09:21 +02:00
Michael Peter Christen
bb4bf3d8fd infinity timeout bug protection patch 2013-04-30 11:06:48 +02:00
Michael Peter Christen
1b102d98d8 - added index deletion to index administration submenu 
- added index deletion processes to the process scheduler/recorder
 2013-04-30 02:11:28 +02:00
Michael Peter Christen
ee95e772cf Merge branch 'master' of git://gitorious.org/~saranshupscale/yacy/yacy-india-rc1 2013-04-30 00:20:42 +02:00
Saransh Sharma
ab686900c1 New Hindi Translation 2013-04-30 03:33:21 +05:30
Michael Peter Christen
d1be4127e7 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-04-29 19:31:40 +02:00
Michael Peter Christen
0e2ee00fea added an index deletion servlet and some style changes for the 
'dangerous' engage-button
 2013-04-29 19:30:53 +02:00
Michael Peter Christen
1aac722cc6 added another solr connector, the ConcurrentUpdateSolrConnector which 
does not block when long-running updates to solr are made. This is
realized using blocking queues which process all long-running tasks in
the background. Also some bugfixes to existing connectors.
 2013-04-29 19:30:04 +02:00
Michael Peter Christen
0af7803367 added more features to ScoreMap (pretty toString) 2013-04-29 19:28:17 +02:00
Michael Peter Christen
f36a7da5f6 - re-introduced existById in solr connector. 
- intruduced raw-queries for the re-introduced byId-Queries (they are
hopefully faster than full edismax queries)
- removed the cached solr connector (testing this) to rely only on the
solr built-in search caches. That should save some RAM (also). We will
see if this is usable.
 2013-04-28 21:20:14 +02:00
Michael Peter Christen
e4f7e5bcfe fixed bad css change 2013-04-28 20:09:45 +02:00
reger
46fa800bc7 added httpstatus_i to automatically switched on fields (used in all search queries) 2013-04-27 03:11:44 +02:00
Michael Peter Christen
3502b4c697 refactoring (renaming) of yacy-solr api 2013-04-27 01:32:18 +02:00
Michael Peter Christen
3a0fcfbeda Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-04-26 10:50:08 +02:00
Michael Peter Christen
25499eead5 - added a new field for the regular expression in crawl start 
- added the field in crawl profile
- adopted logging end error management
- adopted duplicate document detection
- added a new rule to the indexing process to reject non-matching
content
- full redesign of the expert crawl start servlet
The new filter field can now be seen in /CrawlStartExpert_p.html at
Section "Document Filter", subsection item "Filter on Content of
Document"
 2013-04-26 10:49:55 +02:00
reger
0a9b0992f3 RinkingSolr_p: include warning if boost field not in local index 2013-04-26 02:26:38 +02:00
orbiter
e1bfe9d07a - reduction of the concurrently running processes to make YaCy more 
adjusted to smaller and 1-core devices.
- the workflow processor now starts no process at all. these are started
as soon as parser/condenser/indexing queues are filled.
- better abstraction
 2013-04-25 11:33:17 +02:00
Michael Peter Christen
c091000165 added collection attribute also to the rss feed reader 2013-04-24 01:14:35 +02:00
Michael Peter Christen
43ca359e24 Merge branch 'master' of ssh://gitorious.org/yacy/rc1 2013-04-23 21:01:08 +02:00
Michael Peter Christen
2d60dfb3e1 Merge branch 'master' of git://gitorious.org/~saranshupscale/yacy/yacy-india-rc1 2013-04-23 21:00:49 +02:00
orbiter
f7571386a3 added a 'collection' property attribute in yacysearch.html which can be 
used to select between different collections as defined during a crawl
start with the 'collection' attribute. This actually implements the
ability to prepare search tenants which restrict their search results to
a specific collection. The main use for this is to provide tenants to
the yaml4 interface (at this time).
 2013-04-23 20:42:54 +02:00
Saransh Sharma
04b61e08c8 More Translation 2013-04-23 19:31:17 +05:30
orbiter
3e79bd4b1f Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2013-04-23 12:15:46 +02:00
orbiter
d571e739b6 increased row limitation for authorized users from 10000 to 100000000 in 
solr interface
 2013-04-23 12:15:33 +02:00
Michael Peter Christen
d937c55204 extended limitation of dom export size from 100000 to 100000000 2013-04-22 22:33:13 +02:00
Michael Peter Christen
fc2095ac67 some extensions to raster plotter to transform a RGB picture to an 
indexed color scheme. This is needed for gif animations
 2013-04-22 14:33:04 +02:00
Michael Peter Christen
c1a2175fbc added transparency to gif image animation and the integration to the 
YaCy httpd for on-the-fly generated gifs (including animated gifs)
 2013-04-21 12:29:05 +02:00
Michael Peter Christen
a1fffe8e86 fixed default ranking values 2013-04-21 12:27:27 +02:00
orbiter
5d442dad82 avoid NPE in regex checker 2013-04-20 10:53:49 +02:00
Michael Peter Christen
24bcf54100 Merge branch 'master' of git://gitorious.org/~saranshupscale/yacy/yacy-india-rc1 2013-04-19 09:55:33 +02:00
Saransh Sharma
b31793f5d6 Hello world 2013-04-19 13:12:23 +05:30
Michael Peter Christen
50421171c3 added new schema fields: 
hreflang_url_sxt and hreflang_cc_sxt
for
http://support.google.com/webmasters/bin/answer.py?hl=de&answer=189077

navigation_url_sxt and navigation_type_sxt
for
http://googlewebmastercentral.blogspot.de/2011/09/pagination-with-relnext-and-relprev.html

publisher_url_s
for http://support.google.com/plus/answer/1713826?hl=de

all fields are disabled by default and not written to the index.
 2013-04-18 17:21:17 +02:00
Michael Peter Christen
566d6c980c checking of document signature for a double-document check now refers 
only to documents within the same domain
 2013-04-17 16:15:27 +02:00
Michael Peter Christen
1d30082446 added hindi translation configuration 2013-04-17 12:57:27 +02:00
Saransh Sharma
ee9d50e4b8 Hindi Some parts only 2013-04-17 14:41:55 +05:30
Michael Peter Christen
d05dc07cff setting of new default values for ranking 2013-04-16 15:02:00 +02:00
Michael Peter Christen
97775fbebc fixed ranking for add-function queries: this did not work. The option 
was removed. All function queries are now boosts (multiplies the score
according to a function). This is also the recommended way to boost
rankings based on functions as explained in
http://nolanlawson.com/2012/06/02/comparing-boost-methods-in-solr/
 2013-04-16 14:45:14 +02:00
Michael Peter Christen
ac5fa9fe48 fix for result counter logging 2013-04-16 13:32:13 +02:00
Michael Peter Christen
298bf2deb5 fix to ranking configuration servlet 2013-04-16 12:38:16 +02:00
Michael Peter Christen
2db058b551 added in RankingSolr_p.html a select box to switch between different 
ranking situations. By default, four situations can be configured.
 2013-04-16 11:38:51 +02:00
Michael Peter Christen
6fbca35215 fixed api table navigation 2013-04-16 01:39:30 +02:00
Michael Peter Christen
7ab5093321 added new solr title_exact_signature_l and 
description_exact_signature_l to be able to identify unique title and
unique description fields.
 2013-04-16 01:35:15 +02:00
Michael Peter Christen
f24ac518e6 redesign of exists()-query (can now be called with query) and the 
CachedSolrConnector which based its cache on the key value. This will be
used to correct the title_unique_b and description_unique_b field.
 2013-04-15 14:08:30 +02:00
Michael Peter Christen
27d6222880 added new field host_extent_i which, after a crawl and postprocessing, 
holds the number of documents for the host where the document is hosted.
This is necessary for ranking and the norming of references per local
host in the ranking computation.
 2013-04-14 20:52:40 +02:00
Michael Peter Christen
579eb01a49 showing now the details of references count in host browser: 
external (ext), internal (int) and external hosts (hosts) for each
indexed document.
 2013-04-14 11:30:57 +02:00