Commit Graph

11657 Commits

Author SHA1 Message Date
Michael Peter Christen
0dc6e0a5f2 added option to enrich vocabularies with synonyms from synonym database 2014-11-19 18:12:43 +01:00
Michael Peter Christen
6a2a669db4 added loading of the synonyms file from addon/synonyms into the
knowledge loader
2014-11-19 17:36:56 +01:00
Michael Peter Christen
c67c5c0709 added new solr schema fields which record the occurences of vocabulary
matchings. These matches can be used for result boosting, i.e. if a
document contains words from a specific vocabulary, boost it.
2014-11-18 15:02:34 +01:00
Michael Peter Christen
a67a465415 fix field counter for multi-fields in html writer for the solr servlet 2014-11-18 12:11:18 +01:00
Michael Peter Christen
fdba8e2fa0 fix for 2-day network stats table: showing 48 instead of 24 hours from
peer history
2014-11-17 14:23:21 +01:00
Michael Peter Christen
ec9d021568 added option in vocabulary editor to import CSV files with different
encodings (preselected windows-type character encoding which is typical
for CSV files). Fixed also other problems with character encoding in
dictionary files. Automatically generated vocabularies are now also
noted in the API steering.
2014-11-17 14:22:40 +01:00
reger
b558433211 adjust tag cloud font size calculation
to limit max font size to ~ TOPWORDS_MAXSIZE
2014-11-17 01:24:30 +01:00
reger
3c818fc912 add a check of java version string >=1.7 to startup class
stopping start with error msg on version < 1.7
2014-11-16 01:26:07 +01:00
Michael Peter Christen
0550b54d56 added fix to postprocessing: avoid caching of postprocessing collection
to always get fresh lists of documents. This is necessary since the
postprocessing changes the same documents which the
postprocessing-collection query selects.
2014-11-14 16:34:55 +01:00
Michael Peter Christen
68e8039fd1 added high-precision scheduler for API processes. This allows also to
make the execution in dependency of available RAM or CPU load. The
default value for CPU load is 4.0 and the check runs once a minute.
2014-11-14 10:02:50 +01:00
Michael Peter Christen
8aee7f940e added missing class for latest changes 2014-11-13 01:30:12 +01:00
Michael Peter Christen
97039049e4 fix in key enumeration methods for cases where the enumeration is done
in reverse order.
2014-11-13 01:15:31 +01:00
Michael Peter Christen
7e1b0b6712 fix for wildcard patch in search queries 2014-11-13 00:59:30 +01:00
Michael Peter Christen
0a879c98e7 added new 'firstSeen' database table and necessary data structures which
hold a date for each URL to record when a url was first seen. This is
then used to overwrite the modification date for urls upon recrawl in
case that the first-seen date is before the latest document date. This
behaviour is necessary due to the common behaviour of content management
systems which attach always the current date to all documents. Using the
firstSeen database it is possible to approximate a real first document
creation date in case that the crawler starts frequently for the same
domain. As a result the search results ordered by date have a much
better quality and the usage of YaCy as search agent for latest news has
a better quality.
2014-11-13 00:58:58 +01:00
Michael Peter Christen
487a733c99 fix for catchall handling in search 2014-11-12 22:48:33 +01:00
sixcooler
33b0234454 added a input-field for setting 'fileHost'
Set this to avoid error-messages like 'proxy use not allowed / granted'
on accessing your Peer by its hostname.
2014-11-12 21:32:34 +01:00
Michael Peter Christen
421ee64f33 another fix to ordering of table indexes; fixes also network stats
graphics
2014-11-11 13:57:04 +01:00
Michael Peter Christen
1db476c67e fix for bad table iteration 2014-11-10 18:52:01 +01:00
Michael Peter Christen
e05b7332b9 html fix 2014-11-10 02:18:44 +01:00
sixcooler
bad5dd7836 update to httpclient-4.3.6 2014-11-09 23:06:36 +01:00
reger
c1ad265efd remove not used accordion javascript call for facet navs 2014-11-09 22:06:00 +01:00
reger
e4316e2d74 skip creation of local var in proxyhandler.storetocache 2014-11-09 04:17:14 +01:00
reger
8a4a2929ae upd NB project.xml to codec-1.9 2014-11-08 21:10:10 +01:00
sixcooler
9c6e3a6b1c fix assertation-failure in version-string for Solr-4.10.2 by changing
the assert - hope that is ok
+ add forgotten NB-Projekt-changes
2014-11-07 22:43:50 +01:00
sixcooler
725b206fb4 update to solr-/lucene-4.10.2 2014-11-07 18:51:31 +01:00
Michael Peter Christen
ecdfb35f09 added long variables to debug output in index browser 2014-11-07 18:12:09 +01:00
Michael Peter Christen
5c97ecb30f fix of bad query generation for search facets 2014-11-07 18:11:49 +01:00
Michael Peter Christen
95d87f00b3 fix for bad query generation in doublecheck in postprocessing 2014-11-07 18:11:23 +01:00
orbiter
72c2bc5189 fix for search in case where local peer has no local seed address in
portal mode
2014-11-02 21:16:51 +01:00
orbiter
5be352da99 Merge branch 'master' of git@gitorious.org:yacy/rc1.git 2014-11-02 20:35:08 +01:00
orbiter
a2b5cfb3cf added reverse button to tables, by default on now (to see latest entries
first)
2014-11-02 20:30:49 +01:00
orbiter
fceac5d2d4 added (missing) Tables_p.xml for table xml api 2014-11-02 20:10:32 +01:00
orbiter
0fcd8097a3 removed unused options from BusyThreads 2014-11-02 20:08:49 +01:00
Michael Peter Christen
fe8b1d137d emergency bugfix for 100% CPU in image drawing 2014-11-02 13:28:10 +01:00
Michael Peter Christen
92007e5d2d more enhancements to posprocessing speed 2014-11-02 12:52:23 +01:00
Michael Peter Christen
9a7fe9e0d1 fix for bad timing computation in postprocessing 2014-10-31 23:17:56 +01:00
Michael Peter Christen
bd16119a00 another fix for postprocessing (the query for "" on numeric field did
not work in external solr)
2014-10-31 17:44:45 +01:00
Michael Peter Christen
327e83bfe7 more fixes in postprocessing: partitioning of the complete queue to
enable smaller queries
2014-10-31 17:30:24 +01:00
orbiter
2bc6199408 more concurrency for postprocessing 2014-10-30 21:52:52 +01:00
orbiter
a83cf26c38 more fixes and enhancements to postprocessing 2014-10-30 20:53:57 +01:00
orbiter
71758f0d62 enhanced postprocessing by usage of a field-list generation to prevent
lazy initialization of the documents. This is useful because the
documents must be read completely anyway.
2014-10-30 18:05:48 +01:00
orbiter
dbafd4865e enhanced debug code in host browser 2014-10-30 15:47:44 +01:00
orbiter
7856fbdbe8 fix for npe (in rare cases) 2014-10-30 15:20:35 +01:00
orbiter
8a2b569d7c fix for literal computation 2014-10-30 15:01:27 +01:00
Michael Peter Christen
8f6587e87b fix for broken protocol navigation 2014-10-30 12:41:04 +01:00
Michael Peter Christen
5c962dd009 better scaling of network statistic graphs 2014-10-29 21:41:41 +01:00
orbiter
3ffe19b85c replaced old /api/table_p.xml servlet with /Tables_p.xml to avoid double
code
2014-10-29 17:23:58 +01:00
orbiter
856da2712b Merge branch 'master' of git@gitorious.org:yacy/rc1.git 2014-10-29 16:53:18 +01:00
orbiter
ca9cd7b58a more IPv6 fixes 2014-10-29 16:52:58 +01:00
Michael Peter Christen
b4585e9546 added new index size history image in /Status.html page 2014-10-29 13:37:44 +01:00