Commit Graph

13538 Commits

Author SHA1 Message Date
luccioman
e2f6427a63 Added a basic JUnit test for the Visio parser (vsdParser) 2017-11-22 09:06:16 +01:00
luccioman
1e9cdaabd4 Do locale neutral case conversion of HTML charset name.
Required to properly run on systems with default locale set to Turkish
language, as with this locale the 'i' character has different upper and
lower case flavors than with other locales.
2017-11-20 18:52:45 +01:00
luccioman
d41ad7af6f Restore initial locale at the end of a JUnit test case which modify it. 2017-11-20 18:50:49 +01:00
luccioman
7206f1ed71 Do locale neutral case conversions on domain names.
Required to properly run on systems with default locale set to Turkish
language, as with this locale the 'i' character has different upper and
lower case flavors than with other locales.
2017-11-20 18:47:46 +01:00
luccioman
398c66f06c Do locale neutral case conversions in MultiProtocolURL
For any relevant URL parts : host name, URL scheme, session ids or
technical parts (see https://url.spec.whatwg.org/#url-writing and
https://tools.ietf.org/html/rfc3986 for current standard references).

Remaining locale sensitive conversion used for detection of URL word
components in urlComps() makes sense but using detected language would
be preferable than using the default system locale.
2017-11-20 15:23:33 +01:00
luccioman
9531b83598 Do locale neutral case conversions in Classification
Required for people using Turkish language as their default system
locale, as with this locale the 'i' character has different upper and
lower case flavors than with other locales.
2017-11-20 09:48:46 +01:00
luccioman
bab5f0485f Added signing key to developer releases location. 2017-11-17 11:09:55 +01:00
luccioman
d22fc0d0a2 Updated lists of known sponsored and country-code TLDs.
Using current IANA reference list at
https://www.iana.org/domains/root/db .

As for previous update on known generic TLDs list, the generated URL
hashes on these domains stay the same but it improves performance of URL
hash computation for URLs on these domains.
2017-11-16 09:50:55 +01:00
luccioman
ac209cac2e Updated the generic top-level known domains list.
Using current IANA reference list at
https://www.iana.org/domains/root/db

The generated URL hashes on these domains stay the same but performance
is greatly improved as a DNS resolve request is required on URL hash
computation when the TLD part of the host name is unknown.

Hash computation mean time measured on 1541 sample URLs (one on each
TLD) and a computer with a DSL connection : about 230ms before change,
then only 20ms.
2017-11-14 09:42:09 +01:00
luccioman
938d8a9731 Added some JavaDoc 2017-11-14 09:24:13 +01:00
luccioman
c32ac9c4c7 Updated log path in informative message of stop script.
As highlighted by @Lew-Rockwell-Fan in issue #140, the two log paths
mentioned by the stopYACY.sh script were inconsistent.
2017-11-14 09:17:43 +01:00
luccioman
8f07df5f85 Upgraded com.twelvemonkeys.imageio dependencies from 3.3.1 to 3.3.2 2017-11-09 09:30:20 +01:00
luccioman
fcd57e2d0f Improved some JUnit tests isolation and resources release
The modified tests were successfull when run manually from an IDE such
as Eclipse, but failed occasionnally when run with maven as part of the
overall test suite.
2017-11-08 09:33:30 +01:00
luccioman
e0eda84c24 Remove old hard-coded holiday dates from DateDection class.
Replaced with rules based relative to current year as already done for a
part of the supported dates.
2017-11-07 19:02:09 +01:00
luccioman
f61260c4c7 Upgraded icu4j dependency from 59_1 to 60.1 2017-11-06 09:37:44 +01:00
luccioman
73977ec0fe Added a html parser charset detection unit test 2017-11-06 09:14:03 +01:00
reger
d14c47d4d3 upd to pdfbox-2.0.8.jar 2017-11-05 00:52:14 +01:00
luccioman
cb10daba92 Renamed Chinese & Greek lng files using ISO639-1 codes.
Previously named with their ISO 3166-1 country code : this way, when
setting language to "Browser" in ConfigBasic.html, it didn't work
properly when browser preferred language was Chinese or Greek as their
respective language codes are "zh" and "el" (not "cn" and "gr" which are
their country codes)
2017-11-04 11:06:05 +01:00
luccioman
4b61edff32 Added a help link to ISO 639-1 language codes list ref 2017-11-03 10:34:36 +01:00
luccioman
a994d439af Added description of spatial restrictions in search options 2017-11-02 08:57:00 +01:00
luccioman
46f37e38dc Customized Threads with generic name for easier monitoring. 2017-10-31 08:53:17 +01:00
luccioman
8a48f80909 Added language HTML attribute to the search home page. 2017-10-31 08:19:04 +01:00
luccioman
5ff76fdcb9 Fixed spelling 2017-10-31 07:52:30 +01:00
luccioman
2c3f0ff9e8 Updated search page keyboard shortcuts descriptions. 2017-10-31 07:44:37 +01:00
luccioman
af825e9ffc Use accessible labels for search home page radio buttons. 2017-10-30 08:07:59 +01:00
luccioman
046be566e1 Updated a license header typo. 2017-10-30 07:38:47 +01:00
Apply55gx
3c905a2a5c fix typo 2017-10-27 14:00:30 +02:00
luccioman
8e732d437c Enable HTTP Digest authentication for non admin users.
Also ensure authentication is not lost by Digest timeout when navigating
between index.html and search results page.

This way, running searches with extended features on a remote peer or a
password protected peer works with a regular user (with "Extended
search" rights). 
When authenticating on the search page with a user without "Extended
search" rights, it appears as authenticated, but has just its usual
access to the public search features.
2017-10-26 07:51:18 +02:00
luccioman
5161451a35 Stay authenticated when going to the search start page.
Otherwise, when authenticated as admin and navigating from search
results or admin pages to the search start page (/index.html), if
nothing is done on that page within HTTP Digest Auth timeout (about
2mn), then search is performed without authentication and so without
extended search features.
2017-10-24 09:54:54 +02:00
luccioman
d0bed78d02 Use the same top nav bar on index.html and search results.
Thus eventually including the same optional login link/status in the
search start page than in the results page, for the same convenient
login without the need to use the Administration section.
2017-10-24 09:34:03 +02:00
luccioman
d8eaf621cc Fixed blacklist returned location URL on empty parameters 2017-10-24 09:30:21 +02:00
luccioman
f678394ce5 Fixed loss of index page form values on 'more options' link click.
Restores the behavior introduced eleven years ago (see commit
479861a3cf) and lost by mistake 3 years
ago (see commit 617dd9c97b), when the
click handler started referencing a missing HTML id.
2017-10-23 18:28:11 +02:00
reger
b98acb33c3 upd to Solr 6.6.2 2017-10-22 20:00:00 +02:00
luccioman
af198b990b Added an optional login link/status to the search public top nav bar.
Thus allowing a more convenient way (wihout the need to go to the admin
section) to login when searching on your remote or password protected
peer and benefit from extended search features such as Heuristics,
Bookmarking or JavasScript resorting.

Can be disabled using the ConfigSearchPage_p.html.
2017-10-21 10:57:36 +02:00
luccioman
1de86cf1bf Fixed JPEG snapshot resizing when running on OpenJDK.
Resizing JPEG snapshot images through /api/snapshot.jpg failed when
running on OpenJDK, but rendered successfully with a Oracle JDK.
Details in mantis 772 ( http://mantis.tokeek.de/view.php?id=772 ).

Removing any alpha component (useless in snapshot images) from the
rendered resized image solves the issue.
2017-10-19 09:27:52 +02:00
luccioman
c1c4174816 Updated Java version information on Readme 2017-10-18 14:17:06 +02:00
luccioman
a17a418e78 Fixed NullPointerException cases on snapshot images parsing. 2017-10-18 08:31:18 +02:00
luccioman
285f0d6a39 Consistently encode snapshot image with format requested on the API.
Previously, calling /api/snapshot.png rendered JPEG encoded images.
2017-10-18 07:53:07 +02:00
luccioman
4da15db998 Fixed search result Snapshots link.
Previously rendered as a broken URL containing the absolute file path of
a snapshot on the search server.

Now rendered as a valid URL linking to the /api/snapshot API to provide
available snapshot content. Snapshot format is selected among the
available ones in the following order of preference  : JPG/PNG, PDF, and
XML.
2017-10-17 09:41:58 +02:00
luccioman
34ca73d61b Fixed a NullPointerException case on images encoding errors. 2017-10-16 19:47:18 +02:00
luccioman
7c319c841e Fixed pdf2image conversion with imagemagick on PDFs having transparency
The target image format (jpeg) doesn't support transparency, so the
Html2ImageTest produced unusable black images when ran on a linux
machine having imagemagick package installed.
2017-10-16 19:45:17 +02:00
luccioman
6e497241f7 Properly close resources (even on error) on OS and ThreadDump classes.
Also updated some JavaDoc and main() function usage message on the same
ones.
2017-10-16 17:04:22 +02:00
luccioman
fe75f326d8 Fixed ProfilingGraph calculation integer overflows and added test class.
Complementary to fix proposed in PR #128 by @otteresk.
2017-10-16 09:18:12 +02:00
luccioman
5d1ef8fdfc Merge branch 'master' of https://github.com/otteresk/yacy_search_server 2017-10-16 09:01:34 +02:00
luccioman
8303e15419 Reduced number of search navigators refresh requests in JS resort mode
The SearchEvent listen to changes on each of its navigators, and the
information about their overall state is sent with each fetched search
item (as a "data-nav-generation" attribute). Then the browser can
regularly fetch a fresh version of yacysearchtrailer.html only if
necessary (when that nav-generation value change).
2017-10-12 07:16:19 +02:00
luccioman
2ac78e2cca Addedd missing parameters to yacysearchtrailer call on JS resort mode 2017-10-11 07:13:28 +02:00
luccioman
ebe91c7075 Merge branch 'master' of https://github.com/yacy/yacy_search_server 2017-10-09 19:08:57 +02:00
luccioman
5bf76f058a Adjusted ResponseHeaderTest to succeed on slow or highly loaded CPU 2017-10-09 19:08:39 +02:00
luccioman
a545523ab9 Added a Travis build status image to Readme 2017-10-09 14:25:43 +02:00
luccioman
dbff7b14fc Add a configurable limit to tags initially displayed in search results
When the limit is reached, a button allow expanding/collapsing remaining
tags.

When this feature is activated without a limit to the number of
displayed tags, when encountering search results with a very large
number of keywords, the results page can become almost unusable (very
long vertical scrollbar)
2017-10-09 14:13:46 +02:00