Commit Graph

5823 Commits

Author SHA1 Message Date
luccioman
d28d612069 Added option to choose field delimiter in vocabulary import from CSV 2018-02-14 09:29:04 +01:00
luccioman
95f1954c78 Adjusted last blacklist entry example for a more accurate description
As discussed in issue #160 , blacklist entries can indeed currently not
be "complete" regular expressions, but must be structured as a domain
part, a separator character ('/'), and a path part.
2018-02-14 07:51:07 +01:00
luccioman
dbf4c1cd76 Improved blacklist entries editing operations :
- Fixes issue #160 : handle properly syntax exceptions with a user
friendly message
- Fixes loss of information on multiple blacklist entries editions
- Fixes loss of entries when moving entries from one list to another
2018-02-13 18:24:26 +01:00
reger
5df72c1c65 Remove now obsolete html for language-nav and ISO639 jar reference 2018-02-12 01:16:14 +01:00
reger
87077b8fb6 Adjust and move Language Navigator to be member of the navigatior plugin
list.
2018-02-12 00:16:34 +01:00
luccioman
eb20589e29 Fixed issue #158 : completed div CSS class ignore in crawl 2018-02-10 11:56:28 +01:00
luccioman
fa65fb1a03 Fixed loss of search modifiers on bookmark, recommand or delete result 2018-02-08 14:31:26 +01:00
luccioman
0cdee4e26a Fixed loss of "meanCount" search param when using facets or page buttons
Then on new search queries, no suggestions at all could be displayed.
2018-02-08 08:07:30 +01:00
luccioman
117a859879 Do not clear all search modifiers when unselecting one modifier.
Previously, when clicking a selected facet in the search results page to
unselect it, all other eventually selected modifiers/facets were also
removed.
2018-02-07 15:54:46 +01:00
luccioman
a9dc0874c0 Remove old query terms from search results suggestions links.
Especially when old terms were misspelled, suggestions links then
provided most of the time empty results.
2018-02-06 15:14:14 +01:00
luccioman
c71b545235 Enable results suggestions (Did you Mean) even when RWI is not enabled.
RWI is no more necessary for suggestions processing since commit
c40ba51ca6.
Revealed by a question about spell check from ouahpiti on YaCy forum
(http://forum.yacy-websuche.de/viewtopic.php?f=23&t=6084 ).
2018-02-06 12:33:44 +01:00
luccioman
9412881230 Added basic support for autotagging microdata annotated item types.
With the appropriate vocabulary settings in Vocabulary_p.html page, this
can produce Vocabulary search facets displaying item types referenced in
html documents by microdata annotation.
Tested notably, but not limited to, vocabulary classes/types defined by
Schema.org and Dublin Core.
2018-02-06 10:25:38 +01:00
luccioman
539925a275 Added an utility to generate/update XLIFF master file from lng files. 2018-01-29 18:34:47 +01:00
luccioman
41a6b052d9 Updated master and French translation for the IndexReIndexMonitor_p page 2018-01-29 16:51:00 +01:00
luccioman
929e0d6eae Replaced improper ByteBuffer.equals() implementation by Arrays.equals()
Renamed also ByteBuffer.equals() to startsWith() as this is the
appropriate function implementation semantics.
2018-01-29 13:38:25 +01:00
luccioman
8b572b7337 Commit Solr index before simulating or starting recrawl job.
This ensures up-to-date simulation query results, and recrawl
processing.
2018-01-26 10:31:13 +01:00
luccioman
5e2812c060 Automatically refresh running recrawl report when JavaScript is enabled.
For users who would prefer to keep JavaScript disabled, a manual Refresh
button is still available.
2018-01-19 11:58:52 +01:00
luccioman
0fce264ba4 Set reindex page to html5 and removed presentational only html tables. 2018-01-15 18:32:34 +01:00
luccioman
83df922afc Removed unused duplicated HTML id on header hidden field 2018-01-15 17:16:54 +01:00
luccioman
4e03335625 Added more details to the recrawl job report 2018-01-12 11:47:13 +01:00
luccioman
d95d393a0d Add a query link to local Solr to browse selected recrawl candidates 2018-01-12 10:48:54 +01:00
luccioman
59f7763af6 Display recrawl job report also when job is actively running 2018-01-11 09:53:27 +01:00
luccioman
0c9e0b3566 Record recrawl calls to make them schedulable 2018-01-10 17:05:53 +01:00
luccioman
433e241e4f Added a report info box about eventual last terminated recrawl job
For easier monitoring of recrawls.
2018-01-09 22:33:15 +01:00
luccioman
b2af25b14f Added a stop condition to the Recrawl busy thread 2018-01-09 10:22:26 +01:00
luccioman
421728d25a Made possible to customize selection query before launching a recrawl 2018-01-08 21:20:46 +01:00
luccioman
fab6e54fec Enforced controls (HTTP method, token) on ReIndex and ReCrawl operations 2018-01-07 15:25:16 +01:00
luccioman
8a4ea1c11e Added UI switch to control content domain constraint per search request 2018-01-02 08:13:14 +01:00
luccioman
36a45b3905 Added UI setting for strictness of content-type checking on media search 2017-12-29 11:32:42 +01:00
luccioman
e6907fdab3 Added optional search parameter/setting to control content domain filter
Thus allowing to choose at configuration or per search request, whether
extending or not results beyond strict content domain filter (image,
video, audio or application).

Related graphical controls to be added to user interface.
2017-12-23 18:56:17 +01:00
luccioman
d42c1773c8 Added UI setting for optional encryption with https on p2p searches 2017-12-22 11:01:02 +01:00
luccioman
09c4ee56a7 Added optional https support for remote crawl and profile operations 2017-12-21 18:41:32 +01:00
luccioman
5db1c9155a Do locale independant case conversion on hosts, schemes, and file exts.
Required for proper operation when the default system locale is Turkish,
as dottless and dotted i characters have specific case conversion rules
in this language.
2017-12-19 13:52:05 +01:00
luccioman
1c4803e40a Enable optional https support for /yacy/transferURL API calls.
Also updated some Javadoc and consistently use Switchboard instance as a
constructor parameter where relevant.
2017-12-19 12:30:49 +01:00
luccioman
79a2ba306a Updated links to Java Regular Expressions documentation to version 8 2017-12-19 11:14:20 +01:00
luccioman
17e004599d Started implementing optional https preference for protocol operations
Introduced through the new configurable setting
network.unit.protocol.https.preferred, defaulting to false for now.

Let choose to prefer using https when available on remote peers to
perform YaCy protocol operations including notably hello or transferRWI.

Not yet implemented for every YaCy protocol operations.
2017-12-15 11:28:46 +01:00
ScRe13
bb3d3fe074 fixed default loading default settings; load was populated with wrong value 2017-12-12 23:25:56 +01:00
reger
20bba135fe Show hide or show public surftip button depending on current config status,
to show the button to switch the status (hiding button of current status)
2017-12-10 01:25:20 +01:00
Michael Peter Christen
b907819cb4 Merge branch 'master' of https://github.com/yacy/yacy_search_server.git 2017-12-09 22:29:54 +01:00
Michael Peter Christen
25573bd5ab added a crawl filter based on <div> tag class names
When a crawl is started, a new field to exclude content from scraping is
available. The field can be identified with the class name of div tags.
All text contained in such a div tag where the configured class name(s)
match are not indexed, while the remaining page is indexed.
2017-12-09 22:29:35 +01:00
luccioman
640fed2a9c Removed Java 1.8 no more necessary version checking (fixes issue #147)
Java 1.8 is by the way now a prerequisite to run from latest sources.
2017-12-08 15:26:46 +01:00
luccioman
d95b288f19 Removed use of deprecated Jetty IPAccessHandler for client filtering.
Upgraded to InetAccessHandler.
Added InetPathAccessHandler extension to InetAccessHandler to maintain
path patterns capability previously available in IPAccessHandler but
lost in InetAccessHandler.

Filtering on IPv6 addresses is now supported.

Support for deprecated pattern formats such as "192.168." and
"192.168.1.1/path" has been removed, but startup automated migration
should convert such patterns eventually present in serverClient.
2017-12-08 15:12:08 +01:00
Michael Peter Christen
607b39b427 Merge branch 'master' of https://github.com/yacy/yacy_search_server.git
Conflicts:
	htroot/yacysearchitem.java
2017-12-07 15:25:41 +01:00
Michael Peter Christen
4355de0f3c (more!) evaluation of XRealIP from nginx reverse proxy 2017-12-07 15:16:11 +01:00
luccioman
f9cba827c0 Made "tld:" modifier case insensitive and IDN complient.
Thus allowing typing internationalized top-level domains with non ASCII
characters as tld: modifier.
2017-12-04 19:13:16 +01:00
luccioman
c5c3cc1274 Use HTTP Post operation for resetting memory monitoring state.
Fixes issue #145

Also added textual hint on the button, and display it only when it makes
sense, that is to say when the memory state is 'exhausted'.
2017-12-04 08:48:37 +01:00
luccioman
cb10daba92 Renamed Chinese & Greek lng files using ISO639-1 codes.
Previously named with their ISO 3166-1 country code : this way, when
setting language to "Browser" in ConfigBasic.html, it didn't work
properly when browser preferred language was Chinese or Greek as their
respective language codes are "zh" and "el" (not "cn" and "gr" which are
their country codes)
2017-11-04 11:06:05 +01:00
luccioman
4b61edff32 Added a help link to ISO 639-1 language codes list ref 2017-11-03 10:34:36 +01:00
luccioman
a994d439af Added description of spatial restrictions in search options 2017-11-02 08:57:00 +01:00
luccioman
8a48f80909 Added language HTML attribute to the search home page. 2017-10-31 08:19:04 +01:00