Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
13,127 Commits 1 Branch 20 Tags 836 MiB
0feded21dd
Commit Graph

5655 Commits

Author SHA1 Message Date
reger
036c1dc6ef fix CookieTest_p formatting (output of <br> as text), 
change to dataoutput only by servlet, leave formatting to html.
+ removed link to obsolete env/grafics gif
 2016-08-20 22:23:47 +02:00
Michael Peter Christen
ed8bc5f79d fix for bug in network page: the network table pages also loaded the 
network images, even if they had not been showed.
 2016-08-19 17:18:08 +02:00
luccioman
f84e584d74 ConfigPortal : fixed mixed-content security issue with https 
We now use the same protocol as the one used to display the config page
: so when using https, the content is not blocked by the browser
detecting mixed-content.
 2016-08-17 17:05:40 +02:00
luccioman
4743ade66a Search comparison : prefer https URLs 
Also disabled currently not working URLs.
 2016-08-17 16:13:46 +02:00
reger
91ab8a526a add error msg to api/share.html 
and skip display of url on nothing uploaded
 2016-08-17 03:07:26 +02:00
reger
5bbedad091 use config for path to locales in Translator servlet 2016-08-15 04:13:06 +02:00
luccioman
c9868209e4 Fixed opensearch ShortName consistency with link in index.html 
Fix for mantis 679 (http://mantis.tokeek.de/view.php?id=679)
 2016-08-12 12:36:44 +02:00
luccioman
744c9a2615 Opensearch desc : handle https protocol url with default port (443) 
This completes modifications made for mantis 669
(http://mantis.tokeek.de/view.php?id=669)
 2016-08-12 12:18:26 +02:00
Andreas
1f48d65b66 Merge pull request #7 from yacy/master 
Get my fork synced #7
 2016-08-09 21:27:57 +02:00
luccioman
480027ec98 Merge remote-tracking branch 'origin/master' into heroku_experiments 2016-07-28 02:29:40 +02:00
reger
fcad2d0744 add uses of config constant INDEX_RECEIVE_ALLOW 2016-07-27 02:16:20 +02:00
luccioman
d6bf90803f Merged from maain master branch. 2016-07-12 09:05:31 +02:00
luccioman
9b9c112263 Handle more propertly local port configuration by system property 
And prefixed property with "net.yacy" to avoid ambiguity.
 2016-07-12 01:53:01 +02:00
reger
a952787712 adjust opensearchdescription to return url with protocol it was call on 
fix http://mantis.tokeek.de/view.php?id=669
 2016-07-11 02:33:12 +02:00
reger
360b38d9b6 fix CookieTest_p parameter from ResponseHeader to RequestHeader 2016-07-10 05:44:56 +02:00
luccioman
76cf7dea2d Fixed inconsistent port check on ConfigBasic 2016-07-07 03:13:52 +02:00
luccioman
893a40995a Merge branch 'master' of https://github.com/yacy/yacy_search_server.git 2016-07-04 21:24:40 +02:00
Orbiter
50c5ddf1a1 Merge pull request #56 from luccioman/LibreJS 
LibreJS compliance : YaCy JavaScript license information
 2016-07-04 21:07:11 +02:00
Orbiter
82f40aefb5 Merge pull request #47 from reelsense/patch-2 
Distilled sentence - ConfigNetwork_p
 2016-07-04 21:05:59 +02:00
luccioman
6e96c7341a Merge remote-tracking branch 'origin/master' 
Conflicts:
	htroot/Load_MediawikiWiki.java
	htroot/Load_PHPBB3.java
	htroot/ViewImage.java
 2016-07-03 18:59:00 +02:00
reger
7bac756720 prevent dealing with -UNRESOLVED_PATTERN- eventID parameter in html includes 
on first landing on search page
 2016-07-01 00:02:10 +02:00
reger
b71a60c04b fix NPE in CrawlMonitorRemoteStart servlet due to missing startURL 
+ add a startURL attribute while generating news record for above (in Crawler_p)
 2016-06-27 03:12:39 +02:00
Orbiter
5fbe8b8d23 Merge pull request #58 from luccioman/responsive 
Make YaCy usable on small screen devices
 2016-06-19 22:14:51 +02:00
reger
f1be40eb07 Translator servlet filename drop-down list on "filter untranslated" switch, 
filter list to contain only files with not translated items
 2016-06-18 17:06:40 +02:00
luccioman
692aab1318 Merge remote-tracking branch 'origin/master' into responsive 2016-06-16 11:53:22 +02:00
reger
a3ac6ca7df add button (Translator servlet) to switch one already translated line in edit mode 2016-06-15 22:32:27 +02:00
luccioman
c2ade2f209 Fixed rendering for many mobile devices. 2016-06-15 09:07:54 +02:00
reger
e952e355a2 have Translator servlet adhoc apply added translation by translating a single file 
+ fix NPE in Translator, coming from translation read by TranslatorXliff 
  which allows null content for not translated key's
 2016-06-14 22:14:46 +02:00
luccioman
009657791e Merge remote-tracking branch 'origin/master' into LibreJS 2016-06-09 14:44:51 +02:00
luccioman
fbbfeeb313 Added last missing JS licences 2016-06-09 09:32:56 +02:00
luccioman
a73c9327a5 JavaScript License fixes for LibreJS compatibility 2016-06-08 23:16:10 +02:00
luccioman
8b95e5c91f Aplied GNU licensing recommendations. 2016-06-08 09:56:16 +02:00
reger
f86df5c76e add DE translation Translator_p 
+ html conform order form/fieldset
 2016-06-08 01:51:51 +02:00
luccioman
df74c435be Added license info for jquery.tagsinput. 
Also updated some license urls and identifiers to ones recognized by
LibreJS.
 2016-06-07 23:00:41 +02:00
luccioman
d4da22d634 Merge branch 'LibreJS' of 
https://github.com/luccioman/yacy_search_server.git into LibreJS

Conflicts:
	htroot/jslicense.html
 2016-06-07 09:38:00 +02:00
luccioman
f7a4be3c60 Added remaining scripts license information 2016-06-07 09:34:33 +02:00
luccioman
c1cc2d811d Added d3.v3.js license information 2016-06-06 09:44:48 +02:00
luccioman
3f6fefb125 Added license information for YaCy owned js files 2016-06-06 09:20:03 +02:00
reger
5b22c63030 Adjust TranslatorXliff to load default 1st and merge downloaded or modified local translation. 
process 1. load default from locales/*.* 
        2. load and merge(overwrite) from DATA/LOCALE/*.* (can be partial translation as it is merged)
- include all entries from DATA/LOCAL to be edited in Translator servlet
  and save just modifications (instead of full list) to DATA/LOCALE

This shall make it easy to share modifications.
 2016-06-05 23:01:45 +02:00
reger
786c92f95d disable input field of translated text in Translator_p servlet 
As currently only adding missing translation is supported
 2016-06-04 23:51:28 +02:00
luccioman
02ecb8de29 Added JavaScript license information 
First pass applied for YaCy index and administration first page, checked
with LibreJS 6.0.13.
 2016-06-03 22:07:45 +02:00
reger
e46245b819 fix accidental use of Java 8 function in Translator_p 
thanks to luccioman
 2016-06-03 21:42:14 +02:00
reger
1dcb9c5158 on language file delete, del local editfile too (in ConfigLanguage_p servlet) 2016-06-03 02:46:11 +02:00
reger
a6ba1faa80 introduce a translation edit servlet Translator_p.html YaCy's UI text translation 
This is the 1st rudimentary approach to support the translatio utilities.
It allows currently to edit untranslated text and save it in a local translation file
in the DATA/LOCALE directory.
+ refactor Translator (less static's) to leverage on class overrides and support garbage collection for this 1 time routine
+ adjust TranslatorXliff to check for local translations in DATA/LOCALE,
  this includes storing manually downloaded translation files in DATA as well 
  (to keep default untouched)
+ on 1st call of Translator_p a master tanslation file is generated, checking
the supported languages for missing translation text (later this masterfile is planned to part of the distribution, to harmonize translation key text between the languages)
Outlook: the local modifications (possibly as translation fragments instead of complete file) to be shared with maintainer using xlif features.
 2016-06-03 01:46:30 +02:00
reger
c7bd040870 correct translation key "About This ...." for simpleheader.template 
in de, ja
+ rem obsolete key "Peer Owner Profile" in cn, sk, uk for simpleheader.template
+ hide empty account lines in ViewProfile servlet
 2016-05-30 00:53:53 +02:00
reger
f23d8ab47b fix 2 more servlet RuntimeException in intranet mode thrown due to seed.getIP() 
returning null in intranet mode (in servlets: ConfigSearchBox, Load_PHPBB3
+remove unused (const &empty;) seed.IPTYPE
 2016-05-29 20:35:57 +02:00
reger
19509438d9 fix RuntimeException in ConfigRobotsTxt_p in intranet mode 
due to getIP returning null, by removing redundant property setting of "clientname"
which is globally set by template engine
 2016-05-27 17:30:17 +02:00
reger
6384b7d82e fix NPE in Load_MediawikiWiki servlet in intranet mode 
- in intranet mode getip returns null causing a NPE
  - adjust starturl (which was set to http://localip/repository) which is never the start url for the Mediawiki
+ correct javadoc for seed.getIP()
 2016-05-27 03:10:25 +02:00
Michael Peter Christen
bfd4e5f9e5 added hint in status to upgrade to java 1.8 2016-05-25 06:58:51 +02:00
Michael Peter Christen
d52c5b8c4d added column in network table for JRE 2016-05-25 06:47:06 +02:00
Michael Peter Christen
883a96f21a Merge pull request #53 from luccioman/accessibility 
Use aria role to reduce rssTerminal verbosity.
 2016-05-24 13:31:03 +02:00
reger
4cc38e979d add InputStream close after reading input file (Vocabulary_p servlet) 2016-05-24 00:26:28 +02:00
luccioman
623bcc729a Use aria role to reduce rssTerminal verbosity. 
This is a fix for http://mantis.tokeek.de/view.php?id=643
 2016-05-23 07:55:47 +02:00
Burkhard
9a18e2297b Merge pull request #51 from JeremyRand/multiple-boost-query 
Fix multiple boost queries
 2016-05-22 22:24:04 +02:00
reger
f0d7b93372 make use and activate autodetect charset in Vocabulary input from file 
+ revert mistake of empty cn.lng
 2016-05-22 05:38:26 +02:00
JeremyRand
433217b33e Properly support multiple Boost Queries. (Previous code was broken because it concatenated multiple Boost Queries together rather than passing Solr an array.) 2016-05-20 20:17:51 -05:00
reger
d0a571bed2 del cytag trail for own index.html (save resource not used by default) 2016-05-19 01:59:00 +02:00
reger
27a238062f fix MessageSend servlet response if called for passive peer 2016-05-17 02:08:07 +02:00
reger
4e0892962a fix NPE in citation servlet on empty text field 2016-05-14 03:51:13 +02:00
reger
efb9f1a8b7 save resource for unused blacklistFiles map 2016-05-12 00:13:57 +02:00
otter
05288557c1 optimized for small y values 2016-05-08 20:17:18 +02:00
otter
1afefebd0f manually pulled - now pushed 2016-05-08 16:38:17 +02:00
reger
8c9684cc45 optimize surftip data load, 
double load (index, loader) not neccessary, getMetadata already suficient
+ lng file adjustments
 2016-05-08 05:27:19 +02:00
Andreas
91e1e5640b Make sure "leftscale" is positive 2016-05-03 15:00:10 +02:00
reger
f03656394e skip caching of knownURL during transferRWI 
(cache values not used)
 2016-04-28 02:07:15 +02:00
reger
3bd6ae8d8b keep addon/Notepad++ keyword marker on lng export 
(length of remarks devider line)
+ harmonize status_p.inc lng text
 2016-04-21 00:51:08 +02:00
reger
7be1c7a05a fix logger name 2016-04-17 03:20:14 +02:00
reger
ba5161966f support xliff for downloaded of language file 2016-04-10 05:33:21 +02:00
reger
61c5b6b403 fix empty drop down list in ConfigLanguage after wrong/empty download 
+ add xliff translated attribut
+ append japanese lng name
 2016-03-31 01:51:25 +02:00
reger
a28f5d07e8 use UTF8 to write imported language files 2016-03-28 05:31:17 +02:00
reger
d9adc2c255 load handler for Transparent Proxy on startup only if feature is activated 
to save the resources and keep handler chain small if the feature is not used.
+add a warning message on settingsack_p page to restart on first activation
 2016-03-25 05:26:48 +01:00
Andreas
898f6c5891 Merge pull request #4 from yacy/master 
Get my fork synced #4
 2016-03-19 13:13:04 +01:00
Bryan Black
4d781c660d Distilled sentence - ConfigNetwork_p 
I distilled the first "Distributed Computing Network for Domain" description sentence and corrected grammar.
 2016-03-18 22:29:44 -07:00
Bryan Black
94ce617188 Grammar fix 
The sentence sounded odd.
 2016-03-18 21:40:11 -07:00
reger
a57226caa6 put settings_p servlet (back) as start page for System Administration 
to display the options available (as tables_p only used for indepth edit)
see http://mantis.tokeek.de/view.php?id=460
 2016-03-12 01:15:05 +01:00
Michael Peter Christen
b89465d952 0N - basic dump upload servlet infrastructure, to share index dumps 
within an experimental new sharing model
 2016-03-11 18:12:13 +01:00
Michael Peter Christen
f12a900f3e harmonization of http post of files for one and several files - this had 
been differently - and wrong for several files. also: base64-encoding
for gzipped push files because our data structures currently only
supports ASCII POST pushes..
 2016-03-11 08:59:33 +01:00
Michael Peter Christen
2494a820c7 0N - added recording of dump exports if given time frame is not negative 2016-02-24 15:13:20 +01:00
Michael Peter Christen
ef2cc4f690 Merge branch 'master' of git@github.com:yacy/yacy_search_server.git 2016-02-24 11:19:32 +01:00
Michael Peter Christen
a6bf0b1649 0N - added option to generate index export files for a specific number 
of minutes in the past and reverted latest change. The export file dump
will now contain four data elements: f - first date of index entry write
date, l - last date of index write date, n - now-date of index dump
time, c - count of numbers inside the dump. '0N' denotes a series of
changes which will lead to the opportunity to exchange index data dumps
in a way that is needed to integrate ZeroNet index data. This will be
based on index dump sharing; that causes this commit.
 2016-02-23 18:56:20 +01:00
reger
9312fbe563 making WebStructurePicture_p less vulnerable to faulty host input parameter (like host1,,host3) 
by continue host loop on exception

inspired by http://mantis.tokeek.de/view.php?id=637
 2016-02-21 21:38:11 +01:00
reger
b12b8fb1c2 include initial japaneese translation to language selection 2016-02-20 23:17:59 +01:00
Michael Peter Christen
5b9030180c added peer hash to export dump name. 2016-02-19 19:26:02 +01:00
Michael Peter Christen
287b918bd7 Merge branch 'master' of git@github.com:yacy/yacy_search_server.git 2016-02-19 07:52:59 +01:00
reger
dbb28bb4f3 del unused statistic parameter (from status servlet) 2016-02-17 22:47:03 +01:00
Michael Peter Christen
b851308ee6 enhanced robustnes of image computation 2016-02-16 17:36:49 +01:00
reger
06d0e2aeb9 result heuristic (also used in greedy learning mode) to use outbound links if result is full index doc. Otherwise use default loader methode. 
- Above brought up that parser start url parameter, declared as AnchorURL uses only methodes of parent object DigestURL (changed parameter declaration accordingly).
 2016-02-16 02:05:58 +01:00
luc
9f712146df Display icons in ViewFile "links" mode. 2016-02-10 10:08:07 +01:00
luc
8682dfbd5e Updated getpageinfo outputs to return page icons list. 2016-02-10 09:02:21 +01:00
luc
26f1ead57c Created ViewFavicon class specialized in favicon viewing. 
Main image processing is now in ImageViewer, used by both ViewImage and
ViewFavicon.

Fixed URIMetadataNode.getFavicon to use non-standard icons with no size
ass fallback.
 2016-02-09 20:46:44 +01:00
otter
f2e5b3adb7 format2 2016-02-08 17:47:26 +01:00
otter
000ec16bf8 format 2016-02-08 17:44:26 +01:00
otter
c3c5e7928b Correctly handle POSTed parameter also with HTTPS activated 2016-02-06 20:22:17 +01:00
luc
07222b3e1a Added favicon url transmission in RWI chunks. 2016-02-05 17:05:36 +01:00
luc
480772c070 Fixed json search results from commit "Improved URLLicence reliability" 2016-02-05 15:23:29 +01:00
luc
53781299d8 Extracted intranet and filtype related rules from getFaviconURL func 2016-02-04 08:14:49 +01:00
reger
cc79ad8de6 compare search page, remove diminished search target 
(romso.de, dbpedia.neofonie.de )
 2016-02-04 00:47:42 +01:00
luc
3cc5619d93 Improved HTML icons indexing and rendering in search results. 
See http://mantis.tokeek.de/view.php?id=629
 2016-02-02 09:57:54 +01:00
otter
f6e6250b83 Merge branch 'master' of https://github.com/otteresk/yacy_search_server.git 2016-02-01 21:06:00 +01:00
otter
770bb1d41f Improved plotting 2016-02-01 21:04:38 +01:00
luc
db6fc67b77 Merge branch 'master' of https://github.com/yacy/yacy_search_server 2016-02-01 08:32:01 +01:00
reger
8e60788c8f fix json date facet displayname 2016-01-31 02:38:39 +01:00
luc
ef83e34b8a Merge branch 'master' of https://github.com/yacy/yacy_search_server 2016-01-19 08:06:49 +01:00
Michael Peter Christen
5d635879f8 Merge pull request #40 from Scarfmonster/autocrawl 
Automatic crawling
 2016-01-14 22:19:55 +01:00
Ryszard Goń
7d6e0d8470 Add missing settings to autocrawl settings page 2016-01-14 03:27:33 +01:00
Ryszard Goń
7a7a1277bd Add autocrawl settings page 2016-01-14 02:40:46 +01:00
reger
4765e374e6 altered clac. of search result items per page to display 
taking the existing limits into account but make it consistent with search option screen for admin and public user
changes:
  - configured default number of items per page (ConfigPortal_p.html) is used as is (no hardcoded limit)
  - otherwise requests are limited to 100 results per page ( = search option, index.html)
      (this basically is the major change, inc. limit from 20 to 100 for public user)
P.S. - the older grant of more (1000), if no online snippet calculation, is kept (for the time being)

see http://mantis.tokeek.de/view.php?id=627
 2016-01-13 01:30:49 +01:00
luc
231be83eb6 Corrected access to Load_MediawikiWiki.html and Load_PHPBB3.html 
A NullPointerException occured when trying to access theses pages in
Robinson (Search portal) mode
 2016-01-12 22:09:30 +01:00
luc
85a9363012 Merge branch 'master' of https://github.com/yacy/yacy_search_server 2016-01-12 09:32:22 +01:00
reger
abd8ecb503 remove contendom depending override of search result items per page 
initially introduced e4570bffaf (diff-ae6c130fc11088c830b00ed9256ab56b)
(as one part of unexpected difference in actual vs requested results, partial bugfix for http://mantis.tokeek.de/view.php?id=627 )
 2016-01-12 01:04:10 +01:00
luc
41767a01c2 Merge branch 'master' of https://github.com/yacy/yacy_search_server 2016-01-11 23:08:22 +01:00
reger
04161912a5 fix tray icon switch 
(using predefined/correct config name)
 2016-01-09 01:19:06 +01:00
luc
7aa1a29e33 Return more accurate HTTP status 400 with detail message when some error 
occurs on ViewImage :
 - missing required parameters
 - url licence invalid
 2016-01-08 23:18:13 +01:00
luc
cfdbc2b487 Improved URLLicence reliability for use by conccurrent non authaurized 
users.
Removed URLLicence generation when unnecessary (authorized users)
 2016-01-08 20:42:57 +01:00
Michael Peter Christen
9a25751850 Merge pull request #38 from luccioman/master 
Refactoring : use StandardCharsets instead of hardcoded charset names
 2016-01-06 14:55:54 +01:00
reger
c283efdd6d remove obsolete css style for removed file CacheAdmin_p.html 
and remove from translations
 2016-01-06 00:51:49 +01:00
luc
571bc55937 Refactoring : use StandardCharsets constants instead of hard-coded 
charset names.
 2016-01-05 23:37:05 +01:00
reger
e8256bb3b1 remove blekko from opensearch config (not available) 
see https://blekko.com/
http://searchengineland.com/goodbye-blekko-search-engine-joins-ibms-watson-team-217633
 2016-01-04 04:49:10 +01:00
reger
9636a74633 remove local credential on download of config files from remote systems 
(blacklists, language, skins) to reduce risk to expose md5-pwd.
- remove NoSuchElementException in loop
 2016-01-01 06:08:41 +01:00
reger
1636541c48 correct filename input in settings_seed_upload.inc form 
to get filename
(recently introduced by change form "text" to the more convenient "file" input  50f64ddc3b)
 2015-12-30 21:27:44 +01:00
reger
cd26717ba2 fix low memory status hint (dht-in disabled) 
http://mantis.tokeek.de/view.php?id=619
 2015-12-29 20:38:45 +01:00
sixcooler
775e74b055 Merge branch 'master' of https://github.com/yacy/yacy_search_server 2015-12-28 23:23:37 +01:00
sixcooler
dce1cb65c4 Merge remote-tracking branch 'choose_remote_name/master' 2015-12-28 23:20:42 +01:00
Michael Peter Christen
fe308f47d5 added greeting line to interactive search and harmonized display 
position with 'Administration' line in all administration pages
 2015-12-28 23:20:37 +01:00
Michael Peter Christen
a7b41bd206 use curl downloads in download script with silent mode 2015-12-28 22:57:31 +01:00
Michael Peter Christen
7274e50d7f Merge branch 'master' of git@github.com:yacy/yacy_search_server.git 2015-12-26 22:25:15 +01:00
reger
50f64ddc3b apply default css styles (class btn) to submit buttons 2015-12-25 01:08:51 +01:00
reger
09d3dd13d6 limit bookmark tag cloud font-size to 2.0em 2015-12-24 19:39:37 +01:00
reger
3076c87247 fix typo in Steering.html 2015-12-24 04:37:24 +01:00
reger
80e2c82249 fix NPE on empty blog importfile parameter 2015-12-24 02:00:45 +01:00
reger
8a8e53b1a1 apply default css styles (class btn) to blacklist* submit buttons 2015-12-23 23:11:37 +01:00
reger
0fa6340936 apply more default css styles (class btn) to submit buttons 2015-12-23 00:04:40 +01:00
reger
bf098412cf use input type=file for choosing IndexImportMediawiki_p dump file 2015-12-22 20:15:08 +01:00
reger
7c6d6cd69a change some more submit button to bootstrap btn css class 2015-12-22 01:36:23 +01:00
reger
4eb7fb0ee5 just remove debug leftover 2015-12-21 19:47:40 +01:00
reger
45b9bd8403 adjust MultiProtocolURL.protocol detection to handle mailto with "://" in parameters, 
and feeding hyperlinks to webgraph processing.
 2015-12-21 04:42:26 +01:00
reger
67f64af4b4 quick fix: go back to display search results favicon via <img> tag and ViewImage, 
! until better solution is found !.

Reason: in IE-Browser no individual favicon is displayed with <object> tag 
(always the default) and only few individual fav's with Firefox (randomly)

hint: to be able to use <img src=ViewImage/> return of default icon was
added back to it.

relates to http://mantis.tokeek.de/view.php?id=629
 2015-12-21 01:05:59 +01:00
reger
dec3e6ad96 fix: adjust urlstub for mailto links 
(skip protocol)
 2015-12-19 20:13:33 +01:00
reger
0c5548a7ff fix (todo) remove redundant holding of email link nameproperty in parser document 2015-12-18 02:35:44 +01:00
reger
71c416f383 show mailto links in ViewFile.html linklist 2015-12-18 01:11:55 +01:00
luc
befb2415f8 Corrected frames preview displaying eventually incorrectly in local 
administration mode.
 2015-12-16 02:23:58 +01:00
luc
8c4ab9c76b Added an option to eventually limit size of remote solr documents put to 
local index. See mantis #626.
 2015-12-16 02:20:03 +01:00
luc
a2c08402af Merge branch 'master' of https://github.com/yacy/yacy_search_server 2015-12-15 23:30:30 +01:00
luc
55a4d15775 Added a note on deprecated default search field and operator. 2015-12-14 23:55:12 +01:00
sixcooler
1be67d9ab6 CachedSolrConnector was replaced by ConcurrentUpdateSolrConnector years 
ago - time to let it go
Commented out unused table of cache-objects
 2015-12-14 21:33:27 +01:00
reger
28b8bc290a fix use of NETWORK_SEARCHVERIFY for rwi verification 
was not used to set the searchevent parameter (done in SearchEventCache.getEvent)
- remove unused corresponding QueryParams.filterfailurls param.
 2015-12-13 20:01:49 +01:00
reger
020630efd8 remove unused network scanner parameter from queryparameter 
Search event is not using networkscanner 
(removed filterscannerfail param always init to false)
 2015-12-13 02:50:08 +01:00
Michael Peter Christen
967508a87d fix in error handling 2015-12-09 01:21:00 +01:00
reger
7cda48a9d6 add hint to "default max results per page" limit on ConfigPortal 
(limit is applied in yacysearch & max. total results by sum result-stack size)
- remove obsolete search.navigation prop (has moved to ConfigSearchPage_p)
 2015-12-09 00:49:38 +01:00
reger
a622c9b656 upd to Bootstrap v3.3.6 2015-12-07 23:03:59 +01:00
reger
1043fe55a3 add missing bootstrap 3.3.4 glyphicons file 
see comment @luc http://mantis.tokeek.de/view.php?id=623#c1151
 2015-12-07 01:30:00 +01:00
luc
29585e2c5b Corrected return type when licence is gone to be consistent with other 
error cases.
 2015-12-01 09:55:47 +01:00
luc
df77e90ed7 Merge branch 'master' of https://github.com/yacy/yacy_search_server 2015-12-01 09:13:16 +01:00
luc
2a67d2ba6f Corrected error management for unsupported image formats, parsing 
errors, and unavailable resources : avoid logging to much Exceptions as
these errors easily occur when searching images.
 2015-12-01 01:06:01 +01:00
reger
997f18f658 prevent exception on repeated ViewImage with same urlLicense 2015-12-01 00:06:50 +01:00
Michael Peter Christen
7e785dac8e urlproxyheader must be in the default package because all classes in the 
htroot path must be in the default package
 2015-11-30 13:35:41 +01:00
Michael Peter Christen
d82d311995 Merge branch 'master' of https://github.com/luccioman/yacy_search_server 
# Conflicts:
#	.classpath
 2015-11-30 13:34:10 +01:00
reger
1160b13172 remove unused md5 from ViewFile servlet params 2015-11-28 23:09:15 +01:00
luc
e40ae0943b - No max dimensions specified : render raw image data when source and 
target image format are the same.
- Corrected scaling condition.
 2015-11-26 09:30:43 +01:00
luc
5eafce5577 Rendering performance improvement : use EncodedImage constructor with 
BufferedImage parameter to avoid re-rerendering BufferedImage.
 2015-11-20 15:02:58 +01:00
luc
bc6c79fc12 Corrected scaling function for non RGB images. 2015-11-20 14:35:36 +01:00
luc
7f27683831 Correction erreur de compilation. 2015-11-20 09:29:02 +01:00
luc
07437986e7 Merge branch 'master' of https://github.com/yacy/yacy_search_server 2015-11-20 08:15:24 +01:00
reger
97cc03ef6a start using a template for urlproxy header 
It is included as iframe  /proxmsg/urlproxyheader.html
to allow full servlet functionallity and flexibility to display some
index/meta data in future.
 2015-11-20 01:49:56 +01:00
reger
d08e421809 fix link to logo (yacysearch.xsl) 2015-11-19 21:08:00 +01:00
luc
f01d49c37a Process large or local file images dealing directly with content 
InputStream.
 2015-11-18 10:15:38 +01:00
luc
bfe51001e3 Merge branch 'master' of https://github.com/yacy/yacy_search_server 2015-11-17 08:30:32 +01:00
reger
7bf03856d1 add link to quick select blacklist 
from title list
 2015-11-15 00:39:38 +01:00
reger
b4dbff6a6a fix yacysearch.json "totalResults" 
element "totalResults" is included twice (at begin & end), 
only the element after performing the search holds number > 0
see http://mantis.tokeek.de/view.php?id=608
 2015-11-13 20:10:47 +01:00
reger
52a9040ae6 Sort out double keywords (dc_subject) early in parsed documents 
- by direct using Set vs. List
- remove not neede String[] getter
 2015-11-13 01:48:28 +01:00
luc
0e8b3d9a90 Refactoring : default favicon and image processing errors. 
- moved default favicon processing from ViewImage to
yacysearchitem.html : when previewing ico image search results we don't
want a default favicon be displayed
 - throw an IOException ending in a HTTP 500 error when image processing
fails, rather than returning a null result : behavior is more consistent
accross browsers (for exempla Chrome and Firefox), especially with new
default favicon display system
 2015-11-05 09:45:19 +01:00
luc
745e97a575 Merge branch 'master' of https://github.com/yacy/yacy_search_server 2015-11-02 08:10:11 +01:00
reger
a60b1fb6c2 differentiate api call getLocalPort() from getConfigInt() 2015-10-31 23:09:03 +01:00
reger
02afba730e fix detection of https port changed after set in System Admin 2015-10-31 22:53:59 +01:00
luc
e90e1f165d Avoid returning an empty image when target encoding is not supported or 
when an error occured while encoding.
 2015-10-30 16:20:28 +01:00
luc
aa70ff4ff6 Corrected images alpha channel rendering 2015-10-30 05:18:16 +01:00
reger
a2dcf64039 fix IndexImportMediawiki_p servlet's refresh header 
add url parameter to make sure no parameter are included in refresh url 
which could cause unwanted restart of import job

see http://mantis.tokeek.de/view.php?id=591 comments
 2015-10-25 05:41:25 +01:00
Michael Peter Christen
ac034db8bc Merge branch 'master' of https://github.com/luccioman/yacy_search_server 
# Conflicts:
#	htroot/js/highslide/highslide.js
#	source/net/yacy/document/ImageParser.java
 2015-10-24 11:22:35 +08:00
luc
a156fd65d0 Patch to manage render or load errors is still needed after highlight.js 
version upgrade.
Updated patch for better behavior consistency between browsers.
 2015-10-22 00:36:34 +02:00
luc
37e28e0dd3 - Keep aspect ratio of images rendered directly by browser such as gif 
and svg.
- Corrected quadratic rendering of landscape images with height smaller
than maxHeight
 2015-10-21 02:49:51 +02:00
reger
571609c208 upd javascript img viewerto highslide 4.1.13 2015-10-21 02:14:04 +02:00
luc
e2d00585e2 Display full size preview using ViewImage Servlet. 2015-10-20 01:17:37 +02:00
luc
74b0283d57 Added image preview error management. 2015-10-20 01:15:02 +02:00
luc
d6522fa4a2 Integrated haraldk/TwelveMonkeys library to first add TIF image format 
support.
 2015-10-15 10:06:51 +02:00
luc
62e07a26a0 Refactoring : split into sub-functions to make it understanding and 
performance measurement easier.
 2015-10-14 10:15:00 +02:00
reger
c9937973e3 unescape MultiProtocolURL getAttributes() return values. 
use getAttributes() to get query parameters as clear text (w/o url encoding)
use getSearchpartMap() to get in internal format (url encoded)

fix for http://mantis.tokeek.de/view.php?id=606
 2015-10-13 02:43:18 +02:00
reger
10b0eb106f fix link target on iframe list in CrawlProfileEditor 2015-10-11 06:06:40 +02:00
reger
5744342fec handle image preview for url w empty file extension 
fix of commit 688f7b2a5c
 2015-10-06 04:13:04 +02:00
reger
43c27aa550 upd to solr/lucene 5.3.1 2015-10-03 23:20:33 +02:00
reger
688f7b2a5c allow/display svg images in image results previews 
svg is not supported by awt but by most browser. Image content is delivered as received (without size adjustment)
 2015-10-02 01:48:48 +02:00
Michael Peter Christen
225200194a every time a crawl is started, the user expects a different search 
result behaviour. This requires that the search cache is flushed for
each crawl start. TODO: this should also be done if a crawl is
terminated.
 2015-10-01 13:18:44 +02:00
reger
b92d81b073 remove double caching of inputstream in ViewImage 2015-09-27 03:24:28 +02:00
Michael Peter Christen
3c31bf845f fix for latest merge 2015-09-24 13:53:54 +02:00
luc
5578886f6f Merge branch 'master' of https://github.com/luccioman/yacy_search_server.git 2015-09-23 21:04:20 +02:00
reger
2951c9fc40 remove unused check for known fileextension in searchtrailer 
(check is done on add to filetype-nav)
 2015-09-22 03:52:15 +02:00
reger
733d725dec limit css scrolling to result/content window x 
from pull request #10
 2015-09-15 02:11:30 +02:00
Burkhard
4c38083a11 Merge pull request #10 from Raegdan/raegdan-css-layout-fix 
Fixed CSS scrolling
 2015-09-15 02:09:17 +02:00
luccioman
a7179138ce Returned again to main repository location : does anyone want to 
consider mantis 597 ?  (http://mantis.tokeek.de/view.php?id=597)
 2015-09-11 17:23:59 +02:00
luccioman
199b2ce52d Translator refactoring : to simplify locale files writing, process keys 
as simple string and no more as regular expressions.
Updated all locale files to adapt to refectored Translator : removed
useless escaped characters and did minor corrections.
Performed minor syntax corrections on some html source files.
Added an util to translate all html source files with all locales
without launching full YaCy application.
Corrected main arguments parsing on other translation utils.
 2015-09-11 17:20:11 +02:00
luccioman
4dd9c0d5d9 Merge from main repository 2015-09-08 08:54:48 +02:00
Michael Peter Christen
0a37d8af89 in case that a site crawl is started for urls with file:// path, the 
host filter does not work because there is no host given in such urls.
In that case, patch the filter to be a sub-path filter.
 2015-09-05 14:07:23 +02:00
luccioman
9df249296a Return to mai repository version 2015-09-04 13:52:03 +02:00
luccioman
c1d937a90c Merge branch 'master' of ssh://git@github.com/yacy/yacy_search_server 2015-09-04 09:57:49 +02:00
reger
7c1da173e0 fix missing license in image search 
see http://mantis.tokeek.de/view.php?id=522
 2015-09-03 23:36:57 +02:00
luccioman
918ef72bbe Corrected br markup 2015-09-03 08:59:17 +02:00
luccioman
f88bb2277e Corrected bookmark link title 2015-09-03 08:58:14 +02:00
luccioman
802ea66d19 Merge branch 'master' of ssh://git@github.com/yacy/yacy_search_server 2015-09-03 08:04:38 +02:00
reger
5297e80cda fix missing onclick in ConfigPortal 
to enable checkbox
 2015-09-03 00:59:14 +02:00
luccioman
70e483ecc6 Merge branch 'master' of ssh://git@github.com/yacy/yacy_search_server 2015-09-01 08:57:32 +02:00
sixcooler
87e4abe393 fight the fieldcache by usind DocValues: in Solr-5.x the fieldcache has 
moved and was not cleared anymore. This results in an huge fieldcache.
(http://lucene.apache.org/#highlights-of-the-lucene-release-include
https://issues.apache.org/jira/browse/LUCENE-5666)
Here I try to use DovValues where it is possible.
For this I used the Api-Scheme as new basis für the Solr-Schema.
This needs at least a complete optimization of the Solr-Index to get a
smaller FieldCache.
Everything that is indexed with these setting will not use the
Fieldcache at all.
 2015-08-31 20:24:41 +02:00
luccioman
67799ce867 Updated translation of index.html, yacysearch.html and 
simpleheader.template, corrected some special characters not written as
HTML entities.
 2015-08-26 14:40:39 +02:00
Michael Peter Christen
df3314ac1a added a new facet type based on a probabilistic classifier using 
bayesian filters. This can be used to classify documents during
indexing-time using a pre-definied bayesian filter.

New wordings:
- a context is a class where different categories are possible. The
context name is equal to a facet name.
- a category is a facet type within a facet navigation. Each context
must have several categories, at least one custom name (things you want
to discover) and one with the exact name "negative".

To use this, you must do:
- for each context, you must create a directory within
DATA/CLASSIFICATION with the name of the context (the facet name)
- within each context directory, you must create text files with one
document each per line for every categroy. One of these categories MUST
have the name 'negative.txt'.

Then, each new document is classified to match within one of the given
categories for each context.
 2015-08-10 14:27:44 +02:00
Michael Peter Christen
dbbad23e12 removed warnings 2015-08-03 05:37:34 +02:00
reger
9e4043731d add missing ; in base.css 2015-08-02 21:36:44 +02:00
Michael Peter Christen
de8cfbe1d7 added export option to export the fulltext of the search index text only 2015-07-30 03:21:40 +02:00
Kirill Fomchenko
ab22a32c09 Fixed CSS scrolling 
When the sidebar on search page becomes scrollable, the scrollbar shrinks the sidebar and makes the search results weirdly scrollable on X axis by several pixels. Now the sidebar always have a scrollbar, and results are never X-scrollable.
 2015-07-21 08:21:10 +03:00
Michael Peter Christen
785781253e added jsonp to suggest servlet 2015-07-16 23:42:41 +02:00
reger
821262a179 add CommonPattern for multiple spaces 
to eliminate empty split words on following spaces
 2015-07-04 22:49:01 +02:00
Michael Peter Christen
f901e7d3cf fix for non-authorized view of IndexBrowser: show only the number of 
non-failure documents
 2015-06-30 11:12:36 +02:00
Michael Peter Christen
3c4c69adea fix for 
- bad regex computation for crawl start from file (limitation on domain
did not work)
- servlet error when starting crawl from a large list of urls
 2015-06-29 02:02:01 +02:00
Michael Peter Christen
1fec7fb3c1 suppress access to solr when doing search suggestions in case that the 
index has more than two million documents. This protects the index from
beeing flooded with search requests that cannot be resolved before the
real search query has to be computet.
 2015-06-24 13:02:12 +02:00
Michael Peter Christen
886fca2260 Merge branch 'master' of git@github.com:yacy/yacy_search_server.git 2015-06-24 01:59:46 +02:00
Michael Peter Christen
694b22f165 migration to Solr 5.2: huge benefits - this is a lot faster! 
This is a very complex migration: many classes had been renamed or
removed, dependencies changed and the solr index type is now aligned to
be a solr cloud repository.
Together with the Solr 5.2 library update, one other dependent library
had been updated as well: httpclient 4.4->4.4.1

Older indexes are migrated from 4_10 to 5_2. However, the new index
structure is more efficient and we recommend to re-index everything.
Please use the index export before you do the update to a large
surrogate xml file. After the update, start with an empty index and then
initialize this with your dump.
 2015-06-24 01:55:51 +02:00
Michael Peter Christen
6c2e6f1f37 remove redundant code 2015-06-23 23:41:43 +02:00
Michael Peter Christen
9c12555be5 added link to Snapshots in search results if the snapshot exists and 
option is set in ConfigSearchPage_p
(this is a stub: we also need a visualization of pdf files!)
 2015-06-07 20:37:37 +02:00
reger
72f6a0b0b2 enhance recrawl job 
- allow to modify the query to select documents to  process (after job has started)
- allow to include failed urls (httpstatus <> 200)
 2015-06-06 18:45:39 +02:00
Michael Peter Christen
e0a23c56c7 Merge branch 'master' of git@github.com:yacy/yacy_search_server.git 2015-06-05 08:32:55 +02:00
Michael Peter Christen
fb9e1dd3f5 servlet for latest commit 2015-06-05 07:22:35 +02:00
reger
7478338a40 remove augmented parsing activation from frontend 
experimental implementation not used and based on error prone experimental rdfaparser
 2015-06-05 00:51:00 +02:00
reger
11aa2edfe1 remove RDFa parser activation from frontend 
reason: experimental implementatin of RDFa parser not executed (limited to special urls) but may cause error on normal html parsing due to a inputstream.reset
 2015-06-05 00:15:16 +02:00
Michael Peter Christen
ff11ac89f7 Merge branch 'master' of git@github.com:yacy/yacy_search_server.git 2015-06-04 23:04:04 +02:00
Michael Peter Christen
5e2d23b7a0 removed the new index export method from the IndexControlURLs_p.html 
servlet and moved it to a new /IndexExport_p.html servlet. This servlet
is now more prominent linked in the main menu under Production -> Index
Export/Import
 2015-06-04 23:03:46 +02:00
reger
49b79987c9 remove obsolete searchfl work table 
was used to register urls with not complete words in snippet but is never accessed
 2015-06-04 22:44:01 +02:00
Michael Peter Christen
b43811d38c added surrogate import process for exported solr dumps. 
Just throw your solr dump file into DATA/SURROGATES/in/ and it will be
imported!
 2015-05-30 13:19:59 +02:00
Michael Peter Christen
eec78e1b0c added intensity option to graphics 2015-05-30 06:31:08 +02:00
Michael Peter Christen
c7576d6028 added a full solr export to the IndexControlURLs_p.html servlet. The 
export function is also now the default export option. The export file
format for a full solr export is very similar to a solr search result
xml, only the <lst name="responseHeader"> tag is missing.

The exported xml has a special line termination feature: all documents
will be exported into a single line without any CR in between. That
means that every document is completely inside a single line. While this
is not readable at all for humans, it is very useful for linux line
processing scripts, like grep. Using grep it will be easy to select
single documents which match for a given pattern.

Such dumps shall be importable with the DATA/SURROGATE/in import
function, but that import is not yet adopted to the new file format.
 2015-05-29 15:05:52 +02:00
Michael Peter Christen
47682bf467 fix for unresolved pattern 2015-05-28 17:43:52 +02:00
Michael Peter Christen
197f7449e5 All entities of crawl profiles are now editable in the crawl profile 
editor.
 2015-05-28 16:07:40 +02:00
reger
1d8e1e4bac - Image search expand box, adjust javascript hs padtominsize parameter, to make sure expand box doesn't shrink on small images 
- asure ImageResult.imagetext has value for the link text (use filename if no alt text given)
 2015-05-27 02:31:13 +02:00
reger
000dde9511 Eleminate duplication of values for search ResultEntry 
by instatiation from URIMetadataNode, by eleminating differentiation of ResultEntry/URIMetadataNode.
- moved remaining ResultEntry functionallity to URIMetadataNode
   - for 1:1 functionallity added a function makeResultEntry() 
- removed ResultEntry 
- refactored related code

Main difference is after makeResultEntry the text_t content is removed and alternative title/url strings for display are calculated.


Main difference left is, that
 2015-05-26 04:15:00 +02:00
reger
3d53da8236 refactor ResultEntry to be based on MetadataNode/SolrDocument 
to share/reuse common access routines
 2015-05-25 21:28:48 +02:00
reger
17e820cfd7 use doctype() in ViewFile to choose display routines 
in preference of getfileExtension()
 2015-05-25 00:08:38 +02:00
reger
aa83931765 Convert content charset for display via CacheResource_p 
Cached resource charset encoding might not fit to internal handling (using utf-8),
convert resource to utf-8
see http://mantis.tokeek.de/view.php?id=576
 2015-05-23 20:31:37 +02:00
reger
3e742d1e34 Init remote crawler on demand 
If remote crawl option is not activated, skip init of remoteCrawlJob to save the resources of queue and ideling thread.
Deploy of the remoteCrawlJob deferred on activation of the option.
 2015-05-23 02:06:39 +02:00
Michael Peter Christen
dbf9e3503d Merge branch 'master' of git@github.com:yacy/yacy_search_server.git 2015-05-22 11:39:00 +02:00
Michael Peter Christen
8b1a30be50 removed a -UNRESOLVED_PATTERN- 2015-05-22 11:22:36 +02:00
Michael Peter Christen
9938c81378 fix for division by zero 2015-05-22 11:15:53 +02:00
reger
ace71a8877 Initial (experimental) implementation of index update/re-crawl job 
added to IndexReIndexMonitor_p.html
Selects existing documents from index and feeds it to the crawler.
currently only the field fresh_date_dt is used determine documents for recrawl (fresh_date_dt:[* TO NOW-1DAY]
Documents are  added in small chunks (200) to the crawler, only if no other crawl is running.
 2015-05-16 01:23:08 +02:00
Michael Peter Christen
f810915717 added crawl start from a clone with very, very large url: they are now 
encoded as post submit form inside a javascript creation function.
 2015-05-11 16:30:41 +02:00
reger
609c52e987 refactor getBookmark 
to consistenly check existance by != null (w/o throwing exception on not found)
 2015-05-11 00:37:04 +02:00