Commit Graph

237 Commits

Author SHA1 Message Date
luccioman
db3b9db9c2 Crawl from local file : faster task end when manually terminating crawl. 2016-10-22 09:11:20 +02:00
luccioman
6e1959f469 Merge branch 'master' of https://github.com/yacy/yacy_search_server.git
Conflicts:
	htroot/yacysearchitem.java
	source/net/yacy/cora/federate/solr/responsewriter/YJsonResponseWriter.java
	source/net/yacy/search/schema/CollectionConfiguration.java
	source/net/yacy/server/serverObjects.java
2016-10-14 11:29:55 +02:00
luccioman
75bb77f0cb Refactoring : extracted a method to handle authorized action links. 2016-10-12 09:31:42 +02:00
luccioman
c996b04741 HTML validation : fixed URL encoding of search results action links. 2016-10-12 09:16:47 +02:00
luccioman
2b81703828 Refactored search result action links construction.
These are long URLS with common parts : it is valuable to build the
common parts only one time.
2016-10-12 08:45:32 +02:00
luccioman
6e96c7341a Merge remote-tracking branch 'origin/master'
Conflicts:
	htroot/Load_MediawikiWiki.java
	htroot/Load_PHPBB3.java
	htroot/ViewImage.java
2016-07-03 18:59:00 +02:00
reger
06d0e2aeb9 result heuristic (also used in greedy learning mode) to use outbound links if result is full index doc. Otherwise use default loader methode.
- Above brought up that parser start url parameter, declared as AnchorURL uses only methodes of parent object DigestURL (changed parameter declaration accordingly).
2016-02-16 02:05:58 +01:00
luc
26f1ead57c Created ViewFavicon class specialized in favicon viewing.
Main image processing is now in ImageViewer, used by both ViewImage and
ViewFavicon.

Fixed URIMetadataNode.getFavicon to use non-standard icons with no size
ass fallback.
2016-02-09 20:46:44 +01:00
luc
07222b3e1a Added favicon url transmission in RWI chunks. 2016-02-05 17:05:36 +01:00
luc
53781299d8 Extracted intranet and filtype related rules from getFaviconURL func 2016-02-04 08:14:49 +01:00
luc
3cc5619d93 Improved HTML icons indexing and rendering in search results.
See http://mantis.tokeek.de/view.php?id=629
2016-02-02 09:57:54 +01:00
luc
cfdbc2b487 Improved URLLicence reliability for use by conccurrent non authaurized
users.
Removed URLLicence generation when unnecessary (authorized users)
2016-01-08 20:42:57 +01:00
luc
37e28e0dd3 - Keep aspect ratio of images rendered directly by browser such as gif
and svg.
- Corrected quadratic rendering of landscape images with height smaller
than maxHeight
2015-10-21 02:49:51 +02:00
luc
e2d00585e2 Display full size preview using ViewImage Servlet. 2015-10-20 01:17:37 +02:00
reger
5744342fec handle image preview for url w empty file extension
fix of commit 688f7b2a5c
2015-10-06 04:13:04 +02:00
reger
688f7b2a5c allow/display svg images in image results previews
svg is not supported by awt but by most browser. Image content is delivered as received (without size adjustment)
2015-10-02 01:48:48 +02:00
reger
7c1da173e0 fix missing license in image search
see http://mantis.tokeek.de/view.php?id=522
2015-09-03 23:36:57 +02:00
Michael Peter Christen
9c12555be5 added link to Snapshots in search results if the snapshot exists and
option is set in ConfigSearchPage_p
(this is a stub: we also need a visualization of pdf files!)
2015-06-07 20:37:37 +02:00
reger
000dde9511 Eleminate duplication of values for search ResultEntry
by instatiation from URIMetadataNode, by eleminating differentiation of ResultEntry/URIMetadataNode.
- moved remaining ResultEntry functionallity to URIMetadataNode
   - for 1:1 functionallity added a function makeResultEntry() 
- removed ResultEntry 
- refactored related code

Main difference is after makeResultEntry the text_t content is removed and alternative title/url strings for display are calculated.


Main difference left is, that
2015-05-26 04:15:00 +02:00
reger
3d53da8236 refactor ResultEntry to be based on MetadataNode/SolrDocument
to share/reuse common access routines
2015-05-25 21:28:48 +02:00
reger
609c52e987 refactor getBookmark
to consistenly check existance by != null (w/o throwing exception on not found)
2015-05-11 00:37:04 +02:00
reger
4c907bec89 show "Augmented Browsing" link in search result only if urlproxy allowed and option switched on in layout
(AugmentedBrowsing_p.html, ConfigSearchPage_p.html)
as user only gets a error page if the option is not enabled
2015-04-14 02:07:02 +02:00
Michael Peter Christen
fd4e2c809a Show dates in the content of a document in the search result:
- if an eventDate is given in the search result, replace the document
date with the event date and prefix it with the string "on ".
- the document date is omitted if a date from the cent is shown

Added also the date as fields in the json and rss result sets.
2015-03-02 18:00:20 +01:00
reger
1196ff01c8 revert: formatting fix eats also up highlighting
need other solution for snippets with unwanted html code
2015-02-14 02:43:05 +01:00
reger
61f42a7928 fix formatting issue in search result display
if description contains html code
noticed e.g. for id=NmNdJ9uApLaQ  http://hswong3i.net/blog/hswong3i/virtualmin-drupal-7-x-ubuntu-12-04-howto
2015-02-13 00:20:33 +01:00
reger
11b21308c0 fix: malformed filename in image search
fix for http://mantis.tokeek.de/view.php?id=533
2015-02-01 05:35:09 +01:00
reger
4eb89d7f15 revert clickservlet
(default was indeed a mistakenly)
2015-01-05 09:10:20 +01:00
reger
ebe5faeb01 added url to bookmark icon link
url is anyway needed, saves index lookup and works w/o commited url.
Removed unused order parameter
2015-01-05 06:55:53 +01:00
reger
d44d8996d0 Added a “don't store remote search results” option
This is intended for peers who want to participate in the P2P network but don't wish to load/fill-up their index with metadata of every received search result. 
The DHT transfer is not effected by this option (and will work as usual, so that a peer disabling the new store to index switch still receives and holds the metadata according to DHT rules).
Downside for the local peer is that search speed will not improve if search terms are only avail. remote or by quick hits in local index.

To be able to improve the local index a Click-Servlet option was added additionally.
If switched on, all search result links point to this servlet, which forwards the users browser (by html header) to the desired page and feeds the page to the fulltext-index.
The servlet accepts a parameter defining the action to perform (see defaults/web.xml, index, crawl, crawllinks)

The option check-boxes are placed in ConfigPortal.html
2015-01-04 11:10:45 +01:00
reger
0dfeee154a adjustments for Bookmark icon to act on BookmarkDB,
it acts on YMarks but YMark interface seems not maintained,
for future features (e.g. query memory) BookmarkDB is the likely choice to expand, besides the crawlstart bookmark also the result bookmark icon now adds to BookmarkDB.
The YMark related code is (for now) left untouched so both tables are updated.
2015-01-01 02:41:20 +01:00
Michael Peter Christen
ecb6a59e9e do not translate gif images into png images for thumbnails. Instead,
stream the original to the search result thumb viewer. This has two
reasons:
- animated gifs cause 100% cpu and deadlocks in the jvm gif parser; a
known bug which is obviously not yet fixed
- animated gifs now appear in the search result also as animation
2014-12-28 14:53:55 +01:00
reger
7e4e9f7e32 improve yacysearchitem,
prevent allocation of String (modifyURL) if feature not used
2014-12-25 02:16:19 +01:00
Michael Peter Christen
5516819354 preventing the use of no-cache and expires in case that images are
generated dynamically which will stay static in the future. This applies
mainly to the search result favicon in front of search hits. These icons
will now be generated once, but then caches in the browser. There is
also a YaCy-internal cache for these icons which had prevented the
re-generation of the icons in YaCy, but this cache is now superfluous
since the browser should not call the servlet ViewImage again.
2014-12-19 17:41:38 +01:00
Michael Peter Christen
3c71e1c872 show vocabularies in search result (in case of debugging) 2014-11-28 01:19:31 +01:00
Michael Peter Christen
b0bfafa581 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2014-09-12 16:37:21 +02:00
Michael Peter Christen
1735dbc9d9 enhanced image search: bugfixes and performance enhancements 2014-09-12 16:37:01 +02:00
reger
1d5d0b82a6 - skip html template specific servlet post variables (show_xxx) for feeds,
- add <updated> (in required format) to atom feed
2014-09-12 02:10:18 +02:00
reger
19e35a9126 add type attribute to atom feed <link> tag
(for /yacysearch.atom)
2014-08-11 21:46:39 +02:00
Michael Peter Christen
c115f3869c enhanced snippet computation and test method in ViewFile 2014-07-28 15:42:57 +02:00
reger
c798a9d1bb fix unresolved pattern in yacysearch.rss title
and rss xml error due to html & encoding in url entries
2014-06-07 03:01:26 +02:00
orbiter
ce1dbfeb0f fix appearance of image search thumbnails. 2014-05-22 15:01:58 +02:00
Michael Peter Christen
f0db501630 better handling of ranking parameters and new default values for date
navigation which is done using ranking in solr.
2014-05-22 03:01:07 +02:00
Michael Peter Christen
cbdfef7ce1 changed protocol facet to show also all other counts if one facet is
selected
2014-03-27 13:29:14 +01:00
Michael Peter Christen
d1091e79f8 - added stealth button to navigation menu
- more fixes to progress bar
2014-03-21 18:01:26 +01:00
reger
365f77ea8c make internal page links relative to ease any future development for context aware servlets
note also http://bugs.yacy.net/view.php?id=106
2014-02-10 21:40:42 +01:00
Michael Peter Christen
7e71dcc417 removed interaction fragments 2014-02-09 12:25:07 +01:00
reger
bd1685c94a fix not needed getFileExtension().toLower (double)
add missing .getFileExtension
2014-02-05 03:45:02 +01:00
reger
97e84439fb adjusted ConfigHeuristic and changed QueryGoal.getOriginalQueryString to .getQueryString
- since specific heuristic Twitter & Blekko is not longer available or redundant with OpenSearchHeuristic,
adjusted ConfigHeuristic to use OpensearchHeuristic settings only.
For this the default OSD search target list is made available (copied) by default and the other configs are removed.

- the return of QueryGoal.getOriginalQueryString includes the queryModifier, which are held separately in a modifier object,
but in most (all) cases just the query term is expected, clarified and renamed it to QueryGoal.getQueryString which returns
just the search term (if needed a .getOrigianlQueryString could be implemented in Queryparameters, adding the modifiers)

- started to adjust internal html href references from absolute to relative (currently it is mixed).
For future development we should prefer relative href targets (less trouble with context aware  servlets)
2014-01-20 00:58:17 +01:00
reger
eaf596a257 adding proxy status to (private) status box
(show also transparent and url proxy status)

show search result via url proxy only if status=on
2014-01-04 16:10:54 +01:00
Michael Peter Christen
da380343c2 perform greedy learning heuristic only if load < 1.0 2013-12-04 22:44:51 +01:00