Commit Graph

2967 Commits

Author SHA1 Message Date
sixcooler
d8fcc4a2f5 added a timeout on Jetty connectors 2014-10-16 20:36:12 +02:00
sixcooler
72561926aa do not overwrite yacy.conf in case of an exception
may be a fix for http://mantis.tokeek.de/view.php?id=180
2014-10-15 18:13:54 +02:00
orbiter
fa2ad101ec enhanced graphics computation (avoiding long string parsing for colours) 2014-10-15 10:31:24 +02:00
orbiter
ef813cec91 added proper copyright notice to OSM tiles presented at the search
result page
2014-10-15 09:13:23 +02:00
Michael Peter Christen
fca11701f0 better profiling of solr queries 2014-10-15 00:55:42 +02:00
Michael Peter Christen
2e09da9832 npe fix 2014-10-14 12:48:15 +02:00
Michael Peter Christen
d80418f1b1 added partial updates to solr during postprocessing: during
postprocessing the solr documents are now not completely retrieved.
instead, only fiels, needed for the postprocessing are extracted. When
Solr document are written, this is done using partial updates.

This increases postprocessing speed by about 50% for embedded Solr
configurations. For external Solr configurations the enhancement should
be much higher because the postprocessing with remote Solr is very slow.
When doing partial updates to a remote Solr, this method should perform
much better than before, it is expected that this is even much higher
than the increase with local Solr.
2014-10-14 12:19:59 +02:00
Michael Peter Christen
b1cfbc4a04 added new solr field url_paths_count_i which can be used to enhance the
index browser and maybe also for ranking; possibly also for
SEO-with-YaCy applications.
2014-10-13 23:51:19 +02:00
Michael Peter Christen
e69883d5ab fix-fix for
30d4402cd1
2014-10-13 16:51:27 +02:00
Michael Peter Christen
30d4402cd1 fixed location search 2014-10-13 14:28:11 +02:00
Michael Peter Christen
6983dff334 explain crawl denial when not switched to intranet mode 2014-10-11 09:02:12 +02:00
Michael Peter Christen
f818f84adb more ipv6 fixes 2014-10-11 00:34:07 +02:00
Michael Peter Christen
afd5bd5f5f slightly enhanced Network table computation by using a lazy initialized
bitfield for peer flags
2014-10-10 14:40:31 +02:00
Michael Peter Christen
2c2b50e65d refactoring (class name should start with uppercase letter) 2014-10-10 14:32:21 +02:00
Michael Peter Christen
bc275dca07 added network history graph image /NetworkHistory.png which can show
many different statistics about the history of the peer.
2014-10-10 14:06:47 +02:00
Marc Nause
ce9368246b Merge branch 'master' of gitorious.org:yacy/rc1 2014-10-09 13:35:31 +02:00
Marc Nause
5603809deb Minor changes:
*) reduced visibility of a method
*) updated comments
2014-10-09 13:31:36 +02:00
Michael Peter Christen
d8beafba3a fix for values in CrawlProfileEditor table and xml; now the full profile
is available in the xml.
2014-10-09 13:27:20 +02:00
Michael Peter Christen
ec95dfa2e6 fixed crawl profile xml result which did not show the correct crawl
status.
2014-10-08 18:48:57 +02:00
Michael Peter Christen
8c1a89cb34 added another decoration flag to switch off network graphics in crawler
monitor and index browser: decoration.grafics.linkstructure
Please set this to false to remove the graphics from the interface.
2014-10-08 17:12:35 +02:00
Michael Peter Christen
ee27be3399 misc bugfixes (concurrency, memory protection) 2014-10-08 15:22:29 +02:00
Michael Peter Christen
9b1958e8ca more ipv6 bugfixes 2014-10-08 15:21:49 +02:00
Michael Peter Christen
7817fc50c9 added a high cpu cycle monitor to PerformanceQueues 2014-10-08 15:20:43 +02:00
Michael Peter Christen
5082feb103 less volume for effect sounds 2014-10-08 15:04:35 +02:00
Michael Peter Christen
e8392e2ff2 fix for local search 2014-10-08 13:44:03 +02:00
Michael Peter Christen
0bfc69b29b more ipv6 bugfixes 2014-10-08 12:38:56 +02:00
Michael Peter Christen
a27563e5c3 removed the atmo sound clips because they had been too large 2014-10-07 23:42:41 +02:00
Michael Peter Christen
883622306e Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git
Conflicts:
	source/net/yacy/peers/Protocol.java
2014-10-07 23:33:28 +02:00
Michael Peter Christen
97995a1dd9 fix for remote search process 2014-10-07 23:30:32 +02:00
Michael Peter Christen
0843b12ef3 ipv6 fix: avoid that shrinked own ip set is overwritten with (non-valid)
set of local IPs
2014-10-07 22:36:01 +02:00
Michael Peter Christen
92c5d97486 fix for bad node flag setting with IPv6 2014-10-07 22:16:18 +02:00
orbiter
c27bad9326 more ipv6 fixes 2014-10-07 20:09:48 +02:00
orbiter
cddf884bc4 Merge branch 'master' of git@gitorious.org:yacy/rc1.git 2014-10-07 19:27:14 +02:00
Michael Peter Christen
460858fb22 more ipv6 fixes 2014-10-07 18:53:23 +02:00
Michael Peter Christen
5cef88a315 argh.. adding missing java class for latest audio feature 2014-10-07 18:32:39 +02:00
Michael Peter Christen
74957f3760 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2014-10-07 17:51:18 +02:00
Michael Peter Christen
2a052f446a Added an experimental audio feedback system.
This is the first element of a new 'decoration' component which may hold
switches for different external appearance parameters.
The first switch in that context is decoration.audio (as usual in
yacy.init). This value is set to false by default, that means the audio
feedback element is switched off by default. To switch it on, set
decoration.audio = true (using /ConfigProperties_p.html). You will then
hear sounds for the following events:
- remote searches
- incoming dht transmissions
- new documents from the crawler
Sound clips are stored in htroot/env/soundclips/ which is done so
because a future implementation will read these files using the http
client and with configurable urls which will make it very easy for the
user to replace the given sounds with own sounds.
2014-10-07 17:51:07 +02:00
Marc Nause
1e6e69bc40 Finished implementation of UPNP:
*) will try other ports if YaCy standard ports are not available
*) distinguish between internal and external port (not sure if this
works 100%)

Still to add: propery in config to enter own external port (in case of
manually configured NAT)
2014-10-07 13:10:06 +02:00
Michael Peter Christen
d0358e568b Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2014-10-06 17:44:38 +02:00
Michael Peter Christen
e1bc768f9d more IPv6 bugfixes 2014-10-06 17:44:27 +02:00
reger
59c6532a65 add link extraction to pdfParser
this extracts clickable links in pdf and adds it to the list of links

include a test case for this function

this is the corrected comment for commit:
aa2e15d846
2014-10-06 04:51:31 +02:00
reger
aa2e15d846 allow url parameter in worktable apicall
allow url=wwwl?param=a&param=b (with ?, & encoded)
fix:  http://mantis.tokeek.de/view.php?id=100

fix double adding of  '&' in MultiProtocolURL.escape()
2014-10-05 20:05:03 +02:00
orbiter
f3a12801f0 Merge branch 'master' of git@gitorious.org:yacy/rc1.git 2014-10-05 14:50:35 +02:00
orbiter
d93325a578 lazy handling of process_sxt field (part of postprocessing) 2014-10-05 14:50:22 +02:00
Michael Peter Christen
b31db00010 toString fixes 2014-10-05 11:03:57 +02:00
Michael Peter Christen
961f06c0b6 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2014-10-05 01:25:12 +02:00
reger
209e0f2fe8 allow url parameter in worktable apicall
allow url=wwwl?param=a&param=b (with ?, & encoded)
fix:  http://mantis.tokeek.de/view.php?id=100

fix double adding of  '&' in MultiProtocolURL.escape()
2014-10-04 04:11:48 +02:00
reger
b5ca20de15 preserve content_type (mime) if supplied in preference of construct in from file type.
(this eventually can benefit image search by using mime only)

reduce redundant field assignment for Solrdocuments created from URIMetadataNode (URIMetadataNode = SolrDocument with partially assigned fields)
2014-10-03 22:08:07 +02:00
reger
fe9f1c594e fix char encoding parameter in UrlProxy 2014-10-03 08:51:23 +02:00
reger
b0c87d8240 fix image search expand box, cut-off of 2nd capture line height
tested with IE11 and Firefox 32 (change worked for both to show 2nd line without cutting off height)

+fix charset parameter in metadataImageParser
+update start errMsgTxt to "java 1.7"
2014-10-03 01:43:05 +02:00