Commit Graph

11207 Commits

Author SHA1 Message Date
Michael Peter Christen
6491270b3a large IPv6 redesign of peer ping methods!
removed preferred IPv4 in start options and added a new field IP6 in
peer seeds which will contain one or more IPv6 addresses. Now every peer
has one or more IP addresses assigned, even several IPv6 addresses are
possible. The peer-ping process must check all given and possible IP
addresses for a backping and return the one IP which was successful when
pinging the peer. The ping-ing peer must be able to recognize which of
the given IPs are available for outside access of the peer and store
this accordingly. If only one IPv6 address is available and no IPv4,
then the IPv6 is stored in the old IP field of the seed DNA.
Many methods in Seed.java are now marked as @deprecated because they had
been used for a single IP only. There is still a large construction site
left in YaCy now where all these deprecated methods must be replaced
with new method calls. The 'extra'-IPs, used by cluster assignment had
been removed since that can be replaced with IPv6 usage in p2p clusters.
All clusters must now use IPv6 if they want an intranet-routing.
2014-09-30 14:53:52 +02:00
Michael Peter Christen
67cd4c37bd activated the new apk parser which was already ready but not included in
the parser initialization. To make the apk parser usable, the handling
of application type links had to be modified. Now all documents which
have not a parser attached are placed to the noload-queue while all
other documents are parsed using the associated parser class. This may
have side-Effects on other parsers and the display of different file
classes (images, apps, videos).
2014-09-24 13:32:58 +02:00
orbiter
309d978520 Merge branch 'master' of git@gitorious.org:yacy/rc1.git 2014-09-22 15:29:07 +02:00
orbiter
a922b122a3 added a hack to forward solr search results from an external attached
solr to the YaCy built-in solr search servlet. Its not complete and not
fully correct (there is still a utf8 encoding problem) but it is a way
to get easily requests forwarded through YaCy to an external Solr.
2014-09-22 15:28:54 +02:00
reger
46afdf7d21 add link to thread pool settings in status panel 2014-09-21 22:35:03 +02:00
reger
54019313e7 fix NPE in ViewFile - show snippet
on document not in index
2014-09-21 03:48:54 +02:00
reger
4873a2d3a4 adjust link to peer in Network list
(www path obsolete)
2014-09-21 00:10:20 +02:00
reger
537ad6239e upd Maven pom (to current dev version) 2014-09-21 00:04:54 +02:00
Michael Peter Christen
025516f682 fix for crawl limit for number of pages fail 2014-09-20 13:06:46 +02:00
Michael Peter Christen
2645dc816a added warning for not well-formed postprocessing queries 2014-09-18 14:36:57 +02:00
Michael Peter Christen
437ce3b8a0 added internal api for partial updates to Solr 2014-09-18 14:26:45 +02:00
orbiter
3ac31614a3 added option to reverse-sort YaCy tables (internal API change only) 2014-09-18 11:11:09 +02:00
Michael Peter Christen
6d3d4c4ea6 changed the concurrent enumeration of query results in such a way that
it is now possible to get the results in two steps:
- first retrieve all IDs as given for a query
- then retieve each document individually

This was necessary for very large result sets where a query may run for
hours and is possibly terminated by a solr-internal timeout. This occurs
regulary during postprocessing and therefore this commit may fix
unwanted postprocessing terminations.
2014-09-17 13:58:55 +02:00
Michael Peter Christen
ad35d9294f added a 'stats' table which records some peer statistics twice every
hour. The table can be shown with
http://localhost:8090/Tables_p.html?table=stats

The entries have the following meaning: 
aM: activeLastMonth
aW: activeLastWeek
aD: activeLastDay
aH: activeLastHour
cC: countConnected (Active Senior)
cD: countDisconnected (Passive Senior)
cP: countPotential (Junior)
cR: count of the RWI entries
cI: size of the index (number of documents)

The entry keys are abbreviated to reduce the space in the table as the
name is written again for every row.

This is the beginning of a 'yacystats' micro-alternative als built-in
function in YaCy. Graphics may follow after some time if enough test
data is available.
2014-09-17 12:54:50 +02:00
Michael Peter Christen
a11bc10629 next development cycle. Please be careful with the usage of next
commits, maybe new and unstable things will come...
2014-09-17 00:22:23 +02:00
reger
8284ea751a catch TimeoutException during ping and do not delete yacy.conf during prereadconfigfile
found a situation after crash (reboot) with existing running semaphore but YaCy not running.
Ping generated exception which finally deleted the conf file (during pre-read procedure)
- change to ping (catch exception solved it)
- additionally removed delete yacy.conf file (if needed we need to make a backup)
2014-09-16 23:14:13 +02:00
Michael Peter Christen
dc14824814 fix for Mac.app config - but does still not run. looking for build bug. 2014-09-16 17:46:07 +02:00
reger
ffa7c7116f better fix for NPE in image search
replace 8931e14514
2014-09-16 16:43:17 +02:00
Michael Peter Christen
b9de39be11 Release 1.80 2014-09-16 16:02:45 +02:00
Michael Peter Christen
f03dd0df24 updated seedlist 2014-09-16 15:49:03 +02:00
Michael Peter Christen
759e7d9538 fix for http://forum.yacy-websuche.de/viewtopic.php?p=30720#p30720 2014-09-16 14:53:30 +02:00
Michael Peter Christen
bf18a39d0e replaced warning with info 2014-09-16 14:41:04 +02:00
Michael Peter Christen
f1032fb8fe more enhancements to image search in case that a restriction to a single
domain is done
2014-09-16 13:41:01 +02:00
Michael Peter Christen
19c6b97572 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2014-09-16 00:21:07 +02:00
Michael Peter Christen
475125f9d7 hack to get more results when doing a remote site search 2014-09-16 00:13:26 +02:00
reger
ed0d7a80d5 modifiy description for Field-Reindex to act only on local index
http://mantis.tokeek.de/view.php?id=279
2014-09-15 23:29:42 +02:00
Michael Peter Christen
81f9b34da7 increaesed ability ot search for all images on a single server within
the p2p remote search
2014-09-15 20:33:22 +02:00
Michael Peter Christen
2c26013c50 better contentdom abstraction 2014-09-15 14:00:41 +02:00
Michael Peter Christen
6a8fb8190b changed default value for maximum number of connections to 50 2014-09-15 13:50:40 +02:00
Michael Peter Christen
9b92685771 automatically add a wild card if only a search on a single domain is
done. This makes it possible to search all documents on a single domain
even if no search word is given. This is in particular interesting when
searching for all images on a single domain.
2014-09-15 13:38:53 +02:00
Michael Peter Christen
abde89438b fix for favicon 2014-09-15 13:03:08 +02:00
Michael Peter Christen
ca8b2bf099 removed www and welcome servlet, these had been demo servlets and are
not needed any more
2014-09-15 12:48:58 +02:00
reger
5247d01cd4 implement a forward to remote peer link in P2P Network list
Most links in Network.html are only available with transparent proxy = on, which is switched off by default,
to make the provided links useable in default setup a small forward servlet added (goto_p.java),
which takes the peer hash as parameter and forwards to current public ip (optional with path= parameter).
The servlet is protected ( _p ending) to assure forwarding works only for authorized YaCy users.
2014-09-14 23:26:22 +02:00
reger
de7641023c add recommended link "self" to atom feed output 2014-09-14 01:40:27 +02:00
reger
03a7a29db3 limit OAI import urn resolver try for Deutsche National Library
The resolver service of National Library uses name space nbn, limit use of nbn-resolving.de accordingly to urn:nbn:
- add resolver for rfc's
2014-09-14 01:38:27 +02:00
Michael Peter Christen
0838326a76 changed error message, see http://mantis.tokeek.de/view.php?id=439 2014-09-13 17:02:26 +02:00
Michael Peter Christen
805a95a98b fix for http://mantis.tokeek.de/view.php?id=467 2014-09-13 16:26:40 +02:00
Michael Peter Christen
d1604258c1 added a script which retrieves the public ip of the local peer and
prints it out to the terminal using the new servlet /yacy/seedlist.xml.
just call:
bin/myip.sh
2014-09-13 11:09:35 +02:00
Michael Peter Christen
7527ae63e7 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2014-09-13 11:02:38 +02:00
Michael Peter Christen
b93ea4e2a6 Added the option to retrieve only the own seed or a seleceted seed
(selected by peer-hash) in the /yacy/seedlist.[json/xml] api.
Added also a xml-version of the servlet.
The own seed can be retrieved i.e. with
http://localhost:8090/yacy/seedlist.xml?my=
http://localhost:8090/yacy/seedlist.json?my=
and any other peer can be selected with
http://localhost:8090/yacy/seedlist.xml?id=<peerhash>
http://localhost:8090/yacy/seedlist.json?id=<peerhash>
2014-09-13 11:02:11 +02:00
reger
b5e0f70197 - remove repositoryPath post from ConfigBasic (obsolete)
- remove static snippetComputationTime from ResultEntry (not used)
2014-09-13 03:21:52 +02:00
Michael Peter Christen
ffc259c944 changed link to new tutorial repository (yes, Youtube..). The link does
not point to youtube directly to prevent that the referer to the peer
address is given to youtube. Instead, a forwarder address at yacy.net is
used to redirect to the tutorial repository (and can be changed later).
2014-09-13 02:15:36 +02:00
Michael Peter Christen
eab35afb78 Merge branch 'master' of gitorious.org:yacy/icewindxs-rc1
Conflicts:
	locales/ru.lng
2014-09-13 01:19:24 +02:00
reger
8931e14514 fix NPE in image search 2014-09-13 00:27:39 +02:00
malykhin.dmitry
a10f4189f5 Update russian translation 2014-09-13 01:11:25 +04:00
Michael Peter Christen
b0bfafa581 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2014-09-12 16:37:21 +02:00
Michael Peter Christen
1735dbc9d9 enhanced image search: bugfixes and performance enhancements 2014-09-12 16:37:01 +02:00
reger
1d5d0b82a6 - skip html template specific servlet post variables (show_xxx) for feeds,
- add <updated> (in required format) to atom feed
2014-09-12 02:10:18 +02:00
reger
8ed6550261 adding totalResults and id to atom feed output 2014-09-12 00:08:22 +02:00
malykhin.dmitry
9f6ea41353 Update russian translation 2014-09-11 13:34:17 +04:00