Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
8,702 Commits 1 Branch 20 Tags 836 MiB
e816b88b55
Commit Graph

3911 Commits

Author SHA1 Message Date
cominch
e859481889 Add Triplestore settings functionality 
Conflicts:
	htroot/env/templates/header.template
 2012-06-10 10:55:00 +02:00
cominch
b0bc0b4572 Add new demonstration module for client-side key-value store (backend: 
triplestore): /DemoServletInteraction.html

Conflicts:
	source/net/yacy/interaction/Interaction.java
 2012-06-10 10:53:30 +02:00
cominch
c9dc6cda02 Demonstration: include value from interaction in search results 
Conflicts:
	htroot/interaction/OverlayInteraction.html
	htroot/yacysearchitem.java
 2012-06-10 10:51:53 +02:00
cominch
ae8adb0e58 Small changes 2012-06-10 10:44:16 +02:00
cominch
bcbd8eee33 Add several parsers, for RDFa and rdf files. 
Conflicts:
	source/net/yacy/document/TextParser.java
 2012-06-10 10:42:33 +02:00
cominch
9ef5a80f4e add interaction for triples and selector for augmented browsing 
Conflicts:
	htroot/interaction/interaction.js
	source/net/yacy/interaction/Interaction.java
 2012-06-10 10:38:54 +02:00
cominch
5d20cd324a Add Triplestore and RDF query interface 
Conflicts:
	build.xml
	defaults/yacy.init
	source/net/yacy/interaction/AugmentHtmlStream.java
 2012-06-10 10:35:59 +02:00
cominch
bc9a618e0a augmented browsing: ignore js and css, integrate more user interaction 
Conflicts:
	htroot/interaction/Footer.html
	source/net/yacy/interaction/AugmentHtmlStream.java
 2012-06-10 10:29:15 +02:00
cominch
9cbfc1a1c0 augmentedProxy, which forwards every proxy request to a 
rewrite engine to customize existing webpages. originally implemented by
Florian Richter.

Conflicts:
	source/de/anomic/http/server/HTTPDProxyHandler.java
 2012-06-10 10:15:34 +02:00
cominch
1626be7916 Add menu entries for urlproxy / augmented browsing 2012-06-10 09:59:30 +02:00
Michael Peter Christen
5b25272f40 added location search to main menu 2012-06-09 09:10:54 +02:00
Michael Peter Christen
ea0dceb55d bugfix: do not switch off standard memory strategy when performing a 
forced GC
PLEASE CHECK if your peer has standard memory switched on!
 2012-06-08 09:48:46 +02:00
Michael Peter Christen
dd14b19c26 lazy initialization of block rank table ... only normal web search uses 
this. When interactive search or location search is used, the block rank
is switched off
 2012-06-08 09:41:29 +02:00
Michael Peter Christen
701b9a28a0 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 
Conflicts:
	htroot/PerformanceMemory_p.java
 2012-06-08 09:16:16 +02:00
Michael Peter Christen
ab7107b34b fixed RWIProcess queue limits: now discovering hidden results for mass 
result retrieval
 2012-06-08 09:14:54 +02:00
Michael Peter Christen
10c9c17d51 fixed handlemap spread factor and null iterator handling 2012-06-08 09:13:41 +02:00
Michael Peter Christen
a61f44f9e4 lazy initialization of block rank table. 
this causes that the table is not initialized when there is no search is
done. the effect is most strong if YaCy is started headless which causes
no browser pop-up which otherwise would load the search page and
therefore trigger the initialization of the table.
 2012-06-07 13:16:38 +02:00
Michael Peter Christen
c8bbd180e4 enhanced hint for debian package automatic update 2012-06-07 12:36:26 +02:00
Michael Peter Christen
9ad84c5e9f fix for NPE in PerformanceMemory 2012-06-07 12:36:05 +02:00
Michael Peter Christen
96e9d77270 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 
Conflicts:
	source/net/yacy/cora/sorting/WeakPriorityBlockingQueue.java
 2012-06-06 20:13:28 +02:00
Michael Peter Christen
d7eb18cdf2 accept also file names beginning with "file://" for crawl start from 
file.
 2012-06-06 14:27:18 +02:00
Michael Peter Christen
3dd8376825 added automatic cleaning of cache if metadata and file database size is 
not equal. It might happen that these data is different because one of
that caches is cleaned after a while or when it is too big. The metadata
is then not cleaned, but now wiped after a checkup process at every
application start. This should cause a bit less memory usage.
 2012-06-06 14:15:24 +02:00
Michael Peter Christen
d0ec8018f5 fixes for bad long computation 2012-06-06 14:13:31 +02:00
Michael Peter Christen
96c8119b50 added GeoLocation / GeoPoint classes which uses less memory than 
Location/Coordinates and has initializers with correct order of lat,lon
coordinates
 2012-06-06 12:57:42 +02:00
Michael Peter Christen
461a0ce052 removed warnings 2012-06-05 20:03:43 +02:00
Michael Peter Christen
62ae9bbfda allow more POIs, get more at once 2012-06-05 18:29:54 +02:00
Michael Peter Christen
a1fe65b115 performance hacks 2012-06-05 12:06:26 +02:00
Michael Peter Christen
2fe207f813 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-06-04 23:44:38 +02:00
Michael Peter Christen
5aee19daa4 added show from cache in search results (not yet finished) 2012-06-04 23:44:26 +02:00
Michael Peter Christen
e0d8643226 - performance hacks 
- added log warnings in case that search processes run into time-out
situations
- better concurrency for Integer formatter (used a non-synchronized
formatter before)
- bugfix for search termination (a poison pill was missing)
- added timeout parameters for search (again) -> target is, that they
are never reached.
 2012-06-04 15:37:39 +02:00
Michael Peter Christen
cf79b6cee3 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-06-01 08:32:26 +02:00
Michael Peter Christen
6e83b02b83 - bugfix for surrogate file reader 
- bugfix for location search: suppress empty search
 2012-06-01 00:08:31 +02:00
Michael Peter Christen
9b4c699526 ehanced location search: 
- search request are now made using a map boundary
- search results are only computed for the map boundary
- the number of results is adopted to the results in the visible range
- added a double-buffering for the search result markers
- added a search query option for the search results:
/radius/<lat>/<lon>/<radius>
 2012-05-31 22:39:53 +02:00
Michael Peter Christen
434af404c1 - added double-buffering for search layers 
- added automatic zooming to search result
to location search
 2012-05-31 14:05:36 +02:00
Michael Peter Christen
4d9b2dc487 automatically zoom to result layer bounds 2012-05-31 01:12:06 +02:00
Michael Peter Christen
6b40803adf - show number of results in map search interface 
- transfer view radius within query
 2012-05-31 00:47:52 +02:00
Michael Peter Christen
a8778e9c47 npe fix 2012-05-30 15:28:45 +02:00
Michael Peter Christen
1a6fab60e0 added node state to xml 2012-05-30 09:32:25 +02:00
Michael Peter Christen
20e0cc0822 fix for bad location evaluation 2012-05-29 14:46:13 +02:00
Michael Peter Christen
1ab3de0885 fixes to location search 2012-05-29 12:43:14 +02:00
Michael Peter Christen
f167a1c69f removed osmarender from yacysearch_location because that caused a 
javascript error
 2012-05-29 02:22:02 +02:00
Michael Peter Christen
71c3163f3d - fixes to node identification 
- added link to node in network list
- added marking of portal search node peers
 2012-05-29 01:38:54 +02:00
Michael Peter Christen
d1e9fe3db5 enhanced RootState icon 2012-05-29 00:06:33 +02:00
Michael Peter Christen
ad222be7f8 added node state icon in network list 2012-05-25 17:29:54 +02:00
Michael Peter Christen
638390930d another patch to fix the Crawler_p layout 2012-05-25 15:56:21 +02:00
Michael Peter Christen
c846e9ca14 redesign of the crawler monitor page: show crawled pages instead of 
queue of urls that shall be crawled
 2012-05-25 01:45:38 +02:00
Michael Peter Christen
8b974905ee changed log-in text for all servlets with authentication: 
- added hint how to set the password using a shell script
- added a shell script to change the password
 2012-05-24 13:24:31 +02:00
Michael Peter Christen
16b21f7a5b Added more steering in Crawler_p.html interface 2012-05-23 18:00:37 +02:00
Michael Peter Christen
c15fcde1c8 add-on to latest commit 2012-05-21 17:52:30 +02:00
Michael Peter Christen
cf47d94888 performance hack to parse numbers inside of substrings without actually 
generating a substring. This avoids the allocation of a String object
ech time a substring is parsed. Should affect CPU load during RWI
transmission.
 2012-05-21 13:40:46 +02:00
Michael Peter Christen
7bf421b9dd - fixed image search page navigation 
- removed some deadlocks and ConcurrentModificationExceptions during
DidYouMean collection
 2012-05-21 01:58:29 +02:00
reger
6696cb1313 bugfix: lookup of peernames no result for active peer in page IndexControlRWIs_p.html -> Transfer RWI to other Peer 
SeedDB.lookupByName searche for lowercase peerNames, while MapColumnIndex.getIndex uses peername as is in the keyset.
Changed the index init to insert lowercase peer names as key
 2012-05-20 05:25:16 +02:00
Michael Peter Christen
4298f00d2d fixed bad usage of given words 2012-05-20 01:35:49 +02:00
Michael Peter Christen
0d32a766ed relax verify attribute for search widget to make it faster: 
set to "cacheonly"
 2012-05-20 00:50:54 +02:00
reger
ae335a4190 bugfix Tables_p for edit and delete selected row (correction to use "pk_" html prefex) 2012-05-19 23:15:29 +02:00
Michael Peter Christen
f294f2e295 bugfix to http://bugs.yacy.net/view.php?id=181 
tried to make a bit less 'noise' to dns server

also included: less processes in snippet fetch to reduce load during
search on small computers
 2012-05-19 01:06:33 +02:00
Michael Peter Christen
1473e2258e fix for http://bugs.yacy.net/view.php?id=154 2012-05-18 23:56:40 +02:00
Michael Peter Christen
3e1bc9477f Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-05-17 13:58:09 +02:00
Roland 'Quix0r' Haeder
fbb946f913 Made a method static (Eclipse suggested it), removed unused import, pk=null check does now output a warning in logfile 2012-05-17 05:55:44 +02:00
Roland 'Quix0r' Haeder
5f983faef9 No &amp; in JavaScript-embeded URLs, added ability to stop focus in 
ConfigPortal.html preview (is this not secured with _p????)

Conflicts:
	htroot/yacyinteractive.java
	htroot/yacysearch.java
 2012-05-17 05:49:25 +02:00
Michael Peter Christen
5b3acc12cd Pattern.quote() replaces \\Q and \\E according to publication in 
http://www.cs.washington.edu/homes/mernst/pubs/regex-types-ftfjp2012.pdf
 2012-05-17 03:55:10 +02:00
Michael Peter Christen
89142d1e8d removed (not all) warnings 2012-05-16 13:42:32 +02:00
Michael Peter Christen
ffa4553229 typo 2012-05-16 10:14:44 +02:00
Michael Peter Christen
5deebd02ea added serialization 2012-05-15 23:10:47 +02:00
reger
b2175ea4ef Add possibility to set custom Solr field names for the YaCy default Solr attributes. 
- Changing the format of YaCy's solr.key.list while maintainig backward compatibility
  Federated index config screens adjusted accordingly
- modified the Solr update request to use a 3 min Solr autocommit intervall
 2012-05-15 22:34:02 +02:00
Michael Peter Christen
0d58fea210 made multiple connector default 2012-05-12 10:39:01 +02:00
Michael Peter Christen
8864141872 more abstraction in solr connection classes 2012-05-09 17:00:56 +02:00
Michael Peter Christen
c00efc2717 made the solr connection more generic 2012-05-09 16:46:45 +02:00
Michael Peter Christen
f130ab39e8 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-05-08 22:55:52 +02:00
Marc Nause
a691023d04 *) better formatting for network QPM 
*) refactoring
 2012-05-08 20:07:34 +02:00
Michael Peter Christen
dcccbe0be8 removed superfluous column 2012-05-07 00:54:00 +02:00
Michael Peter Christen
77f8e9fb9b Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-05-04 17:29:16 +02:00
Michael Peter Christen
ba6aaabc51 refactoring + parser bugfixes 2012-05-04 17:28:27 +02:00
Michael Peter Christen
a18b6dee04 Merge remote branch 'bbyacy-rc1/master' 2012-04-29 21:22:23 +02:00
reger
ea932f841c changed link to opensearchdescription document to a absolute uri (in yacysearch.html and yacysearch.rss) 
see http://www.opensearch.org/Specifications/OpenSearch/1.1/Draft_5#The_.22Description.22_element
 2012-04-29 05:50:56 +02:00
Michael Peter Christen
453010bd68 - solved problems with backpath normalization 
- redesigned in/outbound link handover
- removed iframe links from inbound/outbound in solr scheme
 2012-04-27 16:48:51 +02:00
Michael Peter Christen
5f5ed33ed8 patch for media search (audio, video apps) 2012-04-27 14:18:02 +02:00
Michael Peter Christen
0e13022147 - enhanced solr field documentation 
- added xml api button to IndexFederated_p - the solr schema.xml file
can be generated by YaCy
 2012-04-26 15:25:07 +02:00
Michael Peter Christen
08dcf3e5d1 hack to get all results if the actual number is between 10 and 64 2012-04-26 00:27:21 +02:00
Michael Peter Christen
19efbf1b0f - apply directDocByURL to NOLOAD Queue 
- choose pushing to NOLOAD as default for site crawl
 2012-04-26 00:23:18 +02:00
Michael Peter Christen
5c66880be2 fix for search result selection in case that contentdom is not set 2012-04-26 00:04:23 +02:00
Michael Peter Christen
3bea25c513 increased image preview size 2012-04-24 16:04:13 +02:00
Michael Peter Christen
a3badd3205 changed search process for images: no more media snippet load process, 
show only links from index which had been on the text search page
before. This creates a superfast search process for images!
 2012-04-24 12:55:58 +02:00
Michael Peter Christen
4aa0eedead one more scroogle... 2012-04-24 12:05:37 +02:00
Michael Peter Christen
347612ddd4 removed scroogle parser 2012-04-24 12:04:44 +02:00
Michael Peter Christen
f8cd57c92f new indexing strategy: ALL links that appear anywhere are indexed, not 
only links where the content can be parsed. All non-parseable links are
placed into the noload queue. The search process must therefore be able
to filter out non-text search results.
- This fixes the problem that image search results appeared in the text
search.
- The interactive search can retrieve now ALL types of links
- The p2p interface is now extended to retrieve only certain types of
links (text, image, video, apps)
- The search process has an extension to filter the right document type
according to the search query
 2012-04-22 02:05:17 +02:00
Michael Peter Christen
14f67f217c refactoring of ContentDomain: now subclass of Classification 2012-04-22 00:04:36 +02:00
Michael Peter Christen
a5d7da68a0 refactoring: removed dependency from switchboard in Balancer/CrawlQueues 2012-04-21 13:47:48 +02:00
Michael Peter Christen
33d1062c79 refactoring: the cache belongs to the crawler 2012-04-21 13:34:07 +02:00
Michael Peter Christen
8429967ea7 no more SVN 2012-04-19 13:29:08 +02:00
Michael Peter Christen
0466bb0ddf no more SVN.. 2012-04-19 13:28:12 +02:00
Michael Peter Christen
4844e124b1 one more warning in case that crawling is paused because of low disk 
space
 2012-04-19 12:35:11 +02:00
Michael Peter Christen
0ec2713af8 'download' 2012-04-19 11:50:24 +02:00
Michael Peter Christen
f30c577fdb add hint to speed up search results 2012-04-19 11:11:14 +02:00
Michael Peter Christen
6b133de3e9 add hint for consulting support 2012-04-19 11:10:48 +02:00
Michael Peter Christen
eb2c8ffa62 display is not used any more 2012-04-17 12:30:14 +02:00
Michael Peter Christen
91a86f0b06 fixed to network graph testing 2012-04-17 11:46:14 +02:00
Michael Peter Christen
f31ad84d98 automatic generation of blacklist pattern, see 
http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2685&p=25305#p25305
 2012-04-17 11:22:19 +02:00
Michael Peter Christen
7b5b9baee0 added citation rank to ranking profile 2012-04-16 23:43:50 +02:00
reger
06951ef751 remove heuristic scroogle from search option help text in index.html 2012-04-16 04:00:04 +02:00
Michael Peter Christen
e377092198 fix to xml output format 2012-04-13 09:02:18 +02:00
Michael Christen
41be98dc9d extended webstructure api to show together with incoming links also 
outgoing links
 2012-04-13 11:53:34 +02:00
Michael Christen
8f89c8ef07 added information about inbound, outbound and citation links into 
yacydoc api servlet
 2012-03-31 07:38:49 +02:00
Michael Christen
71649a1296 added an api to retrieve the new citation.index with the 
webstructure.xml api. This api will respond with details about a single
URL if requested with 'webstructure.xml?about=[url|urlhash|host]'.
 2012-03-29 17:22:31 +02:00
Lotus
3e61287326 some better feedback on properties change 2012-03-25 22:21:42 +02:00
Lotus
96ac95cff9 added hint how to change integration options 2012-03-23 17:02:50 +01:00
Thomas
4f61b8fd82 Fixes for compare-search 2012-03-21 21:43:47 +01:00
Thomas
e0680de7b3 Remove Scroogle from compare-search, Scroogle is dead 2012-03-20 23:00:06 +01:00
Lotus
78f0d8f046 no focus on preview frames for search integration 
fixes bug http://bugs.yacy.net/view.php?id=161
 2012-03-17 21:10:29 +01:00
Lotus
7792ac6406 fix links & bug #163 2012-03-10 10:59:56 +01:00
Michael Peter Christen
532c7cf827 added physics experiment to the graph plotter. not active by default 2012-02-28 13:18:46 +01:00
Michael Peter Christen
aba9b1bfa0 better names for elements of a linked graph 2012-02-27 21:27:17 +01:00
Michael Peter Christen
2fc8ecee36 ConcurrentLinkedQueue has a VERY long return time on the .size() method. 
See
http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/ConcurrentLinkedQueue.html

and the following test programm:

public class QueueLengthTimeTest {


    public static long countTest(Queue<Integer> q, int c) {
        long t = System.currentTimeMillis();
        for (int i = 0; i < c; i++) {
            q.add(q.size());
        }
        return System.currentTimeMillis() - t;
    }

    public static void main(String[] args) {
        int c = 1;
        for (int i = 0; i < 100; i++) {
            Runtime.getRuntime().gc();
            long t1 = countTest(new ArrayBlockingQueue<Integer>(c), c);
            Runtime.getRuntime().gc();
            long t2 = countTest(new LinkedBlockingQueue<Integer>(), c);
            Runtime.getRuntime().gc();
            long t3 = countTest(new ConcurrentLinkedQueue<Integer>(),
c);

            System.out.println("count = " + c + ": ArrayBlockingQueue =
" + t1 + ", LinkedBlockingQueue = " + t2 + ", ConcurrentLinkedQueue = "
+ t3);
            c = c * 2;
        }
    }
}
 2012-02-27 00:42:32 +01:00
Michael Peter Christen
8aba045ba1 if a new pop-up page is set in config portal, then this page applies 
also to the default page configuration for the httpd if no path is
given.
 2012-02-26 20:53:32 +01:00
Michael Peter Christen
fa7b3481b3 better navigation in file search: less results by first try, but much 
faster. after the first search is done, buttons appear to get more
results for the same search
 2012-02-26 17:32:45 +01:00
Michael Peter Christen
8c06925984 animation of the web structure picture 2012-02-25 15:42:29 +01:00
Michael Peter Christen
99c74699de removed scroogle (scroogle is dead) 2012-02-25 12:57:59 +01:00
Michael Peter Christen
6e51a00a2f Revert "fix for page navigation: show only as much pages as are available for given navigation constraints, not as given by total results size" 
This reverts commit 73f5a9e8b3.
 2012-02-24 02:46:56 +01:00
Michael Peter Christen
73f5a9e8b3 fix for page navigation: show only as much pages as are available for 
given navigation constraints, not as given by total results size
 2012-02-24 02:31:03 +01:00
Michael Peter Christen
9c51dc0f13 fixed a bug with navigation: if a navigation was applied to file type or 
protocol, then it was not possible to remove that again. This is the fix
for that.
 2012-02-24 02:28:40 +01:00
Michael Peter Christen
8bfc987374 enhanced hint how to enter file:// urls 2012-02-24 02:14:54 +01:00
Michael Peter Christen
c6c61be3f0 fix for http://bugs.yacy.net/view.php?id=148 2012-02-24 00:38:57 +01:00
Michael Peter Christen
edaa8ac94c Merge commit 'e15e633a0128b8d31011283a65b4ef26a6dddcd8' 2012-02-23 10:07:13 +01:00
reger
e15e633a01 Bugfix for IE9 (doesn't accept html form within form) 
changes of API schedule row data changed form input form to unique field names
using row pk.
Fix for issue 96 http://bugs.yacy.net/view.php?id=96

IE9-64bit doesn't interprete iframe with align parameter as desired
misaligns following content (in CrawlProfileEditor_p.html)
 2012-02-23 02:40:07 +01:00
Michael Peter Christen
a9b4d49b75 removed debug output 2012-02-21 22:31:14 +01:00
Michael Peter Christen
8d63a5887c bugfixes 2012-02-02 23:38:23 +01:00
Michael Peter Christen
9ad1d8dde2 complete redesign of crawl queue monitoring: do not look at a 
ready-prepared crawl list but at the stacks of the domains that are
stored for balanced crawling. This affects also the balancer since that
does not need to prepare the pre-selected crawl list for monitoring. As
a effect:
- it is no more possible to see the correct order of next to-be-crawled
links, since that depends on the actual state of the balancer stack the
next time another url is requested for loading
- the balancer works better since the next url can be selected according
to the current situation and not according to a pre-selected order.
 2012-02-02 21:33:42 +01:00
Michael Peter Christen
e2f8f263e8 changed storage of search words: keep order 2012-02-01 18:13:31 +01:00
Michael Peter Christen
c166eb68b6 fixes in solr schema file 2012-02-01 00:22:43 +01:00
Michael Peter Christen
2e5cd6a1b2 fixed parser extension deny list generation and usage 2012-02-01 00:15:59 +01:00
david
f077b11d38 Merge branch 'master' of git://git.gitorious.org/yacy/rc1.git 2012-01-30 20:02:11 +01:00
Lotus
29675d9766 more label on search options (usability) 2012-01-30 20:02:02 +01:00
Michael Peter Christen
b4bc1e2875 remote search does not do snippet generation 2012-01-29 19:25:09 +01:00
Lotus
335a776351 xss hardening on Status.html 2012-01-28 13:25:12 +01:00
Michael Peter Christen
4c5edab1ec added option to have exception search result windows 2012-01-26 15:32:30 +01:00
Michael Peter Christen
32adad7dd5 show less navigation by default 2012-01-26 01:09:34 +01:00
Michael Peter Christen
88b86afc89 no DoS protection for intranet mode 2012-01-25 12:13:03 +01:00
Michael Peter Christen
0f443ac755 automatic switching off of navigation that is not useful 2012-01-25 12:07:24 +01:00
Michael Peter Christen
852ce43d99 better rules for default open/close of navigation objetcs 2012-01-25 11:53:25 +01:00
Michael Peter Christen
e8d24fd802 author navigator can be switched off 2012-01-25 11:11:42 +01:00
Michael Peter Christen
558ab7bd4e made the protocol navigator reversible 2012-01-25 02:54:52 +01:00
Michael Peter Christen
96cb75f1d4 made the filetype navigator be able to deselect the search constraint 2012-01-25 02:50:06 +01:00
Michael Peter Christen
30891d026f added a remove-navigation for vocabularies 2012-01-25 00:22:51 +01:00
Michael Peter Christen
d5ead5314d changed navigation links: now using checkboxes. 
This looks better and allows that negative checkboxes (such that remove
the navigation) are possible. These are not yet implemented (comming
next)
 2012-01-24 19:03:47 +01:00
Michael Peter Christen
ef5192f8c9 using the generic document parser for crawl starts instead of the html 
parser. This makes it possible that every type of document can be a
crawl start point, not only text documents or html documents. Testet
this with a pdf document.
 2012-01-23 17:27:29 +01:00
Michael Peter Christen
ce620be783 for for crawl start with smb url 2012-01-19 23:07:15 +01:00
Lotus
18227790ca variable search works at least on Firefox again 2012-01-19 16:54:09 +01:00
Lotus
c73af39e54 refactoring of tray icon class, 
now uses Java 6 methods natively
 2012-01-18 20:47:09 +01:00
Michael Peter Christen
5afec4af39 fix for http://bugs.yacy.net/view.php?id=131 2012-01-18 11:58:51 +01:00
Michael Peter Christen
dc165275ad bugfix for usage of multiple vocabulary navigators 2012-01-18 01:00:30 +01:00
Michael Peter Christen
190b77c55e added Ukrainian translation 2012-01-17 17:45:28 +01:00
Michael Peter Christen
7f9b6b7a0c added switches to ConfigParser to accept/deny documents by their 
extension
 2012-01-17 16:43:34 +01:00
Michael Peter Christen
83009d86f7 added the vocabulary navigator. It can be very simply tested by 
switching on the locale dictionaries.
 2012-01-17 01:53:08 +01:00
Lotus
37d43e5589 jump back to where we want to modify the scheduler in api steering after 
submit
 2012-01-16 20:08:24 +01:00
Michael Peter Christen
26e227f17d Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-01-13 11:28:48 +01:00
Michael Peter Christen
7053f8ab46 added automatic generation of a solr schema.xml file 2012-01-13 11:26:22 +01:00
apfelmaennchen
c7f88f3fd1 fix for http://bugs.yacy.net/view.php?id=101 - the default crawl 
depth for bookmarks is now editable.
 2012-01-12 23:30:23 +01:00
Michael Peter Christen
c602eaaf46 enhanced search process 2012-01-10 03:00:55 +01:00
Michael Peter Christen
a71f751cc8 Merge branch 'master' of git://gitorious.org/~apfelmaennchen/yacy/apfelmaennchens-rc1 2012-01-09 23:33:18 +01:00
sof
a94ced7cb2 Solved (http://bugs.yacy.net/view.php?id=124) accessibility issue with 
page 'Ranking_p.html'
 2012-01-09 22:43:55 +01:00
Michael Christen
eff966f396 fix for search process (it was aborted too early during remote search) 2012-01-09 03:02:35 +01:00
Marek Otahal
6d8faa8066 small fix 
Signed-off-by: Marek Otahal <markotahal@gmail.com>
 2012-01-09 01:19:51 +01:00
Marek Otahal
1dc5d9f0f3 make ConnectionInfo comparable and sort list of connections in Connections_p 
ConnectionInfo compare by initTime
Connections_p implement wish to sort connections, descending

Signed-off-by: Marek Otahal <markotahal@gmail.com>
 2012-01-09 01:14:41 +01:00
Marek Otahal
c1973c78a3 Blog + BlogComments method propose() make similar + fix anonymous poster's IP resolve in Blog 
this method share very much common code in both classes,
would be nice to unify
+fix some ugly code

Signed-off-by: Marek Otahal <markotahal@gmail.com>
 2012-01-09 01:10:50 +01:00
Marek Otahal
7189a70a62 BlogComments dateString() - use localized date/time format 
use same method from Blog class, reduce code duplication

Signed-off-by: Marek Otahal <markotahal@gmail.com>
 2012-01-09 01:09:59 +01:00
Marek Otahal
f727c35722 Blog dateString() print in localized date/time format 
Signed-off-by: Marek Otahal <markotahal@gmail.com>
 2012-01-09 01:09:12 +01:00
Marek Otahal
8318103f25 Blacklist addBlacklistEntry() - dont add an empty entry 
Signed-off-by: Marek Otahal <markotahal@gmail.com>
 2012-01-08 23:43:04 +01:00
Michael Peter Christen
972d411d99 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-01-07 17:51:37 +01:00
Michael Christen
bd40a10230 added autotaggig stub .. only reading and parsing of vocabularies at 
this time
 2012-01-07 17:34:38 +01:00
Michael Peter Christen
f214f6ebb4 added no-load queues to the crawler monitor 2012-01-07 17:17:11 +01:00
Michael Peter Christen
2ee8cbeb2c Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 
Conflicts:
	source/net/yacy/search/Switchboard.java
 2012-01-05 18:37:46 +01:00
Michael Peter Christen
992dbdf4bb added noload statistic to servlets 2012-01-05 18:33:05 +01:00
Michael Christen
216a287a85 Merge commit '6d4e08ed06c5cd28c45981b2ebe31c7f7ec6fd83' into quix0r 
Conflicts:
	source/de/anomic/crawler/CrawlQueues.java
 2012-01-04 20:16:37 +01:00
stbrumm
d18095dc48 Patch fuer Issue 0000102 
and fixes to Patch (private peer status is a property of a peer, not a
status)
 2012-01-03 17:49:37 +01:00
Michael Christen
ce289eb64f remove heuristics if index receive is switched off 2011-12-30 23:51:07 +01:00
Roland 'Quix0r' Haeder
fa08ed5ae5 Fixed a lot CHMOD rights (no need for execute flag on *.java/*.html) and introduced local/remote crawl size ratio based check 2011-12-29 00:33:16 +01:00
Michael Christen
1d66496362 added more properties to opensearchdescription, see also: 
http://bugs.yacy.net/view.php?id=114
 2011-12-28 10:49:32 +01:00
Michael Christen
1cf0f35621 the link to the path shall be the path 2011-12-28 01:12:44 +01:00
Michael Christen
87c16fb67f show server list as server side include 2011-12-28 01:12:23 +01:00
Michael Christen
5bfb287753 make a bad fix even worse 2011-12-27 20:59:03 +01:00
Michael Christen
4656bb0365 allow html in greeting line since that is set by administrator only. 
html in that line may be wanted
 2011-12-27 20:51:13 +01:00
Michael Christen
85bd4cc8bc better lookup for peer names 2011-12-25 10:14:15 +01:00
Michael Christen
20e3084bd4 redesign of fining of peers by ip: more leightweight method to read the 
seed databases
 2011-12-21 01:14:43 +01:00
sof
b09988c9f8 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2011-12-18 23:16:39 +01:00
Michael Christen
c67c30db34 added language selector for russian and chinese in ConfigBasic 2011-12-18 16:07:21 +01:00
Michael Christen
9e5894c784 Removed handling of components objects for URIMetadataRows. 
This is a preparation to replace this rows with nodes from the node
store.
 2011-12-17 01:27:08 +01:00
Michael Christen
c04bfaa51b refactoring 2011-12-16 23:59:29 +01:00
Michael Christen
e9dc99fe15 added rules to set specific RWIs as private RWIs which are not 
transmitted to remote peers. This will be used for private index copies
and phonetic indexes.
 2011-12-14 22:15:51 +01:00
Michael Peter Christen
0bcef2d156 added feature as requested in 
http://forum.yacy-websuche.de/viewtopic.php?f=18&t=3461
The search can now be configured with a non-display host list.
the search will always exlude the given list of host unless they are
requested directly using the host navigation
 2011-12-13 00:16:05 +01:00
apfelmaennchen
66e4832989 work around for overflow in YMarks.html import tab 2011-12-11 22:42:55 +01:00
Michael Christen
d6e6f7715b added "about" box configuration 2011-12-10 02:04:36 +01:00
Michael Christen
204c29f010 small bugfixes for search result display and cache display 2011-12-10 01:35:38 +01:00
Michael Peter Christen
9aa73a13a8 stop words are on by default in site crawl. This causes normally nothing 
since the stopwords are empty by default.
 2011-12-08 18:48:30 +01:00
Michael Christen
17f962fceb translator updates: 
- config string for chinese
- do not copy the language file to DATA/LOCALE any more (and do not use
them there, this is really confusing for new translators)
 2011-12-08 10:25:26 +01:00
lofyer
ac5b1e891d new 2011-12-08 16:44:53 +08:00
admin
391fc9bd57 Merge branch 'master' of gitorious.org:yacy/rc1 2011-12-07 01:01:20 +01:00
admin
23afee58fe Merge branch 'master' of git://github.com/f1ori/yacy 2011-12-07 01:01:04 +01:00
apfelmaennchen
ff19fcdb28 bugfix for YMarks XBEL import and export; thanks to Dominic 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8138 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-12-06 23:17:57 +00:00
Michael Christen
c715d19c09 fixes for dependency on svn 2011-12-06 22:05:22 +01:00
Michael Christen
0d1042363c showing /16 and /24 subnet option instead of 'bigrange' in network 
scanner
 2011-12-06 11:37:05 +01:00