Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
8,591 Commits 1 Branch 20 Tags 836 MiB
613b45f604
Commit Graph

3861 Commits

Author SHA1 Message Date
Michael Peter Christen
6d17686258 made triplestore persistent by default 
added a size display in triplestore servlet
 2012-06-15 19:13:07 +02:00
Michael Peter Christen
8d6e77ad0c Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-06-15 14:38:46 +02:00
cominch
2ac7a5c1f2 Augmented browsing: Add overlay bar which shows the vocabulary tags 2012-06-15 14:32:16 +02:00
Michael Peter Christen
777d22e145 renamed "augmented proxy" to "augmented browsing" 2012-06-15 11:54:33 +02:00
cominch
bddac2839e add missing files for tag display 2012-06-15 10:46:19 +02:00
cominch
441430f507 Merge remote-tracking branch 'original yacy/master' 2012-06-15 10:44:12 +02:00
cominch
3c255c025b Show tags in search results (if activated in ConfigPortal_p.html) 2012-06-15 10:43:05 +02:00
Michael Peter Christen
1f9120d189 create new vocabularies also without an objectspace. this creates an 
empty vocabulary
 2012-06-15 02:43:55 +02:00
Michael Peter Christen
a5cdfb91de - fixed Cache link (below snippet) 
- added 'Augmented Proxy' link below snippet
- added configuration options for augmented proxy
 2012-06-14 19:55:34 +02:00
Michael Peter Christen
492b3e09f2 added api icon to triplestore 2012-06-14 19:11:19 +02:00
Michael Peter Christen
16d8f33795 added objectlink generation to vocabulary generation and editor 2012-06-14 18:50:35 +02:00
Michael Peter Christen
f1f97b7c95 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-06-14 18:45:38 +02:00
Michael Peter Christen
b3eaaf5ebc check also delete triplestore by default 2012-06-14 18:14:45 +02:00
cominch
f2f07a11f1 hotfix for unresolved pattern 2012-06-14 18:05:10 +02:00
cominch
5fd1a15fcf hotfix until we have updated query routine for tags 2012-06-14 17:56:38 +02:00
cominch
f49d92d8da Cleanup of interaction class and helper routines 2012-06-14 17:41:45 +02:00
cominch
56b0115054 Triplestore: modify routines to access per user store 2012-06-14 15:44:27 +02:00
Michael Peter Christen
d45718251e refactoring (Localization -> Location) 2012-06-14 09:45:57 +02:00
Michael Peter Christen
b8b3c87ba7 - renamed localization to location (that was confusing) 
- renamed 'Locale' navigator to 'Location'
- produce Location navigation only if geolocation libraries are loaded
 2012-06-14 09:44:14 +02:00
sixcooler
f64e78497a fix for reload-feature in Crawler_p 2012-06-14 02:13:23 +02:00
Michael Peter Christen
e89747bb67 - added automated generation of vocabularies from url stubs 
- added clear of all terms for vocabularies
- added deletion of vocabularies
 2012-06-13 15:53:18 +02:00
Michael Peter Christen
79464189a4 The 'Locale' vocabulary, which is generated by geo data, has now the 
objectspace "http://dbpedia.org/resource/"
 2012-06-13 13:05:41 +02:00
Michael Peter Christen
eca38c53e7 added a vocabulary editor 2012-06-13 12:12:20 +02:00
Michael Peter Christen
80e8aaabc8 moved new servlets into one submenu "Content Semantic" 2012-06-12 02:12:01 +02:00
Michael Peter Christen
2bbb6c52cf added option to clean the triplestore when deleting the index 2012-06-12 01:54:36 +02:00
Michael Peter Christen
8b53771db2 changed behavior of navigation processing: 
- vocabulary annotation is not done any more into the metadata of urldb
- vocabularies are written into the jena triplestore using a rdf
vocabulary
- vocabularies for rdf tripel must be updated; refactoring done
- with the new navigation tags in the triplestore a faster
pre-urldb-lookup is possible: navigation is processed now within the RWI
during pre-ranking retrieval
- added also a Owl vocabulary stub to add the plain-text url to the
triplestore using the owl:sameas predicate
 2012-06-11 23:49:30 +02:00
Michael Peter Christen
5fc6524ca8 - moved triple store to net.yacy.cora.lod (should be generalized there 
later
- added abstract add, delete, get methods in the triplestore
- added generation of triples after auto-annotation
- migrated all MultiProtocolURI objects to DigestURI in the parser since
the url hash is needed as subject value in the triples in the triple
store
 2012-06-11 16:48:53 +02:00
cominch
c90f174799 preparation and generalization of augmented browsing methods 2012-06-11 09:23:44 +02:00
Roland 'Quix0r' Haeder
edaa09b9b1 Rewrote all String blacklist types to enum 'BlacklistType', closes bug 
#143

Conflicts:
	htroot/Supporter.java
	htroot/yacy/crawlReceipt.java
	htroot/yacy/transferRWI.java
	htroot/yacy/transferURL.java
	source/de/anomic/crawler/CrawlStacker.java
	source/de/anomic/data/ListManager.java
	source/net/yacy/peers/Protocol.java
	source/net/yacy/repository/Blacklist.java
	source/net/yacy/repository/LoaderDispatcher.java
	source/net/yacy/search/Switchboard.java
	source/net/yacy/search/index/MetadataRepository.java
	source/net/yacy/search/index/Segment.java
	source/net/yacy/search/query/RWIProcess.java
	source/net/yacy/search/snippet/MediaSnippet.java
 2012-06-11 00:17:30 +02:00
Roland 'Quix0r' Haeder
213f006bf1 One is okay ... 
Conflicts:
	htroot/Trails.html
 2012-06-10 23:40:07 +02:00
Roland 'Quix0r' Haeder
af5a597e47 Scroogle is not comming back, remove dead code 
Conflicts:
	source/net/yacy/search/Switchboard.java
 2012-06-10 23:38:41 +02:00
cominch
7a4dab6d1d - removed unused variables 
- do not replace malformed or invalid URLs in urlproxy

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7835
6c8d7289-2bf4-0310-a012-ef5d649a1542

Conflicts:
	source/de/anomic/http/server/HTTPDFileHandler.java
 2012-06-10 23:33:09 +02:00
Michael Peter Christen
90c6fc4b63 load all - but not the persistent local.rdf - triples from 
DATA/TRIPLESTORE at startup time. The local.rdf is loaded only if the
persistent switch is on (as before).
 2012-06-10 21:49:02 +02:00
Michael Peter Christen
a9eb40c160 fix for autocomplete in index.html 2012-06-10 14:44:37 +02:00
Michael Peter Christen
dd020a1a8a removed autocrawler and feedback servlet link since that was not 
cherry-picked
 2012-06-10 13:17:23 +02:00
cominch
aa0295917c augmentation 
Conflicts:
	source/net/yacy/interaction/AugmentHtmlStream.java
 2012-06-10 13:10:21 +02:00
cominch
87a3fbb3c2 interaction javascript 2012-06-10 13:09:00 +02:00
cominch
ed2ea0f08e augmented browsing modification 
Conflicts:
	htroot/interaction/OverlayInteraction.html
	source/net/yacy/interaction/AugmentHtmlStream.java
 2012-06-10 13:07:57 +02:00
cominch
d4802dc8d5 small change 2012-06-10 13:02:30 +02:00
cominch
a120ef660b RDF demo servlet 2012-06-10 13:02:11 +02:00
cominch
09a34cfe1b prepare RDF dump routines 2012-06-10 12:58:40 +02:00
cominch
300b235ce8 Updated Demo Servlet 
Conflicts:
	htroot/About.html
	htroot/DemoServlet.html
	htroot/DemoServlet.java
	htroot/interaction/interaction.js
	source/net/yacy/interaction/Interaction.java
 2012-06-10 12:58:29 +02:00
cominch
90512640bf Added config switches for custom parser 
Conflicts:
	source/net/yacy/document/TextParser.java
 2012-06-10 12:49:36 +02:00
cominch
a12cbcba36 Add a global value store 2012-06-10 12:45:01 +02:00
cominch
e14f2881ae interaction: add special table interaction 
Conflicts:
	source/net/yacy/interaction/Interaction.java
 2012-06-10 12:41:16 +02:00
cominch
4e4e7a99f8 interaction: add global variable store 
Conflicts:
	source/net/yacy/interaction/Interaction.java
 2012-06-10 12:34:36 +02:00
cominch
bde07ed7a8 Add tagging overlay element 
Conflicts:
	htroot/env/templates/jqueryheader.template
	htroot/yacysearchitem.java
	source/net/yacy/interaction/Interaction.java
 2012-06-10 12:28:50 +02:00
cominch
bee3bee8f3 Small fix - return value of JSON should be empty 2012-06-10 12:20:13 +02:00
cominch
ff4ba3ee05 Small fix 
Conflicts:
	htroot/yacysearchitem.java
 2012-06-10 10:56:39 +02:00
cominch
f05e3968f7 Quick fix 2012-06-10 10:55:09 +02:00
cominch
e859481889 Add Triplestore settings functionality 
Conflicts:
	htroot/env/templates/header.template
 2012-06-10 10:55:00 +02:00
cominch
b0bc0b4572 Add new demonstration module for client-side key-value store (backend: 
triplestore): /DemoServletInteraction.html

Conflicts:
	source/net/yacy/interaction/Interaction.java
 2012-06-10 10:53:30 +02:00
cominch
c9dc6cda02 Demonstration: include value from interaction in search results 
Conflicts:
	htroot/interaction/OverlayInteraction.html
	htroot/yacysearchitem.java
 2012-06-10 10:51:53 +02:00
cominch
ae8adb0e58 Small changes 2012-06-10 10:44:16 +02:00
cominch
bcbd8eee33 Add several parsers, for RDFa and rdf files. 
Conflicts:
	source/net/yacy/document/TextParser.java
 2012-06-10 10:42:33 +02:00
cominch
9ef5a80f4e add interaction for triples and selector for augmented browsing 
Conflicts:
	htroot/interaction/interaction.js
	source/net/yacy/interaction/Interaction.java
 2012-06-10 10:38:54 +02:00
cominch
5d20cd324a Add Triplestore and RDF query interface 
Conflicts:
	build.xml
	defaults/yacy.init
	source/net/yacy/interaction/AugmentHtmlStream.java
 2012-06-10 10:35:59 +02:00
cominch
bc9a618e0a augmented browsing: ignore js and css, integrate more user interaction 
Conflicts:
	htroot/interaction/Footer.html
	source/net/yacy/interaction/AugmentHtmlStream.java
 2012-06-10 10:29:15 +02:00
cominch
9cbfc1a1c0 augmentedProxy, which forwards every proxy request to a 
rewrite engine to customize existing webpages. originally implemented by
Florian Richter.

Conflicts:
	source/de/anomic/http/server/HTTPDProxyHandler.java
 2012-06-10 10:15:34 +02:00
cominch
1626be7916 Add menu entries for urlproxy / augmented browsing 2012-06-10 09:59:30 +02:00
Michael Peter Christen
5b25272f40 added location search to main menu 2012-06-09 09:10:54 +02:00
Michael Peter Christen
ea0dceb55d bugfix: do not switch off standard memory strategy when performing a 
forced GC
PLEASE CHECK if your peer has standard memory switched on!
 2012-06-08 09:48:46 +02:00
Michael Peter Christen
dd14b19c26 lazy initialization of block rank table ... only normal web search uses 
this. When interactive search or location search is used, the block rank
is switched off
 2012-06-08 09:41:29 +02:00
Michael Peter Christen
701b9a28a0 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 
Conflicts:
	htroot/PerformanceMemory_p.java
 2012-06-08 09:16:16 +02:00
Michael Peter Christen
ab7107b34b fixed RWIProcess queue limits: now discovering hidden results for mass 
result retrieval
 2012-06-08 09:14:54 +02:00
Michael Peter Christen
10c9c17d51 fixed handlemap spread factor and null iterator handling 2012-06-08 09:13:41 +02:00
Michael Peter Christen
a61f44f9e4 lazy initialization of block rank table. 
this causes that the table is not initialized when there is no search is
done. the effect is most strong if YaCy is started headless which causes
no browser pop-up which otherwise would load the search page and
therefore trigger the initialization of the table.
 2012-06-07 13:16:38 +02:00
Michael Peter Christen
c8bbd180e4 enhanced hint for debian package automatic update 2012-06-07 12:36:26 +02:00
Michael Peter Christen
9ad84c5e9f fix for NPE in PerformanceMemory 2012-06-07 12:36:05 +02:00
Michael Peter Christen
96e9d77270 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 
Conflicts:
	source/net/yacy/cora/sorting/WeakPriorityBlockingQueue.java
 2012-06-06 20:13:28 +02:00
Michael Peter Christen
d7eb18cdf2 accept also file names beginning with "file://" for crawl start from 
file.
 2012-06-06 14:27:18 +02:00
Michael Peter Christen
3dd8376825 added automatic cleaning of cache if metadata and file database size is 
not equal. It might happen that these data is different because one of
that caches is cleaned after a while or when it is too big. The metadata
is then not cleaned, but now wiped after a checkup process at every
application start. This should cause a bit less memory usage.
 2012-06-06 14:15:24 +02:00
Michael Peter Christen
d0ec8018f5 fixes for bad long computation 2012-06-06 14:13:31 +02:00
Michael Peter Christen
96c8119b50 added GeoLocation / GeoPoint classes which uses less memory than 
Location/Coordinates and has initializers with correct order of lat,lon
coordinates
 2012-06-06 12:57:42 +02:00
Michael Peter Christen
461a0ce052 removed warnings 2012-06-05 20:03:43 +02:00
Michael Peter Christen
62ae9bbfda allow more POIs, get more at once 2012-06-05 18:29:54 +02:00
Michael Peter Christen
a1fe65b115 performance hacks 2012-06-05 12:06:26 +02:00
Michael Peter Christen
2fe207f813 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-06-04 23:44:38 +02:00
Michael Peter Christen
5aee19daa4 added show from cache in search results (not yet finished) 2012-06-04 23:44:26 +02:00
Michael Peter Christen
e0d8643226 - performance hacks 
- added log warnings in case that search processes run into time-out
situations
- better concurrency for Integer formatter (used a non-synchronized
formatter before)
- bugfix for search termination (a poison pill was missing)
- added timeout parameters for search (again) -> target is, that they
are never reached.
 2012-06-04 15:37:39 +02:00
Michael Peter Christen
cf79b6cee3 Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2012-06-01 08:32:26 +02:00
Michael Peter Christen
6e83b02b83 - bugfix for surrogate file reader 
- bugfix for location search: suppress empty search
 2012-06-01 00:08:31 +02:00
Michael Peter Christen
9b4c699526 ehanced location search: 
- search request are now made using a map boundary
- search results are only computed for the map boundary
- the number of results is adopted to the results in the visible range
- added a double-buffering for the search result markers
- added a search query option for the search results:
/radius/<lat>/<lon>/<radius>
 2012-05-31 22:39:53 +02:00
Michael Peter Christen
434af404c1 - added double-buffering for search layers 
- added automatic zooming to search result
to location search
 2012-05-31 14:05:36 +02:00
Michael Peter Christen
4d9b2dc487 automatically zoom to result layer bounds 2012-05-31 01:12:06 +02:00
Michael Peter Christen
6b40803adf - show number of results in map search interface 
- transfer view radius within query
 2012-05-31 00:47:52 +02:00
Michael Peter Christen
a8778e9c47 npe fix 2012-05-30 15:28:45 +02:00
Michael Peter Christen
1a6fab60e0 added node state to xml 2012-05-30 09:32:25 +02:00
Michael Peter Christen
20e0cc0822 fix for bad location evaluation 2012-05-29 14:46:13 +02:00
Michael Peter Christen
1ab3de0885 fixes to location search 2012-05-29 12:43:14 +02:00
Michael Peter Christen
f167a1c69f removed osmarender from yacysearch_location because that caused a 
javascript error
 2012-05-29 02:22:02 +02:00
Michael Peter Christen
71c3163f3d - fixes to node identification 
- added link to node in network list
- added marking of portal search node peers
 2012-05-29 01:38:54 +02:00
Michael Peter Christen
d1e9fe3db5 enhanced RootState icon 2012-05-29 00:06:33 +02:00
Michael Peter Christen
ad222be7f8 added node state icon in network list 2012-05-25 17:29:54 +02:00
Michael Peter Christen
638390930d another patch to fix the Crawler_p layout 2012-05-25 15:56:21 +02:00
Michael Peter Christen
c846e9ca14 redesign of the crawler monitor page: show crawled pages instead of 
queue of urls that shall be crawled
 2012-05-25 01:45:38 +02:00
Michael Peter Christen
8b974905ee changed log-in text for all servlets with authentication: 
- added hint how to set the password using a shell script
- added a shell script to change the password
 2012-05-24 13:24:31 +02:00
Michael Peter Christen
16b21f7a5b Added more steering in Crawler_p.html interface 2012-05-23 18:00:37 +02:00
Michael Peter Christen
c15fcde1c8 add-on to latest commit 2012-05-21 17:52:30 +02:00
Michael Peter Christen
cf47d94888 performance hack to parse numbers inside of substrings without actually 
generating a substring. This avoids the allocation of a String object
ech time a substring is parsed. Should affect CPU load during RWI
transmission.
 2012-05-21 13:40:46 +02:00