yacy_search_server/source/net/yacy/repository
Michael Peter Christen 8df8ffbb6d enhanced the snapshot functionality:
- snapshots can now also be xml files which are extracted from the solr
index and stored as individual xml files in the snapshot directory along
the pdf and jpg images
- a transaction layer was placed above of the snapshot directory to
distinguish snapshots into 'inventory' and 'archive'. This may be used
to do transactions of index fragments using archived solr search results
between peers. This is currently unfinished, we need a protocol to move
snapshots from inventory to archive
- the SNAPSHOT directory was renamed to snapshot and contains now two
snapshot subdirectories: inventory and archive
- snapshots may now be generated by everyone, not only such peers
running on a server with xkhtml2pdf installed. The expert crawl starts
provides the option for snapshots to everyone. PDF snapshots are now
optional and the option is only shown if xkhtml2pdf is installed.
- the snapshot api now provides the request for historised xml files,
i.e. call:
http://localhost:8090/api/snapshot.xml?urlhash=Q3dQopFh1hyQ
The result of such xml files is identical with solr search results with
only one hit.
The pdf generation has been moved from the http loading process to the
solr document storage process. This may slow down the process a lot and
a different version of the process may be needed.
2014-12-09 16:20:34 +01:00
..
Blacklist.java refactore URIMetadataNode to further unify interaction with index 2014-04-20 01:41:30 +02:00
BlacklistFile.java Rewrote all String blacklist types to enum 'BlacklistType', closes bug 2012-06-11 00:17:30 +02:00
BlacklistHelper.java First draft of a blacklist API. 2014-04-30 00:48:38 +02:00
FilterEngine.java - the webgraph shall store all links which appear on a web page and not 2013-09-15 00:30:23 +02:00
LoaderDispatcher.java enhanced the snapshot functionality: 2014-12-09 16:20:34 +01:00
RegexHelper.java avoid NPE in regex checker 2013-04-20 10:53:49 +02:00