Commit Graph

3194 Commits

Author SHA1 Message Date
orbiter
fc2e41e691 added a forwarder for the default page. The forwarder forwards a browser to a different page if the root file index.html is accessed. This can be done by setting the name of the forwarder page to the field
"Default index.html Page (by forwarder)" in /ConfigPortal.html
The purpose is to forward to /yacyinteractive.html for the 27C3 FTP search plattform

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7365 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-12-07 15:46:04 +00:00
orbiter
db99db4be9 some redesign of the search-fail-response mechanism:
when a search fails for a single url because the snippet cannot be generated, then the url reference is deleted from the index. This mechanism was redesign and enhanced. The process now also writes into the work tables into the table searchfl to prepare a re-indexing mechanism.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7364 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-12-06 14:34:58 +00:00
f1ori
4915d1781a * use local backup-file, if remote network-definition is not availible
* resolve single point of failure in networks, managed by central network-definitions

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7363 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-12-06 11:09:16 +00:00
orbiter
18d33b5c6d fixed several search result navigation bugs
fixed bad behaviours during search result collection

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7362 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-12-05 23:54:00 +00:00
orbiter
49b5a206cd - better caclculation of search result size
- predefined search recommendations

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7361 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-12-02 12:19:59 +00:00
orbiter
f0651e5f2f added image search to yacyinteractive.html
this causes that the search result view switches from list format to image preview format when a search is restricted to png, gif or jpg documents

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7358 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-12-01 18:48:21 +00:00
orbiter
fffb91447a fixed crawl queue delete function
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7357 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-12-01 14:55:40 +00:00
orbiter
4e771e2063 enhanced interactive search:
- better table design
- less enumeration of same table structure (prepared now for streaming)
- added a 'remove filetype' link

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7356 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-12-01 14:43:07 +00:00
orbiter
b769cce433 - added a catch-all parser for all documents that cannot be parsed: they will contributed with their document url for the search index only
- enhanced the pdf and torrent parser: better documents titles
- enhanced the ftp client: more time-out time
- fixed bugs in json for search results
- enhanced yacyinteractive.html: added a file type navigator and a download-script generator for search result files

Please have a look at yacyinteractive.html: this will become the hacker-download tool for 27c3!

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7355 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-30 16:13:55 +00:00
lotus
6692e862ae do not reset language on config change
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7354 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-29 20:09:53 +00:00
orbiter
cc6499bf8d - added http://blekko.com as search heuristic (like scroogle). This was easy since they deliver their search results also as rss feed
- renamed YaCys search result modifications keywords for RECENT, NEAR and language: to the blekko slashtag naming scheme. YaCy now supports the following blekko-like slash built-in slashtags:
/date
 - for search results ordered by date (most recent up)
 /near
 - for search results where search words appear near to each other (closest up)
 /language/<lang>
 - for a sorting by language where the wanted language gets up. Example: /language/de
  

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7350 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-29 18:08:20 +00:00
orbiter
a9f754c45f removed unused CR accumulation and distribution process
this was never used and extended in the last years. The resulting YBR ranking criteria
is still a good idea and will be used in the future. Possible generation methods for YBR
ranking are:
- "trust-rank" using the link structure as can be discovered in a single crawl (idea from FSCONS)
- "block-rank" calculated from the local link structure
- a distributed "block-rank" using the xml API to the link structure from other peers

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7349 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-29 11:07:42 +00:00
orbiter
d4a1a1850b removed warnings
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7347 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-29 07:52:10 +00:00
low012
9b3fae9496 *) cleaning up the code a little bit
*) program to interface, not implementation

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7345 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-28 02:57:31 +00:00
orbiter
7bb4b001ed - view image files from cache
- fixed generic header settings; affects CORS functionality

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7344 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-27 09:16:16 +00:00
low012
e7552bd719 *) cleaning up the code a little bit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7343 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-27 00:54:59 +00:00
low012
a9741cc876 *) HTML fixes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7342 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-26 22:38:08 +00:00
low012
01ddb6d2ef *) HTML fixes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7341 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-26 22:19:31 +00:00
apfelmaennchen
737aaf6952 various small changes to ymarks
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7339 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-25 21:16:47 +00:00
apfelmaennchen
8a50670546 some code clean up for the last post
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7338 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-24 23:40:55 +00:00
apfelmaennchen
442497868d another step towards an auto tagging function for YMarks
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7337 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-24 23:26:29 +00:00
low012
dad5818b40 *) cleaning up the code a little bit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7336 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-24 01:31:41 +00:00
low012
9057e4d58c *) hopefully fixed bug described in http://www.yacy-forum.org/viewtopic.php?f=12&t=385
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7335 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-23 23:10:45 +00:00
low012
eb79b952ef *) cleaner code
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7331 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-21 03:39:53 +00:00
low012
38fdf43587 *) renamed classes according to standard Java coding conventions
*) String.isEmpty() was introduced in Java 1.6, but we still use Java 1.5

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7330 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-21 01:29:32 +00:00
low012
8281d12305 *) Ooops!
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7329 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-21 00:41:45 +00:00
apfelmaennchen
54e63b556e intermediate step for a YMark auto-tagging function based on word frequencies.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7325 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-17 15:17:29 +00:00
apfelmaennchen
403ee9c014 added a drill-down for metadata and word count to /api/ymarks/test_treeview.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7324 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-16 00:48:38 +00:00
mikeworks
28a290336d de.lng: Started translation of YMark feature here http://localhost:8080/Table_YMark_p.html
Table_YMark_p.html: removed some labels because columns is not filled yet and added ids for other label references and added full enabled="enabled" tags because SHORTTAG is not specified in XHTML 1.0 Strict

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7322 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-15 01:48:54 +00:00
lotus
25426c6548 change language immediately
there is a bad behaviour:
if language is changed, servlets are translated but the returning servlet is yet the old one. therefore the page is loaded again after language changed.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7321 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-13 14:27:30 +00:00
apfelmaennchen
11ae5b108e enabled rebuildIndex for /Table_YMark_p.html (rebuilds the tags and folders index)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7320 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-13 13:02:56 +00:00
apfelmaennchen
f147a022f8 enabled YMark Import for /Table_YMark_p.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7319 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-13 10:32:37 +00:00
apfelmaennchen
2726606fc8 slightly enhanced interface for /Table_YMark_p.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7318 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-12 21:13:27 +00:00
apfelmaennchen
94a9be18a4 added a ymark table administration: /Table_YMark_p.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7316 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-10 22:53:27 +00:00
apfelmaennchen
25339f93c7 more updates to ymarks
- working xbel import/export
- exported xbel includes yacy specific metadata but still validates against PUBLIC DTD


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7315 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-09 17:01:31 +00:00
apfelmaennchen
cdd65aca71 update to ymarks
- get_xbel.xml is almost working
- startet ymark api documentation info.html

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7313 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-07 20:03:01 +00:00
apfelmaennchen
808edffaf6 ymarks
- some refactoring
- working xbel and html import (/api/ymarks/test_import.html)
- working treeview (/api/ymarks/test_treeview.html)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7312 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-06 20:26:13 +00:00
f1ori
2c539b514a * add domaincheck (local/global/domainlist) to urlcleaner
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7311 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-06 16:50:33 +00:00
mikeworks
274d5b3a87 de.lng: Added missed translation string in SVn 7301
ConfigHTCache_p.html: Added missing id for label pointing to actualCacheSize in span tag for XHTML 1.0 Strictness

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7302 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-04 07:30:00 +00:00
mikeworks
9239ac1e56 de.lng: Added translation for new page http://localhost:8080/ConfigHTCache_p.html and old one http://localhost:8080/IndexControlRWIs_p.html
ConfigHTCache_p.html: Removed additional </form> and changed title text

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7301 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-04 00:53:25 +00:00
apfelmaennchen
43586a2ace a update to ymarks (please test if you wish):
- import HTML (e.g. FF export) via /api/ymarks/import.html
- view your import via /api/ymarks/test.html
- get a xml list via /api/ymarks/get_ymark_list.xml?tags=&folders=
- delete bookmark tables via standard interface /Tables_p.html
it is still very experimental!! 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7299 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-03 22:52:03 +00:00
orbiter
790e0b1894 - enhanced index deletion in IndexControlRWIs_p: delete also robots.txt database and cache if demanded
- added option for details of deletion
- added deletion to new ConfigHTCache_p servlet

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7294 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-03 18:31:36 +00:00
orbiter
3e303db1fa added search suggestions in XML format. This also supports CORS
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7293 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-03 09:20:36 +00:00
apfelmaennchen
f5324b27f2 more updates to the new bookmarks (ymarks)....
- split YMarkTables and YMarkIndex in two different classes
- HTML import is working properly
- XBEL import is still broken


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7292 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-03 06:47:02 +00:00
orbiter
445619f3ec added a submenu ConfigHTCache_p.html to set the size of the HTCache separately from the proxy configuration.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7291 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-02 23:57:11 +00:00
orbiter
70c95608d4 Added CORS Access header for yacysearch.rss output
used some of the recommendations from Copro:
http://forum.yacy-websuche.de/viewtopic.php?p=21015#p21015
Original Request:
http://forum.yacy-websuche.de/viewtopic.php?p=20829#p20829

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7288 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-11-02 16:28:40 +00:00
apfelmaennchen
efe0667fdd more new bookmark (ymarks) code with experimental html and xbel import
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7281 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-10-28 15:24:15 +00:00
mikeworks
caabebf9be Fixed spelling mistake omiting -> omitting in debug messages in ConfigUpdate_p.java and Switchboard.java
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7280 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-10-28 04:03:11 +00:00
orbiter
155d556568 - better memory protection
- more logging
- little bit of refactoring

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7278 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-10-27 13:21:18 +00:00
mikeworks
a16e5e7a63 ConfigPortal.html: Added safe space character because empty <dt> produces a warning in HTML validator
de.lng: Added translation for added ConfigPortal selection for target selection and WikiHelp changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7277 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-10-26 21:14:37 +00:00