Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
6,989 Commits 1 Branch 20 Tags 836 MiB
a025b1da89
Commit Graph

4642 Commits

Author SHA1 Message Date
f1ori
a025b1da89 * fix bug when browsing local filesystem (e. g. repository) with yacy 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7323 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-15 14:47:16 +00:00
apfelmaennchen
11ae5b108e enabled rebuildIndex for /Table_YMark_p.html (rebuilds the tags and folders index) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7320 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-13 13:02:56 +00:00
sixcooler
b87bf88ac8 using less memory on merging and rewriting blobs 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7317 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-12 16:02:20 +00:00
apfelmaennchen
94a9be18a4 added a ymark table administration: /Table_YMark_p.html 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7316 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-10 22:53:27 +00:00
apfelmaennchen
25339f93c7 more updates to ymarks 
- working xbel import/export
- exported xbel includes yacy specific metadata but still validates against PUBLIC DTD


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7315 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-09 17:01:31 +00:00
f1ori
d62e449a11 * fix FilterEngine, forgot comparision-operator 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7314 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-08 09:37:44 +00:00
apfelmaennchen
cdd65aca71 update to ymarks 
- get_xbel.xml is almost working
- startet ymark api documentation info.html

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7313 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-07 20:03:01 +00:00
apfelmaennchen
808edffaf6 ymarks 
- some refactoring
- working xbel and html import (/api/ymarks/test_import.html)
- working treeview (/api/ymarks/test_treeview.html)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7312 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-06 20:26:13 +00:00
f1ori
2c539b514a * add domaincheck (local/global/domainlist) to urlcleaner 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7311 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-06 16:50:33 +00:00
orbiter
117fc86b3d fix for http://forum.yacy-websuche.de/viewtopic.php?p=21199#p21199 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7308 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-05 13:19:37 +00:00
orbiter
441fbc26e2 security patch for WeakPriorityBlockingQueue (produced a deadlock) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7307 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-05 09:38:31 +00:00
orbiter
5dcb838293 - removed thread overhead when calling dns services 
- fixed localsearch (changed it by accident)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7306 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-05 00:29:32 +00:00
orbiter
4c50d3428e smaller file size for array stacks to support smaller deletion sizes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7305 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-04 13:29:19 +00:00
orbiter
09badc697b - low-memory patch for crawler 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7304 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-04 13:26:27 +00:00
orbiter
becc463d8a enhanced did-you-mean 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7300 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-04 00:25:19 +00:00
apfelmaennchen
43586a2ace a update to ymarks (please test if you wish): 
- import HTML (e.g. FF export) via /api/ymarks/import.html
- view your import via /api/ymarks/test.html
- get a xml list via /api/ymarks/get_ymark_list.xml?tags=&folders=
- delete bookmark tables via standard interface /Tables_p.html
it is still very experimental!! 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7299 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-03 22:52:03 +00:00
orbiter
93c535d111 fixed http://forum.yacy-websuche.de/viewtopic.php?p=21113#p21113 
fixed a concurrent modification exception during search and a time-out problem

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7298 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-03 20:58:50 +00:00
orbiter
04932dc268 added rdf data structure for rss feeds 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7297 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-03 20:06:23 +00:00
orbiter
84f2953cd8 fix for rss loader / rss type recognition 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7296 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-03 19:58:01 +00:00
orbiter
4c72885cba added a sitemap entry parser and loader for sitemaps 
(a recursion if a sitemap refers to another sitemap)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7295 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-03 19:48:33 +00:00
orbiter
790e0b1894 - enhanced index deletion in IndexControlRWIs_p: delete also robots.txt database and cache if demanded 
- added option for details of deletion
- added deletion to new ConfigHTCache_p servlet

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7294 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-03 18:31:36 +00:00
apfelmaennchen
f5324b27f2 more updates to the new bookmarks (ymarks).... 
- split YMarkTables and YMarkIndex in two different classes
- HTML import is working properly
- XBEL import is still broken


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7292 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-03 06:47:02 +00:00
orbiter
445619f3ec added a submenu ConfigHTCache_p.html to set the size of the HTCache separately from the proxy configuration. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7291 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-02 23:57:11 +00:00
sixcooler
85c65475fa smal but important correction of last commit @ HTTPClient 
(if there is a response it really should be taken to its end)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7290 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-02 21:18:44 +00:00
f1ori
acd93b1b31 * add failsafe mechanisme to domainlist retrieval 
domainlist is saved locally, if none of the given urls in network.unit.domainlist
  could be retrieved, the file from the last boot is used instead

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7289 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-02 17:57:48 +00:00
orbiter
70c95608d4 Added CORS Access header for yacysearch.rss output 
used some of the recommendations from Copro:
http://forum.yacy-websuche.de/viewtopic.php?p=21015#p21015
Original Request:
http://forum.yacy-websuche.de/viewtopic.php?p=20829#p20829

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7288 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-11-02 16:28:40 +00:00
lotus
18729351e7 upnp: hint for wrongly detected local ip address 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7286 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-31 20:05:59 +00:00
f1ori
def4253555 * add option to network definition to provide a domainlist (syntax like in blacklists) 
* crawler and search allow only urls matching one in domainlist (if list is provided)
* this may be useful to prevent dedicated networks from being "polluted"
* FilterEngine is improved Backlist-object, Blacklist may inherit from FilterEngine in the future

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7285 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-30 14:44:33 +00:00
orbiter
ac6b503adf untar files without gzip decompression even if the file has gz extension. this is done when the decompression fails. 
decompressed gzip files with gz extension may appear if the server sets a gzip compression header

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7282 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-28 23:12:33 +00:00
apfelmaennchen
efe0667fdd more new bookmark (ymarks) code with experimental html and xbel import 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7281 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-28 15:24:15 +00:00
mikeworks
caabebf9be Fixed spelling mistake omiting -> omitting in debug messages in ConfigUpdate_p.java and Switchboard.java 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7280 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-28 04:03:11 +00:00
orbiter
fb92f9ae8e added mime type image/jpeg (image/jpg is wrong but it is left here because it does not harm and this error also exists in configuration of web servers) 
see also:
http://forum.yacy-websuche.de/viewtopic.php?p=21129#p21129

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7279 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-27 21:53:11 +00:00
orbiter
155d556568 - better memory protection 
- more logging
- little bit of refactoring

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7278 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-27 13:21:18 +00:00
f1ori
7d8de34778 * add a bit documentation to DigestURI, use DigestURI(string) instead of DigestURI(string, null) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7276 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-26 16:10:20 +00:00
orbiter
25a8e55bc9 more logging about bad seeds 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7275 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-26 15:00:22 +00:00
orbiter
959b8c6fa0 - allow greater seed size 
- more logging for bad seeds

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7274 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-26 14:40:19 +00:00
orbiter
e103419a56 - removed <3 peers barrier for peer ping feedback 
- more logging

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7273 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-26 13:08:09 +00:00
apfelmaennchen
d0e6c03b51 some updates to the new bookmark code... 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7272 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-25 22:44:05 +00:00
orbiter
facfd204e9 added a parent configuration option. 
see /ConfigPortal.html
requested here:
http://forum.yacy-websuche.de/viewtopic.php?p=21099#p21099

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7271 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-25 22:16:07 +00:00
orbiter
e3964f2c31 better catch of network definition load error; continue with secondary network load definition location 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7270 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-25 09:20:45 +00:00
low012
65a0381f76 *) cleaning up code (still not done) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7267 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-24 23:57:07 +00:00
orbiter
e3e3b49d52 - enhanced main release recognition 
- yacybot user agent now includes the yacy network name (not the peer name!)
- refactoring and clean-up (mostly turned tab into spaces)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7266 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-24 21:43:01 +00:00
apfelmaennchen
9c94ebdee4 small changes to new bookmark code... 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7265 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-22 13:14:09 +00:00
apfelmaennchen
244b56e9d3 an update to the new bookmark code... 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7264 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-21 19:18:17 +00:00
low012
dc40f51b8d *) added headlines as proposed by Vega 
*) <pre> will be displayed monospaced in wiki and blog again
*) bugfix for <pre> spanning multiple lines
*) replaced deprecated <s> tag with <span> equivalent

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7262 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-18 23:00:43 +00:00
apfelmaennchen
f035f257da added some more bookmark code... 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7261 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-18 21:09:41 +00:00
low012
22ed9c380c *) fixed bug which was introduced in r7226 (shame on me) which made wiki unusable (all entries were stored with empty subject as key -> edits were lost) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7260 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-18 21:06:23 +00:00
f1ori
60fd2e549d * log failures when writing config file 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7259 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-18 15:00:29 +00:00
orbiter
58e74282af added a word counter statistic in condenser which is used by the did-you-mean to calculate best matches for given search words. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7258 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-18 11:35:09 +00:00
orbiter
863065abc4 added user agent logging to access tracker 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7256 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-10-18 08:09:59 +00:00