Commit Graph

6894 Commits

Author SHA1 Message Date
orbiter
2971c91988 fix for http://forum.yacy-websuche.de/viewtopic.php?p=20977#p20977
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7228 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-10-08 21:44:28 +00:00
orbiter
091dd3f6ec - enhanced intranet search speed
- enhanced intranet portscan speed (better time-out)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7227 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-10-08 10:54:13 +00:00
low012
b9f405d1e8 *) added comments
*) more beautyful and easier to understand code (IMO)
*) added display= parameter to a lot of links in Wiki.html

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7226 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-10-08 00:32:50 +00:00
mikeworks
70576e88d2 de.lng: Added some more untranslated strings I found and uncommented old ones that were removed
terminal_p.html: Put back the old ID which was really easy to find
IndexCreate.js: Because XHTML 1.0 Strict does not allow name tags for some elements rewrote most element access functions to use getElementById
Table_API_p.html and all other html pages: Some XHTMl 1.0 Strict fixes, changed checkAll javascript, marked the first row with checkboxes as unsortable where applicable
Table_API_p.java and all other java pages: URLencoded lines with possible ampersands & -> & for validation XHTML 1.0 Strict sourcecode
--> All Index Create pages should validate now. Hope I did not break anything else (too much :-)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7225 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-10-06 00:00:23 +00:00
orbiter
efa59250f8 release 0.98 for SuMa-eV Demo tomorrow
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7224 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-10-05 17:52:54 +00:00
orbiter
6e6994e328 latest bugfixes to search and indexing function after test of demo presentation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7223 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-10-05 17:49:53 +00:00
orbiter
c3bf17a3a1 fixed must-match filter for smb crawling
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7222 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-10-05 00:05:08 +00:00
orbiter
099def2a04 small changes in search widget appearance
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7221 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-10-04 23:43:33 +00:00
orbiter
50586a0dfd rename of widget to 'widget'
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7220 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-10-04 19:57:29 +00:00
apfelmaennchen
dffa142529 Fix for author navigator in yacyui-portalsearch.js
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7219 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-10-04 19:27:12 +00:00
orbiter
574346f8ce better must-match pattern for intranet file-crawls
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7218 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-10-04 12:55:39 +00:00
orbiter
aacf572a26 - enhancements for search speed
- bug fixes in many classes including basic data structure classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7217 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-10-04 11:54:48 +00:00
sixcooler
aa6075402a smal fix for crawling from 'sitelist' at changes from 7214
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7216 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-10-01 22:41:28 +00:00
sixcooler
61c82f3105 gzip-compresson @ transferRWI & transferURL back again
This reduce upload-volume to suit limited bandwidth of home-users like me :-)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7215 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-10-01 00:42:43 +00:00
orbiter
2c549ae341 fixed a number of small bugs:
- better crawl star for files paths and smb paths
- added time-out wrapper for dns resolving and reverse resolving to prevent blockings
- fixed intranet scanner result list check boxes
- prevented htcache usage in case of file and smb crawling (not necessary, documents are locally available)
- fixed rss feed loader
- fixes sitemap loader which had not been restricted to single files (crawl-depth must be zero)
- clearing of crawl result lists when a network switch was done
- higher maximum file size for crawler

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7214 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-30 23:57:58 +00:00
orbiter
f6eebb6f99 replaced auto-dom filter with easy-to-understand Site Link-List crawler option
- nobody understand the auto-dom filter without a lenghtly introduction about the function of a crawler
- nobody ever used the auto-dom filter other than with a crawl depth of 1
- the auto-dom filter was buggy since the filter did not survive a restart and then a search index contained waste
- the function of the auto-dom filter was in fact to just load a link list from the given start url and then start separate crawls for all these urls restricted by their domain
- the new Site Link-List option shows the target urls in real-time during input of the start url (like the robots check) and gives a transparent feed-back what it does before it can be used
- the new option also fits into the easy site-crawl start menu

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7213 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-30 12:50:34 +00:00
mikeworks
63e387508c ConfigLanguage_p.java: Fixed the filename for the API call to ConfigLanguage_p.html - previously ConfigLanguage.html was recorded and the action could not be replayed with error 404 - Not found
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7212 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-30 03:16:17 +00:00
mikeworks
f468d377d7 Collage.html and Collage.java: Added <p> in body before images for XHTML 1.0 Transitional validationg and alt tag to images as well as closing tag <img (...) />
terminal_p.html: Set new link for starting a crawl to CrawlStartSite_p.html and replaced the old embed object of the Among.us Flash object by their new JS which takes care of adding the object correctly
de.lng: Moved the translations for the JS part from yacyinteractive.html to the yacyinteractive.js part
--> Terminal page is now valid XHTML 1.0 Transitional


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7211 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-29 23:11:29 +00:00
orbiter
3057a0b939 - intranet scanner now produces urls with host names, not ips if possible
- CrawStartIntranet servlet shows IPs and host names

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7210 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-29 22:44:49 +00:00
orbiter
75964909aa added missing path to htroot (may only be necessary for cross-linking of servlet classes)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7209 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-29 22:19:41 +00:00
mikeworks
421aa6a8bb ConfigLiveSearch.html: Fixed some HTML problems to validate at least XHTML 1.0 Transitional - strict is not possible because iframes are used. Replacing iframes with embedded object tag does not work in IE
ConfigPortal.html: Fixed some HTML problems to validate at least XHTML 1.0 Transitional - for strict the target attribute of the a link has to be removed
yacyinteractive.html: Moved all JS code to an external yacyinteractive.js file in JS folder
yacysearch.html: Removed embedded scripts from in between the body tags - now everything is loaded in the header
de.lng: Just in case JS files will be parsed at some point added translation for yacyinteractive.html result counter

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7208 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-29 20:06:39 +00:00
orbiter
c60aed4435 no caching in browser of dynamic web pages sent by YaCy http
this may prevent unnecessary IO caused by cache storage of the browser

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7207 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-29 19:56:42 +00:00
mikeworks
41a93ff565 de.lng: Small changes to the German translation and added missing part in Surftips.html and YaCy statistics link in left menu in header.template
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7206 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-29 06:08:19 +00:00
mikeworks
b7bb0cabaf Blacklist_p.html: Minor HTML and Javascript changes to get XHTML 1.0 Strict validation, lowercae onchange, id tags instead of name tags
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7205 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-29 05:41:32 +00:00
mikeworks
cd505d7e30 de.lng: German translation of the new Intranet Servlet introduced in SVN 7203 in CrawlStartIntranet_p.html
CrawlStartIntranet_p.html: New Intranet Crawl Start Servlet - minor HTML changes to get XHTML 1.0 Strict validation, remove (double) name tags, remove single ending </dt>

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7204 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-28 22:05:12 +00:00
orbiter
e63896f2a8 added an intranet scanner and a servlet which shows all intranet addresses and an option to start a site-crawl for all these addresses at once.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7203 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-28 12:18:54 +00:00
orbiter
e54cb7fb0c more bugfixes (also for latest commit)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7202 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-28 10:20:46 +00:00
orbiter
be6b48311c misc bugfixes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7201 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-28 10:00:33 +00:00
suessthomas
44874f2cb9 Added "encoding =" UTF-8 "in the RSS files
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7200 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-27 20:36:49 +00:00
orbiter
d2fd93135c - moved yacybot user agent string definition to MultiProtocolURI since there are basic access mechanisms where the bot string is needed
- migrated the 'yacy' user agent to 'yacybot' in many client methods since the 'yacy' user agent is only used for the proxy

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7199 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-27 14:54:32 +00:00
orbiter
fc3ee9d8a1 changed paths for new libs also in Info.plist
when changing paths in build.xml, please also update the paths in /addon/YaCy.app/Contents/Info.plist

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7198 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-27 14:49:51 +00:00
sixcooler
f4357dff03 bump to httpclient-4.0.3 which fixes a number of bugs
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7197 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-27 13:24:40 +00:00
mikeworks
ad7efe6016 rssTerminal.html: Fixing the 'null' is null or not an object in rss2.js when viewing the YaCy default Status page http://localhost:8080/Status.html with Internet Explorer
feed.xml: copy of feed.rss that helps Internet Explorer also read the Feed - workaround for the fix above
Problem is described in the forums and should be fixed better ;-(http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2766&p=20702)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7196 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-26 22:55:52 +00:00
mikeworks
190de644dd de.lng: Added German translations for some missing table content on Network view
WatchWebStructure_p.html: Added JS verification of RGB color codes (currently only RGB value is checked but this could be enhanced to also check for websafe colors)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7195 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-26 22:28:32 +00:00
orbiter
d5dc88a351 shop cleanup button only if servlet was called without post/put arguments.
This should avoid confusion after a search for a word where it is possible to delete the word. If a delete button is shown to delete the word, then there should not be a button available to delete the whole index to avoide a wrong usage when a user searches only for a word to delete it.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7194 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-26 21:11:08 +00:00
low012
afa708d552 *) added <s>...</s> tag to WikiCode -> works just as the HTML equivalent
*) code changes (PMD) without functional changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7193 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-26 12:57:07 +00:00
orbiter
a83186ac7d fix for bug in cytrails
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7192 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-26 10:32:40 +00:00
mikeworks
b019426811 de.lng: Added German translations for new Index Creation pages RSS Feeds and adapted text in Tables_p.html and CrawlStartExpert_p.html to match some typos, also changed one name tag to id to conform with XHTML 1.0 Strict
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7191 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-26 01:39:51 +00:00
orbiter
48c0d508ac fixes for crawling of smb links (file length not always available)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7190 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-25 22:32:26 +00:00
mikeworks
965a40b623 de.lng: Added some German translation for Admin Console -> Advanced Settings and started to add some translations for new Index Creation pages
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7189 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-25 21:53:13 +00:00
orbiter
0bc6284e27 - added bugfix for access tracker in case of concurrency conflicts
- added missing entry for new icu4j path in Mac App

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7188 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-25 21:10:50 +00:00
f1ori
6295cac35a set reasonable mime-type for icu4j-core.jar
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7187 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-25 20:04:34 +00:00
f1ori
e670e1ef8e add charset auto-detection for htmlParser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7186 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-25 18:02:21 +00:00
f1ori
ddcd5ae78c fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2989
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7185 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-25 15:07:25 +00:00
f1ori
8fe1102452 fix http://forum.yacy-websuche.de/viewtopic.php?p=20889#p18426
reuse code from htmlParser

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7184 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-24 13:14:20 +00:00
orbiter
daeea96aea renamed servlet CrawlStart_p.html to CrawlStartSite_p.html to circumvent problem with translation which still showed old expert crawl start page
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7183 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-22 21:46:31 +00:00
orbiter
10a9cb1971 simplified snippet computation process and separated the algorithm into two classes
also enhances selection criteria for best snippet line computation

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7182 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-22 20:50:02 +00:00
lotus
4450c240b7 npe fix http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2982
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7181 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-22 20:24:07 +00:00
orbiter
84a023cbc8 fixed several search bugs
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7180 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-21 21:48:42 +00:00
lotus
937dd956d3 save default number of search items via web interface
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7179 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-21 19:45:49 +00:00