Commit Graph

3092 Commits

Author SHA1 Message Date
mikeworks
63e387508c ConfigLanguage_p.java: Fixed the filename for the API call to ConfigLanguage_p.html - previously ConfigLanguage.html was recorded and the action could not be replayed with error 404 - Not found
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7212 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-30 03:16:17 +00:00
mikeworks
f468d377d7 Collage.html and Collage.java: Added <p> in body before images for XHTML 1.0 Transitional validationg and alt tag to images as well as closing tag <img (...) />
terminal_p.html: Set new link for starting a crawl to CrawlStartSite_p.html and replaced the old embed object of the Among.us Flash object by their new JS which takes care of adding the object correctly
de.lng: Moved the translations for the JS part from yacyinteractive.html to the yacyinteractive.js part
--> Terminal page is now valid XHTML 1.0 Transitional


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7211 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-29 23:11:29 +00:00
orbiter
3057a0b939 - intranet scanner now produces urls with host names, not ips if possible
- CrawStartIntranet servlet shows IPs and host names

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7210 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-29 22:44:49 +00:00
mikeworks
421aa6a8bb ConfigLiveSearch.html: Fixed some HTML problems to validate at least XHTML 1.0 Transitional - strict is not possible because iframes are used. Replacing iframes with embedded object tag does not work in IE
ConfigPortal.html: Fixed some HTML problems to validate at least XHTML 1.0 Transitional - for strict the target attribute of the a link has to be removed
yacyinteractive.html: Moved all JS code to an external yacyinteractive.js file in JS folder
yacysearch.html: Removed embedded scripts from in between the body tags - now everything is loaded in the header
de.lng: Just in case JS files will be parsed at some point added translation for yacyinteractive.html result counter

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7208 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-29 20:06:39 +00:00
mikeworks
b7bb0cabaf Blacklist_p.html: Minor HTML and Javascript changes to get XHTML 1.0 Strict validation, lowercae onchange, id tags instead of name tags
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7205 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-29 05:41:32 +00:00
mikeworks
cd505d7e30 de.lng: German translation of the new Intranet Servlet introduced in SVN 7203 in CrawlStartIntranet_p.html
CrawlStartIntranet_p.html: New Intranet Crawl Start Servlet - minor HTML changes to get XHTML 1.0 Strict validation, remove (double) name tags, remove single ending </dt>

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7204 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-28 22:05:12 +00:00
orbiter
e63896f2a8 added an intranet scanner and a servlet which shows all intranet addresses and an option to start a site-crawl for all these addresses at once.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7203 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-28 12:18:54 +00:00
suessthomas
44874f2cb9 Added "encoding =" UTF-8 "in the RSS files
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7200 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-27 20:36:49 +00:00
orbiter
d2fd93135c - moved yacybot user agent string definition to MultiProtocolURI since there are basic access mechanisms where the bot string is needed
- migrated the 'yacy' user agent to 'yacybot' in many client methods since the 'yacy' user agent is only used for the proxy

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7199 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-27 14:54:32 +00:00
mikeworks
ad7efe6016 rssTerminal.html: Fixing the 'null' is null or not an object in rss2.js when viewing the YaCy default Status page http://localhost:8080/Status.html with Internet Explorer
feed.xml: copy of feed.rss that helps Internet Explorer also read the Feed - workaround for the fix above
Problem is described in the forums and should be fixed better ;-(http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2766&p=20702)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7196 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-26 22:55:52 +00:00
mikeworks
190de644dd de.lng: Added German translations for some missing table content on Network view
WatchWebStructure_p.html: Added JS verification of RGB color codes (currently only RGB value is checked but this could be enhanced to also check for websafe colors)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7195 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-26 22:28:32 +00:00
orbiter
d5dc88a351 shop cleanup button only if servlet was called without post/put arguments.
This should avoid confusion after a search for a word where it is possible to delete the word. If a delete button is shown to delete the word, then there should not be a button available to delete the whole index to avoide a wrong usage when a user searches only for a word to delete it.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7194 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-26 21:11:08 +00:00
orbiter
a83186ac7d fix for bug in cytrails
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7192 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-26 10:32:40 +00:00
mikeworks
b019426811 de.lng: Added German translations for new Index Creation pages RSS Feeds and adapted text in Tables_p.html and CrawlStartExpert_p.html to match some typos, also changed one name tag to id to conform with XHTML 1.0 Strict
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7191 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-26 01:39:51 +00:00
orbiter
48c0d508ac fixes for crawling of smb links (file length not always available)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7190 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-25 22:32:26 +00:00
orbiter
daeea96aea renamed servlet CrawlStart_p.html to CrawlStartSite_p.html to circumvent problem with translation which still showed old expert crawl start page
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7183 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-22 21:46:31 +00:00
orbiter
10a9cb1971 simplified snippet computation process and separated the algorithm into two classes
also enhances selection criteria for best snippet line computation

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7182 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-22 20:50:02 +00:00
orbiter
84a023cbc8 fixed several search bugs
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7180 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-21 21:48:42 +00:00
lotus
937dd956d3 save default number of search items via web interface
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7179 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-21 19:45:49 +00:00
orbiter
4e8cf0c72c added a search box and navigation to api steering servlet
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7178 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-21 13:10:18 +00:00
orbiter
97ee278931 enhanced search speed:
- better control of number of running search threads
- no time-out waiting time when no ranking feeding takes place
- local search queries by a remote peer may be faster up to 300 milliseconds
- a local search may even be faster

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7176 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-20 13:17:25 +00:00
orbiter
377f001e0d sorting of crawl profile names in crawl profile editor, see
http://forum.yacy-websuche.de/viewtopic.php?p=20851#p20851

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7172 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-20 09:09:38 +00:00
orbiter
a2f9974745 some redesign in the access tracker to realize sixcoolers question about "smartes way for deleting the first Object":
- not so much abstraction for a collection, makes use of remove() (no operands) possible
- different way to delete elements in track (destructive, not constructive (less copies of elements in new queue))
- more abstraction for class api since no static class must be used any more

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7169 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-19 23:00:24 +00:00
low012
f32bb5e51f *) Changed image in Steering.html from linked image to embedded image because shutdown is so fast now, browsers can't load image before Yacy instance is gone already. Had to make image smaller since IE does not accept large Base64 encoded images.
*) Decreases wait time in Steering.html before first check since 
*) HTML fixes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7165 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-18 00:18:52 +00:00
orbiter
37baa8bae3 - fixes for concurrency exceptions and failed database integrity verification
- added link to yacystats peer when peer is more than one day old

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7164 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-17 10:20:04 +00:00
orbiter
29fe401f93 - some layout and text enhancement for site crawl start
- Quix0rs patch from http://forum.yacy-websuche.de/viewtopic.php?p=20839#p20839 (parts)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7163 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-16 23:00:07 +00:00
orbiter
8c1da27347 - added more comments for user in site crawl servlet
- added a disable/enable function in case that 'sitemap' is selected for functions that do (not) apply
- better naming of menu items
- limit default crawl depth

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7162 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-16 22:04:14 +00:00
orbiter
58b7417a59 - added a new 'easy' crawl start menu which can be used for the special case of loading a complete domain
- the previous crawl start servet was renamed to CrawlStartExpert_p
- easy crawl start is now default

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7160 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-16 12:02:43 +00:00
orbiter
461a2a6ec7 enhanced remote crawling:
- 300 ppm is default now (but this is switched off by default; if you switch it on you may want more traffic?)
- better timing for busy queue
- better amount of remote url retrieval
- better time-out values
- better tracking of availability of remote crawl urls
- more logging for result of receipt sending

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7159 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-16 09:34:17 +00:00
orbiter
670ba4d52b - removed the remote crawl option from the network configuration submenu and
- added a remote crawl menu item to the index create menu. This menu also shows a list of peers that provide remote crawl urls
- set remote crawl option by default to off. This option may be important but it also confuses first-time users


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7158 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-16 00:39:05 +00:00
orbiter
ac1c08924e more performance hacks
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7149 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-14 15:27:27 +00:00
orbiter
39f409a7bb performance hacks
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7147 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-14 14:32:24 +00:00
orbiter
2e75879504 fix for latest commit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7145 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-14 13:01:18 +00:00
orbiter
6e4653cf50 remove DoS protection in remote search for intranet hosts
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7144 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-14 12:38:05 +00:00
orbiter
3c0e07ba72 removed all delays in shutdown process
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7143 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-14 09:13:28 +00:00
orbiter
906c572621 - enhanced index create menu structure
- clear search log caches each time a search is done

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7142 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-14 09:06:27 +00:00
orbiter
fc924f024e import of oai sources from a list using a command line interface:
if you have a list of oai servers you can import them all using the linux command:
bin/importOAIList.sh <name-of-oai-list-file>


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7141 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-13 10:13:34 +00:00
orbiter
64860dc1bb enhanced search event logging (to be used for further improvements)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7140 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-13 09:33:04 +00:00
sixcooler
17eebd4ef8 counting crawler traffic again:
fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2808

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7138 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-11 15:58:15 +00:00
lotus
547d5226ae fix banner reload parameters (were no html errors)
adapted default colours

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7137 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-11 11:05:07 +00:00
orbiter
34a25856a5 - added navigation to next/prev search page using arrow keys (left/right)
- better information text for YaCy GUI application

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7134 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-10 10:42:01 +00:00
lotus
5ce679a053 focus search field on load, no click necessary anymore
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7132 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-10 08:45:26 +00:00
orbiter
013926f01c added 'francais' as language option for default configuration
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7131 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-10 08:35:47 +00:00
orbiter
4c21d8dc9d - changed default values for online caution (the pausing may not be necessary any more)
- fixed bug in WeakPriorityBlockingQueue
- show favicon faster using pre-loading (same technique as used for fast image search)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7130 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-09 23:25:19 +00:00
orbiter
348dece62f redesign of the SortStack and SortStore classes:
created a WeakPriorityBlockingQueue as special implementation
of a PriorityBlockingQueue with a weak object binding.
- better abstraction of ordering technique
- fixed some bugs according to result numbering (distinguish different counters in Queue)
- fixed a ordering bug in post-ranking (ordering was decreased instead of increased)
- reversed ordering numbering using a reversed ordering. The higher the ranking number the better (now).

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7128 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-09 15:30:25 +00:00
orbiter
114bdd8ba7 fixed old sitemap importer which was not able to parse urls containing post elements
- removed old parser
- removed old importer framework (was only used by removed old parser)
- added a new sitemap parser in parser framework
- linked new parser with parser access in old sitemap processing routines

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7126 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-08 14:13:15 +00:00
lotus
b73ea6581d fix json in case of query includes "
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7125 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-08 11:54:25 +00:00
orbiter
5fe828fa06 - replaced pdfbox and fontbox version 1.1.0 with 1.2.1
- added some clear statements that shall clear static cache size within the pdfbox library
- the pdfbox library contains a memory leak; it is unsafe to run a peer with pdf parser permanently on.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7120 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-07 17:13:47 +00:00
lotus
5dff8f62c4 fix tray information display for non-windows
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7117 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-06 13:30:40 +00:00
orbiter
fb828f3767 - performance enhancements in search response time using faster query ID computation and an ID cache
- code cleanup

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7113 6c8d7289-2bf4-0310-a012-ef5d649a1542
2010-09-06 10:00:07 +00:00