Commit Graph

100 Commits

Author SHA1 Message Date
lotus
fbca4f8354 more stability on watchcrawler
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5861 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-04-23 18:42:15 +00:00
orbiter
61f9dbf0cc - fixed a display problem in watch crawler
- another small enhancement in balancer

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5729 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-03-18 21:25:52 +00:00
orbiter
98f36a801a - small update to search result layout
- some more mime types

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5669 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-03-06 10:16:14 +00:00
orbiter
dedfc7df7f removed distinction between DHT-in and DHT-out. This is necessary to make room for the new cell data structure, which cannot use this this distinction in the first place, but will enable the same meaning with different mechanisms (segments, later)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5511 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-01-22 00:03:54 +00:00
orbiter
6663365720 adopted many calls to new api path
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5498 6c8d7289-2bf4-0310-a012-ef5d649a1542
2009-01-16 00:02:55 +00:00
lotus
b1e211b258 no error-alert: http://forum.yacy-websuche.de/viewtopic.php?t=1639
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5391 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-12-13 12:04:08 +00:00
lotus
1545e5440a * index deletion: checkbox-confirmation
* watch crawler: less load on exhausted peers; wait for data before reloading again

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5359 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-11-23 12:02:58 +00:00
apfelmaennchen
8d1bedfc3a - added bookmarkTitle to CrawlStart_p.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5068 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-21 21:07:21 +00:00
lotus
f9fb2d0fee hide memory tables by default
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5020 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-07-27 06:59:24 +00:00
orbiter
01b3e9431a - fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1140&p=7626#p7626
- less dots for ppm bar in watchcrawler (one dot for each 10 ppm)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4846 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-24 11:30:16 +00:00
orbiter
9935e83c86 added new news window into the status page. At this moment it is just a test.
The news inside the window are about peer arrivals and departures, remote search accesses and crawls

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4739 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-04-26 01:00:10 +00:00
orbiter
e90282da1c added experimental javascript terminal for rss feeds (not used yet anywhere yet, expect the worst)
.. possibly to be used as content for iframes within monitoring pages
not ready yet!

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4727 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-04-22 23:09:24 +00:00
f1ori
76eac114ed * define global javascript-variable with var to get rid of warnings
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4624 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-03-30 19:51:19 +00:00
orbiter
f4c73d8c68 - fixed highslide usage
- some enhancement to index management, better types

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4497 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-02-19 14:13:35 +00:00
orbiter
3441ec3928 - some small changes to highslide integration to get it working... (does not work yet)
- performance enhancement for url list parser

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4495 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-02-18 23:49:03 +00:00
orbiter
6c3cd2b4f2 - added new way to watch images from the image search:
they appear as separate, floating window above the search results,
  not in a new window
- added highslide javascript library for feature mentioned above
- removed dir servlet. This thing was not used as it was supposed to be (as an example applet)
  and was a major problem for intranet-indexing when files are hosted on the same peer.
- added yacy-httpd-internal directory listing. Because YaCy is a search engine,
  directory listings are similar to search result listings. Intranet indexing from the same peer
  will get nice index pages for document collections.
- removed unused test applet

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4494 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-02-18 16:38:06 +00:00
orbiter
a8a5df4a51 - more dublin core naming of page metadata
- better presentation of result counters in search results

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4420 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-01-30 21:58:30 +00:00
apfelmaennchen
e68b133b35 added JavaScript for folder tree view
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4355 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-01-22 19:05:36 +00:00
orbiter
89b9b2b02a redesigned remote crawl process:
- instead of pushing urls to other peers, the urls are actively pulled
  by the peer that wants to do a remote crawl
- the remote crawl push process had been removed
- a process that adds urls from remote peers had been added
- the server-side interface for providing 'limit'-urls exists since 0.55 and works with this version
- the list-interface had been removed
- servlets using the list-interface had been removed (this implementation did not properly manage double-check)
- changes in configuration file to support new pull-process
- fixed a bug in crawl balancer (status was not saved/closed properly)
- the yacy/urls-protocol was extended to support different networks/clusters
- many interface-adoptions to new stack counters

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4232 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-11-29 02:07:37 +00:00
fuchsi
0e1738899f * Complete number localization and provide a more reasonable interface to serverObjects:
- put(key, value) methods are now used if a value added to the map should be kept as it is. Numbers are transformed (but not formatted) to an equivalent String representation.
- putASIS(...) have been removed, now done with simple put(...) (see above).
- puNum(...) can be used for number values which should be stored in a formatted way, either depending on the current locale setting for yacy (default) or in a "none" locale (see javadocs and setLocalize()).
- putHTML(...) escapes special characters into corresponding HTML enities ('<' => '&lt;') which was done with put(...) before and so was called too often, becauses it is necessary only for very few cases. Additionally there is a "forXML" mode which only replaces < > & ".
In short: Use put(...) for almost everything, use putXY(...) if you need some special transformation of the value.
A few bugs have been fixed as well, and there should be a small performance improvement for complex pages with a lot of values.

* added additional Sum/Avg rows to access tracker pages, see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=456
* removed duplicate code (mostly related to the big changes above).

TODO:
- make sure, number formats work as expected _everywhere_, report overseen stuff http://forum.yacy-websuche.de/viewtopic.php?f=5&t=437
- probably a good idea to add special putDate() methods as they are used in many pages and create duplicated formatting code + maybe some centralized handling for memory value formatting.
- further improve the speed of page creation for the WatchCrawler.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4178 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-10-24 21:38:19 +00:00
fuchsi
9524b9c16a second try of rev 4100 :). Tested in Iceweasel/Firefox 2.0.6, Konqueror 3.5.7, Opera 9.23 (all linux) and IE6-SP1 (wine)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4102 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-09-17 19:39:15 +00:00
fuchsi
6b8faaadb6 undo last commit for further evaluation, a progressbar element is used on other pages as well...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4101 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-09-17 03:36:35 +00:00
fuchsi
1880bba420 A few changes to the progress bar and search result statistics layout influenced by the discussion in <http://forum.yacy-websuche.de/viewtopic.php?f=5&t=268> with the idea of saving vertical space. Please check in every available browser and comment wether it's better than before. ;)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4100 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-09-16 14:30:53 +00:00
fuchsi
e78098be9b According to HTML-Specs "name" and "id" attributes share the same namespace. So we can't have one element with name="offset" and another one with id="offset". Additionally IE6's getElementById() returns elements with matching names as well and Opera is mimicing this behaviour.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4094 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-09-11 16:21:14 +00:00
orbiter
6c3bcadc1c - re-implemented image search
- generalized search result status bar, is now also visible during text search


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4077 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-09-06 13:26:38 +00:00
orbiter
4779f314fe first version of next-generation search interface:
- snippets are not fetched by browser using ajax, they are now fetched internally
- YaCy-internat threads control existence of snippets and sort out bad results
- search results are prepared using SSI includes
- the search result page is visible right after the search request, the results drop in when they are detected
- no more time-out strategy during search processes, results are shifted within queues when they arrive from remote peers
- added result page switching! after the first 10 results, the next page can be retrieved
- number of remote results is updated online on the result page as they drop in
- removed old snippet servelet (which had been also a security leak btw)
- media search is broken now, will be redesigned and fixed in another step


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4071 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-09-03 23:43:55 +00:00
orbiter
71e5d24f4a fix for watch crawler, see http://forum.yacy-websuche.de/viewtopic.php?p=1771#p1771
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4064 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-08-28 12:20:19 +00:00
orbiter
e332b844b2 - enhanced remote search: during waiting time for remote crawls
some urls are fetched so the url cache can be filled with these urls
- the url-prefetch is used to sort out some unresolved urls
- the snippet-fetcher is triggered with the search event id. This is used
  to remove missing snippets from the search cache so they will not be displayed again


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4060 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-08-26 18:18:35 +00:00
michitux
5cf634a4a4 New media-search ui:
- uses the progressbar introduced in the image-search
- results are displayed using the same layout as the text-search
- results are displayed in the order they arrive


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4041 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-08-11 22:20:01 +00:00
orbiter
62347b50f4 added security layer for ViewImage:
- images may be requested by localhost and authorized users only, if the request is done using a clear-text URL
- the image may be requested also using a code that can be a license to retrieve a URL for everyone
- some servelets produce URL licenses for ViewImage, like image search results


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4027 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-08-03 23:06:53 +00:00
michitux
8ebfd732ce - Fix for the redisplay of hidden results in Opera, see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=140 for details.
- Now the message that there are hidden results is hidden when all results are displayed again.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3994 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-07-19 19:49:48 +00:00
orbiter
89e1848db6 fixed problem with favicons:
target servers had been able to see search words from the referrer of the favicon fetch.
This has been removed by using the getImage - servlet for favicon fetch.
Since java does not support loading of bmp and ico-Images, such parsers had been added.
The image parser had been coded from their original microsoft documentation.
This influences also the image-search functionality: there can now be a preview
of found bmp-images. Another benefit: favicons for search results are now cached with the HTCACHE.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3965 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-07-15 01:34:01 +00:00
michitux
110a1a2b16 - fixed the handover of the searchterm and -type on index.html when the user clicks on "more options..."
- some small changes to make index.html and the menu valid XHTML 1.0 strict
- changed the inconsistent eol - characters in index.html to unix-ones


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3940 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-29 19:23:42 +00:00
orbiter
1d0cce8f3a documentation update
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3911 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-17 22:35:16 +00:00
michitux
25529290ca - 2 small changes in documentation
- hopefully fixed logging of GCs (in order to avoid things like "performed necessary GC, freed 18014398509481565 KB (requested/available/average: 4096 / 1631 / 2957 KB)") with the help of KoH


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3909 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-17 19:32:38 +00:00
michitux
184ba22ce9 New image - search HTML/JavaScript - frontend:
- <noscript> - area for non-JS - Browsers
- progressbar for the loading - process (may be used in other searches too)
- the image that is available first ist displayed first, so the images aren't moved around when new results arrive
- the correct number of results is displayed
- successfully tested in IE 5.5 and 6, Opera, Firefox and Konqueror (recent versions)



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3904 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-16 21:55:15 +00:00
theli
339153d40e *) favicons that are specified in the document content via html link-tags
are now detected and displayed on the search page (requested by allo).

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3845 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-09 15:22:37 +00:00
allo
38c180b28b hide results with wrong("red") snippet.
(maybe not as default? But it works pretty good for me)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3842 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-08 23:59:18 +00:00
allo
559d1c447f Bookmarks tag suggestion
AJAX fix for configadvanced
empty bugs are not a interface bug, but a scraper bug.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3821 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-07 22:49:31 +00:00
theli
e75ca857c3 *) Bugfix for problem with ajax graphic
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3815 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-07 07:40:32 +00:00
allo
54ddb3262c enter on webstrucutre
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3783 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-02 14:50:36 +00:00
allo
d0f8254f95 better refresh ui
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3779 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-06-01 13:58:48 +00:00
(no author)
5cc8bb075b Syntaxfehler beseitigt
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3764 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-05-26 19:51:05 +00:00
(no author)
ef24bed406 Sorry...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3760 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-05-24 16:25:07 +00:00
(no author)
a29cb2e1af blupp
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3759 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-05-24 16:14:46 +00:00
orbiter
a3ecfe0a45 replaced failed-icon by new 'bad'-icon
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3680 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-05-07 14:05:49 +00:00
theli
6f46245a51 *) Bookmarks: Ajax icon is displayed while loading title
*) First version of a sitemap parser added
   - currently only autodetection of sitemap files is supported
*) DB-Import restructured
   - pause/resume should work again now


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3666 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-05-06 09:52:04 +00:00
michitux
56b30d6593 * fixed a bug in ie: class-names for image-snippets were set, but had no influence on ie, they have to be set with className =
* fixed a bug in safari (hopefully, sorry that I removed the old fix, the divs): yacy-logo is now above the fieldset, the fieldset clears and has a margin-left set
 * fixed a bug with the dls: for example in ViewProfile.html the dt's (the terms/keys) had not the same height as the dd's, so the dt's were not in the same row with the coresponding dd's towards the bottom
 * moved my new css-classes to the right place in base.css

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3572 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-04-13 23:14:12 +00:00
michitux
e917bfcae3 * Bugfix: changed handling of the query-string to be independent from input-elements
* removed unnecessary divs

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3571 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-04-13 18:01:02 +00:00
michitux
4990909178 Some bugfixes, new layout/style for image search results:
* removed divide by zero bug when 20_dhtdistribution_busysleep is 0
 * replaced German comment with wrong charset in source/de/anomic/plasma/plasmaCrawlBalancer.java by an English one
 * replaced the table-fix for floating behind snipped images by a br with clear
 * removed unnecessary old xhtml-files (were not in use, they were created when we weren't having xhtml for testing)
 * new layout for image-search results: replaced the old one with spans and tables inside (not valid) with new divs, now each image snippet container has the same size
TODO:
 * the ids of the snippetLoading-divs aren't valid because ids must start with an alphabetic letter or an underscore, they have to be prefixed
 * in the returned snippet-xml is an unresolved pattern for status (the status is only set for text snippets)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@3566 6c8d7289-2bf4-0310-a012-ef5d649a1542
2007-04-12 18:21:17 +00:00