Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
6,563 Commits 1 Branch 20 Tags 836 MiB
bfdb9f4e06
Commit Graph

2943 Commits

Author SHA1 Message Date
orbiter
25aef069a6 continuing String-hash - to - byte[]-hash redesign that was started in SVN 6775 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6790 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-08 00:11:32 +00:00
mikeworks
aad0a3045a Fixed CSS file yacyui-portalsearch.cs 
border-width: small; --> border-width: thin;
margin-botton --> margin-bottom
Forum Thread: http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2767

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6789 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-06 00:04:01 +00:00
mikeworks
6ef4900ec5 Changed function name status() -> statuscheck() in yacyui-portalsearch.js fixing the "status is not a function" bug visible in Firefox error console: 
http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2625

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6788 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-04-05 12:44:35 +00:00
mikeworks
7a3c19846f Updated German translation de.lng: added new Table_RobotsTxt_p.html and some other changes 
Changed 'Sprache' -> 'Language' in yacydoc.html and added translation in de.lng

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6783 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-29 00:56:51 +00:00
low012
000dad034b *) better usage of JS timers (intervals) 
*) added messages when status is checked

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6781 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-28 22:17:36 +00:00
orbiter
1e8e79b9ef redesign of reference hash (URL-hash) parameter hand-over: 
pass value as byte[], not as String. This should cause that less
byte[] <-> String conversions are made during time-critical tasks.
This redesign is not yet complete, more to come ..

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6775 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-26 18:33:20 +00:00
orbiter
9ddb8e4a43 set an option for the java-internal image parser that prevents that the image is cached using the file-system in a temporary file. This should speed up image parsing during image indexing dramatically and should also cause better performance when showing the yacy banner and OSM tiles. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6772 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-26 10:43:31 +00:00
orbiter
0018163c07 moved table row/column matching method from front-end to back-end 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6770 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-26 10:01:27 +00:00
orbiter
21fcbcc35f added sorting function in network table, reverting SVN 6736 (not removing new sorttable) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6768 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-25 07:47:41 +00:00
orbiter
fad3abb524 Tables_p.html servlet can now show tables with selected rows using a search field 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6765 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-24 10:52:37 +00:00
low012
4c6dc396d8 *) more beautyful (IMO) code, no functional changes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6764 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-23 21:05:26 +00:00
orbiter
6c093d6aed - enhanced domain navigator computation 
- fixed domain navigator content in case that a mustmatch constraint was given

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6763 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-23 13:41:41 +00:00
orbiter
bb63c5d075 using a Pattern object with precompiled regular expressions to apply must-match constraints to search results: should speed up pre-sorting of search results and should cause richer search result sets 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6762 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-23 10:17:28 +00:00
suessthomas
5233177a7f A small typo fixed 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6761 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-23 08:44:34 +00:00
lotus
ea69300857 fix bad floating navigators on little results 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6758 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-20 22:09:55 +00:00
orbiter
bfb518cd47 some refactoring to get the LoaderDispatcher a little bit more independent from the switchboard 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6755 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-20 10:28:03 +00:00
orbiter
0465f28f7f applied 'null in rss2.js' fix from Quix0r, see 
http://forum.yacy-websuche.de/viewtopic.php?p=19612#p19612

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6752 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-20 09:58:05 +00:00
orbiter
748abfcffa added patches to prevent yacy-protocol DoS settings 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6751 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-19 15:31:15 +00:00
orbiter
e820ed061a avoiding excessive DNS lookups to determine localhost 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6750 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-19 14:28:25 +00:00
orbiter
0f8004f9da enhanced html parser to recognize a href tags inside header tags 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6743 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-11 17:52:07 +00:00
orbiter
3300930fc5 - (almost) fixed FTP crawler 
- integrated/fixed SMB crawler

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6742 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-11 15:43:06 +00:00
orbiter
61493a9a9f added more information about metadata in ViewFile.html 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6740 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-11 00:11:14 +00:00
orbiter
1198b9989d bugfixes, more sorttable 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6739 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-10 15:39:36 +00:00
orbiter
27b2998eb4 added searchtable function to more tables in interface 
you can now sort by any column in most tables in YaCy just by clicking on the headline column of the table

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6738 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-10 10:05:41 +00:00
orbiter
9623d9e6d2 added a smb loader component for the YaCy crawler 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6737 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-10 08:55:29 +00:00
orbiter
c77fbd0390 added sorttable (http://www.kryogenix.org/code/browser/sorttable/) 
javascript library to make tables sortable

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6736 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-09 23:40:16 +00:00
orbiter
3014e5f6f9 - integrated live search in the IndexControlURLs input window for URLs: 
this searchs for occurrences of the given word in URLs and presents them
  in a pop-up list below the input line
- some bugfixes for the new robots table viewer

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6735 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-09 15:44:11 +00:00
orbiter
0769517129 added a robots.txt monitor in the crawler monitor submenu 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6733 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-09 11:31:15 +00:00
orbiter
72f00dee59 removed never-used server access account function 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6731 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-08 22:30:45 +00:00
orbiter
474bb4de82 ups 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6730 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-07 23:32:18 +00:00
orbiter
8c88abf685 added follow-me link for twitter in status hints 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6729 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-07 23:29:29 +00:00
orbiter
58d75a6bde allow more results for a single query at the same time if the client is not authorized. This is necessary for the search widget where the default number of results is now set to 20 instead of 10 to cause that a scroll bar is shown which is necessary to get a trigger for new searches for more results. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6728 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-07 22:49:20 +00:00
orbiter
4144927d94 show less errors 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6725 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-07 21:02:08 +00:00
orbiter
c4bdb1e7f2 added one more option in ViewFile to show an iframe like for the orginal web page content but using the cache than the direct link to the content in the web. Upgraded the very old and previously not any more used CacheResource_p servlet to a new and working version. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6719 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-06 23:41:51 +00:00
orbiter
884b262130 - added a new Wiki Namespace Navigator 
- some redesign of Navigator data structures

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6716 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-05 21:25:49 +00:00
orbiter
b0c6d0108b fix for select-all toggle in tables servlet 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6715 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-05 16:15:59 +00:00
orbiter
270fb38674 - fixed some bugs in Table viewer 
- added 'select all' feature in Tables_p
- enhanced ViewFile.html: has now an input field to load arbitrary resources from the web and analyze them (!!!)
- included the ViewFile servlet into the Index Administration menu
- show in ViewFile if ressource is in url-db and/or in Web cache
- bugfixes to BEncodedHeap and Tables management

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6713 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-05 15:41:15 +00:00
orbiter
38d7a28cd2 fix in viewfile needed when ViewFile is called only with 'url' parameter 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6712 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-05 12:24:15 +00:00
orbiter
599c3766c4 added authentication to automated API call 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6711 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-04 14:10:03 +00:00
orbiter
727dd9b193 - fixed a bug in robots.txt parser 
- moved storage of robots.txt entries to WorkTables, so it is now possible to browse the robots entries with the table browser

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6710 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-04 11:58:07 +00:00
lotus
f336ed568d fir for parameters on reload 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6708 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-03 21:14:12 +00:00
lotus
ddc21e7b73 adapted banner colors to current standard template 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6707 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-03 21:10:02 +00:00
mikeworks
e76f1e6cc0 Update property name bookmarks to display_bookmarks (changed in SVN 5554) to fix the broken Bookmark RSS feed 
- Fixes RSS part of http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1824

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6706 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-28 11:22:01 +00:00
lotus
8a19be24de tell non-windows users about the tray-icon 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6703 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-26 19:44:23 +00:00
orbiter
6b89f681c5 added bing, dbpedia and wolfram alpha to the compare-search options 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6701 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-26 10:34:36 +00:00
orbiter
4b6efe3b48 more ergonomic default values for crawl start 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6698 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-24 22:33:51 +00:00
orbiter
f175f9a2d3 changed way how number of search requests are counted: 
so far only search requests at the remote search interface had been counted.
This was done to protect the privacy of searchers, because counting was not done and published at the own search interface.
This caused that no search requests of robinson peers had been counted, becuase they cannot be counted at remote peer.
This change introduces a distinction of locally done search requests at the local search interface from search requests that are on the local interface but had been submitted from a remote IP without authentication.
Now 3 counters are maintained:
- partial count of remote searches
- total count of local searches on robinson peers from non-authenticated clients
- total count of local searches on robinson peers from localhost or authenticated clients
In the global statistic of search requests now the first two counters of the three cases are added
Because we habe a large number of robinson peers with a large number of remote non-authenticated requests the statistic should show at least three times of the number of search requests.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6696 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-24 13:53:55 +00:00
orbiter
23bf81d225 new buttons to select/deselect all entries 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6681 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-18 23:31:23 +00:00
orbiter
564927ce72 redesign of CrawlResult data structures because of OOM occurrences during URL deletion processes. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6675 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-16 23:06:04 +00:00
orbiter
d378ca4604 better handling of concurrency in seed 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6669 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-15 15:57:35 +00:00
orbiter
6538043d89 fix for http://forum.yacy-websuche.de/viewtopic.php?p=19189#p19189 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6668 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-15 15:45:31 +00:00
orbiter
047f8718a7 added kiosk-mode button on standard search page and interactive search page 
see also:
http://forum.yacy-websuche.de/viewtopic.php?p=19178#p19178

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6667 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-15 12:54:53 +00:00
orbiter
ac492fa2a5 added a close button image 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6666 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-15 10:40:33 +00:00
suessthomas
1a4cb51744 ups, unclosed HTML tag fixed.... 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6664 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-11 22:13:52 +00:00
suessthomas
9e14958115 minor corrections and bug fixes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6663 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-11 15:05:38 +00:00
lotus
945e0ba5a5 allow global search if res. observer disabled index transmission 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6658 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-09 17:14:16 +00:00
suessthomas
063b29060c Minor Changes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6656 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-08 10:13:14 +00:00
lotus
11188cd7eb resource observer now uses the Java 6 method to check for free space. thus, disk observing now needs Java 6 installed. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6652 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-06 18:48:06 +00:00
orbiter
be18b5d8cd fix for 'cannot switch back to default language'-bug 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6649 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-04 23:53:02 +00:00
orbiter
fe78edac32 - view API calls in correct date-order 
- execute recorded API calls in date-order

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6646 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-04 15:51:54 +00:00
orbiter
308a973503 refactoring of tables data organisation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6644 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-04 11:26:23 +00:00
orbiter
8a76f38d26 Added a new steering servlet that can be used to repeat actions that had been made on the yacy interface. This can be used to: 
- start again a previously started crawl
- submit settings (again). This option will be used to transmit
  all settings of one peer to another peer if the remote-peer
  steering function is ready
This steering framework will also be used for a 'schedule-everything'
which will also include a new scheduler for crawling.
  

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6642 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-03 09:31:12 +00:00
lotus
530b77d05a adapted loglevel for accepted urls 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6641 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-02 09:47:43 +00:00
lotus
5f661d9f5d fix receive of crawl receipt in private peer mode 
http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2655

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6640 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-02 09:42:14 +00:00
orbiter
840527689b more simplification of bookmark class 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6639 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-01 23:04:52 +00:00
orbiter
ada0ce9de3 refactoring of bookmarks: there is a big performance problem in the bookmarks code and furthermore the bookmarks 
will loose its leading role for the re-crawl funtion when the new api tables will work. To be prepared for a replacement
of such functions the bookmark class is re-organised.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6637 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-01 22:18:56 +00:00
orbiter
24060885b6 - added Tables abstraction in data.Tables.java 
fix for
http://forum.yacy-websuche.de/viewtopic.php?p=18910#p18910
http://forum.yacy-websuche.de/viewtopic.php?p=18894#p18894
http://forum.yacy-websuche.de/viewtopic.php?p=18814#p18814


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6631 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-29 18:02:09 +00:00
orbiter
7fdf59a77f misc NPE check 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6630 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-29 15:59:24 +00:00
suessthomas
a29b17a2fd Minor HTML Changes, Images recompress. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6627 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-27 21:38:54 +00:00
suessthomas
67b8d5149e Small Changes on Network.html - Specifying image dimension 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6625 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-26 14:56:27 +00:00
mikeworks
e87f0e83a2 Updated German translation: 
- Fixed a typo in ConfigPHPBB3Search.html
- Added section for ConfigPHPBB3Search.html and translation

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6620 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-24 13:12:10 +00:00
mikeworks
d915deaa2b Fixed type in CrawlStart_p.html 
Changed to German language file:
- Updated Crawl Start Page
- Added section for indexing MediaWikis
- Fixed some more start and end tags so that syntax highlighting works correctly

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6617 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-22 23:00:05 +00:00
orbiter
3889438db6 fix for bookmarks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6615 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-22 14:20:24 +00:00
orbiter
597a410f58 ups 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6609 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-21 22:16:28 +00:00
orbiter
8ce936bcdd added an api recording function: it shall be possible to record 
all operations on YaCy in a database that should make it possible
1) to re-create a setting on fresh peers
2) to transmit a setting from one peer to another
3) to re-create crawl starts after a complete deletion of the index
This functionality will also support
4) scheduled re-crawls (new implementation)
To implement this, a new database structure has been crated that stores maps into blob heaps. to encode maps the b-encoding technique was used (this is the same encoding that torrent files use)
- added a b-encoder
- enhanced the b-decoder
- added a b-encoded map heap data structure
- added a table organisation based on b-encoded heaps
- added a servlet to maintain such tables (see Tables_p.html)
- integrated the servlet into the Advanced Settings menu
- added an api recording based on the new tables

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6606 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-21 22:06:03 +00:00
orbiter
8df1694288 - added options to switch on/off search domains (text, image, audio, video, app) 
- more memory by default


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6605 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-21 22:03:02 +00:00
mikeworks
eac3b696ab Fixed typos in WikiHelp.html 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6599 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-20 07:34:43 +00:00
mikeworks
03fb4a1980 Fixed typos CrawlProfileEditor_p.html 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6598 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-20 07:33:10 +00:00
orbiter
78daea95c3 show more navigation pages if there are too less locally 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6595 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-19 00:21:55 +00:00
mikeworks
25c545c4b0 first commit fixing two typos 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6594 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-18 07:25:41 +00:00
orbiter
234f733a3d - relocation of seed db is better for network switch than re-initialization because of the embedding of the peers object in other objects 
- small refactoring of blacklist interface code to remove PMD warnings


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6593 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-18 00:07:20 +00:00
orbiter
763e0d447c removed unnecessary cast 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6586 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-14 14:23:26 +00:00
orbiter
34354cf9b2 added a servlet that has been removed in SVN 4881; this servlet is now splitted and will be used for a simple crawl start and a remote crawl monitor (not yet integrated into the interface) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6582 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-13 23:19:45 +00:00
orbiter
81035e7080 moved a sub-menu entry 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6578 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-13 16:18:23 +00:00
suessthomas
5f1565cc95 Fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2540&sid=507e4dd9e234af001ba60c0d8adf45be, and small Layout Changes. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6577 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-13 07:57:36 +00:00
orbiter
82f57f79e5 more PMD enhancements 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6576 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-13 00:23:07 +00:00
orbiter
5d930c96f0 more fixes to search result page navigation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6575 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-13 00:04:37 +00:00
orbiter
a06f7ddb33 more PMD recommendations 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6572 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-12 20:53:19 +00:00
orbiter
9bbd546e64 in live search, show at least 20 entries instead of only 10 
this is a work-around for the problem that the search widget
does not load a second page if the first page did not fill up
the window with enough lines such that a scrollbar ist visible.
Because the scrollbar triggers loading of following pages, this
must be enforced with the trick that more result lines must be
shown after the first search.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6570 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-12 15:23:01 +00:00
orbiter
18172451a0 better search computation: 
- increased sort limit, now 3000 entries, before: 1000
  this should cause that more results can be shown in case
  of strong limitating constraints, like domain navigation
- enhanced the sort process
- check against domain navigator bugs
- fix in sort stack
- showing now all naviagtion pages at first search (not only next page)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6569 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-12 15:01:44 +00:00
orbiter
d126d6c1b5 renamed the servlet WatchCrawler_p to Crawler_p 
this was done because that servlet may be used for wget/cronjob
triggered crawl starts and it appears to be confusing that the
name of the crawl start servlet looks like a pure monitoring tool.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6568 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-12 10:05:28 +00:00
orbiter
909a4f91c7 added a logging output for crawl starts that shows the URL that can be used to start the crawl again 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6566 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-11 18:10:39 +00:00
orbiter
2113fcd7e5 - fixed usage of isEmpty() which is not available in java 1.5 
- increased visibility of some methods

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6564 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-11 12:33:40 +00:00
orbiter
dd459281c8 applied code changes that are recommended by PMD 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6563 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-10 23:09:48 +00:00
lotus
eac2daf2e8 * reenable DHT if yet enough memory is available 
* reset treshold on reconfiguratoin
(thanks to sixcooler)

* display status message in web interface

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6562 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-10 19:04:43 +00:00
orbiter
d77a8f3b3e added some modifications recommended by PMD for better performance 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6560 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-10 01:40:26 +00:00
orbiter
a3b8b7b5c5 some redesign of the main menu structure: 
- moved all index generation servlets to it's own main menu item, including proxy indexing
- removed external index import because this operation is not recommended any more. Joining an index can simply be done by moving the index files from one peer to the other peer; they will be merged automatically
- fix to prevent endless loops when disconnecting http sessions
- fix to prevent application of bad blacklist entries that can cause a 'Dangling meta character' exception

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6558 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-10 00:10:43 +00:00
lotus
32972139af added nice configuration for the resource observer 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6554 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-07 17:19:50 +00:00
low012
04d91c67d6 *) added message if unrestricted access from localhost is possible 
*) added link to admin access config page

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6543 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-28 14:01:41 +00:00
suessthomas
d933763b12 Small Changes on the de. language File and small correction to the ConfigAppearance_p.html page. Please test and report bugs to me (th-suess@gmx.de). 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6534 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-12-26 21:18:20 +00:00