Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
5,799 Commits 1 Branch 20 Tags 836 MiB
99fa265e1d
Commit Graph

5799 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
orbiter
95e8cbd1c3 new fully redesigned balancer and bugfixes regarding lost profile handles and killed crawls 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6025 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-06 01:56:31 +00:00
orbiter
c062385552 fix for http://forum.yacy-websuche.de/viewtopic.php?p=15555#p15555 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6024 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-05 18:18:16 +00:00
orbiter
42ae40b9f6 some bugfixes to database close() methods 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6023 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-04 22:43:46 +00:00
orbiter
a0c53abbe1 - wait until local results are computed during search, see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2167&hilit=&p=15521#p15521 
- show only x+1 pages in page navigator

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6022 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-04 20:58:47 +00:00
orbiter
94f3d90af2 added a hint about regular expressions in crawl start 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6021 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-04 20:03:26 +00:00
orbiter
9bfd22f65d fix for http://forum.yacy-websuche.de/viewtopic.php?p=15523#p15523 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6020 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-04 19:57:25 +00:00
orbiter
1c77db670f re-designed response format for navigation: 
- changed json and rss response templates


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6019 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-04 10:54:49 +00:00
orbiter
15fad767c0 some refactoring of topic generation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6018 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-03 23:49:06 +00:00
orbiter
f28f62fb21 added servlet for easy wiki content and search window integration 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6017 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-03 22:22:20 +00:00
lotus
efe97f446a better proxy configuration in case of remote proxy 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6016 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-03 19:03:03 +00:00
orbiter
cc49aedf12 - fixed problem with remote search NPE 
- more abstraction for search requests

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6015 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-03 08:49:54 +00:00
f1ori
9e18abc2ac * fix charset detection, see http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2137 
* why has this been uncommented???


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6014 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-02 20:54:13 +00:00
orbiter
c38c852090 modified access method to get index entries out of a array of BLOBs: 
iterate them, then merge; not collect them and merge then.
This should use less memory and may behave better in an environment with many queries.
To ensure that too many queries will not cause total blocking,
a time-out of one second was also added. After the time-out
the index data that was collected so far is returned.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6013 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-02 16:53:45 +00:00
apfelmaennchen
55ff919b5d - yacysearchtrailer.html ... just an idea for a timeline 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6012 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-02 16:47:39 +00:00
orbiter
ab06a6edd2 renamed topwords to topics and enhanced computation methods of topics 
topics will now only be computed using the document title, not the document url,
because the host navigator is now responsible for statistical effects of urls.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6011 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-02 15:20:10 +00:00
lotus
61d9e131b4 better/new proxy auto config 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6010 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-02 12:18:29 +00:00
apfelmaennchen
0d44a6d503 - yacy portalsearch experiments with navigation in sidebar (topwords & domains) 
- not yet functional ...

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6009 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-02 11:02:36 +00:00
apfelmaennchen
9f9a1b4ad8 - yacysearchtrailer.html small temporary work around for jquery-css display bug 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6008 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-02 09:08:34 +00:00
apfelmaennchen
b0e2d854e0 - fixed sidebar for yacysearch.html & yacysearchtrailer.html 
- @orbiter: please do not use <h2> or <h1> tags in the context of the sidebar!!!

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6007 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-02 07:32:45 +00:00
orbiter
a5d481eab1 enhanced navigation 
- fixed too early computation of navigation
- moved navigation rendering to yacysearchtrailer
- added more asserts

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6006 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-01 22:45:28 +00:00
orbiter
3ca1f109c4 added more jquery themes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6005 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-01 21:49:18 +00:00
orbiter
3ea399ec91 fix for absolute paths for repository path 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6004 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-01 10:54:41 +00:00
lotus
6b92155eb6 corrected spelling 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6003 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-01 09:48:08 +00:00
orbiter
5eac607166 fixed configuration of repository path 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6002 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-06-01 00:13:23 +00:00
apfelmaennchen
daee735ad7 - fix for yacysearch.html 
- navigation/sidebar JavaScript is now also triggerd by #(navigation)# instead of display=3

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6001 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-31 07:29:24 +00:00
apfelmaennchen
8fe69da2bb - some fixes for prev. post 
- better resizing and dragging

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6000 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-30 17:06:47 +00:00
apfelmaennchen
0eb3bffe97 - added 'drawer' (sidebar) for future navigational items to yacyui-portalsearch.js 
- http://forum.yacy-websuche.de/viewtopic.php?f=9&t=1904#p15311

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5999 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-30 14:32:29 +00:00
orbiter
7639ec2f38 - fixed letter case bug for dc record creation 
- dc parser is now lazy against letter cases

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5998 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-29 15:09:37 +00:00
apfelmaennchen
34af8b4877 - yacysearch.html compromise for positioning the sidebar 
- position is now fixed on the right top side
- should scale down to window width 800px, smaller windows will cause distortions
- see http://forum.yacy-websuche.de/viewtopic.php?f=9&t=1904#p15293

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5997 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-29 14:57:09 +00:00
orbiter
4522c13ee7 added option for a table prefix when importing phpbb3 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5996 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-29 14:29:02 +00:00
orbiter
1c69d9b8b6 more refactoring of the index classes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5995 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-29 14:16:41 +00:00
orbiter
3d5f2ff544 - added new servlets to support search portal administrators for the integration of yacy search fields in their web pages 
- moved some servlets from here to there..
- changed menu structure
- removed yacyui-portaltest.html which contained an example for the live search which is now integrated on all pages in yacy. The code snippet example from that page is integrated into the ConfigLiveSearch.html servlet


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5994 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-29 14:16:03 +00:00
orbiter
4d4315687f fix for problem with concurrency in host navigator, bug reported by wsb 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5993 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-29 10:52:50 +00:00
orbiter
88426912ad more refactoring to make the segment object easier to use and to be prepared to integrate author navigation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5992 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-29 10:03:35 +00:00
lotus
d813fd26ed reset sent/received counters on index delete 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5991 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-28 15:49:42 +00:00
orbiter
99bf0b8e41 refactoring of plasmaWordIndex: 
divided that class into three parts:
- the peers object is now hosted by the plasmaSwitchboard
- the crawler elements are now in a new class, crawler.CrawlerSwitchboard
- the index elements are core of the new segment data structure, which is a bundle of different indexes for the full text and (in the future) navigation indexes and the metadata store. The new class is now in kelondro.text.Segment

The refactoring is inspired by the roadmap to create index segments, the option to host different indexes on one peer.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5990 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-28 14:26:05 +00:00
orbiter
876746602d catch problems of file hash computation, see also: 
http://forum.yacy-websuche.de/viewtopic.php?p=15245#p15245

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5989 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-28 10:08:36 +00:00
orbiter
fec6f9054f some refactoring of search methods 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5988 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-27 23:51:34 +00:00
orbiter
d793bb0d76 the mysql lib was not in releases included; moved library from libx to lib 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5987 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-27 20:23:28 +00:00
orbiter
3d4b826ca5 migration of all databases that use the deprecated BLOBTree format into the BLOBHeap format. Old databases are migrated automatically. 
This removes the last very IO-intensive data structures which were still used for Wiki, Blog and Bookmarks. Old database files will still remain in the DATA subdirectory but can be deleted manually if no major bugs appear during migration. There is no need for any user action, all migration is done automatically.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5986 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-27 15:04:04 +00:00
orbiter
4b4bddca00 added new submenu to crawler menu: import of phpbb3 forum postings from mysql 
- yacy can import phpbb3 posts without crawling
- all data is written as surrogate
- indexed surrogate files can be re-used

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5985 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-27 14:53:23 +00:00
apfelmaennchen
f1a9253baa see: http://forum.yacy-websuche.de/viewtopic.php?f=9&t=1904#p15198 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5984 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-27 08:56:55 +00:00
apfelmaennchen
369bb78a7a - added "visibility: hidden;" for yacysearch.html sidebar 
- this only works with javascript....sorry

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5983 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-27 07:30:59 +00:00
apfelmaennchen
16a226b016 removed debug-alert() from last commit 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5982 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-27 07:13:45 +00:00
apfelmaennchen
01334686bd small fixes for yacysearch.html sidebar (display=3) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5981 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-27 07:06:13 +00:00
orbiter
d8284046b0 enhanced speed of site navigation computation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5980 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-26 22:30:20 +00:00
orbiter
c72a5cf326 added stub for PHPBB3 extraction code using direct access to mySQL 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5979 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-26 15:58:49 +00:00
orbiter
e735d3a69f fix for http://forum.yacy-websuche.de/viewtopic.php?p=15175#p15175 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5978 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-26 15:03:50 +00:00
orbiter
63a0255166 - refactoring: added new content package, which will contain connector classes for different types of data sources to import texts into the YaCy index 
- refactoring: migrated data objects for the new connector classes
- added a DAO interface class to specify an abstract interface for database retrieval connector methods

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5977 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-26 07:44:22 +00:00
orbiter
f246928c20 first attempt to add 'real' Navigation to yacy search results: host navigation 
- after a search is started, it is analysed how many hits are in each site
- this can be done really efficient, because the navigation information is hidden in the url hash and can be computed very fast
- the search result shows a column on the right with the hosts and the hits per host
- after a click on a host the search is modified using the efficient site: - operator

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5976 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-05-25 22:27:34 +00:00