Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
5,200 Commits 1 Branch 20 Tags 836 MiB
92d77c3bef
Commit Graph

442 Commits

Author SHA1 Message Date
apfelmaennchen
92d77c3bef Major update to YaCy-UI...still not perfect...but I thought I share my progress :-) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5519 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-01-26 18:38:58 +00:00
orbiter
dedfc7df7f removed distinction between DHT-in and DHT-out. This is necessary to make room for the new cell data structure, which cannot use this this distinction in the first place, but will enable the same meaning with different mechanisms (segments, later) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5511 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-01-22 00:03:54 +00:00
orbiter
6663365720 adopted many calls to new api path 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5498 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-01-16 00:02:55 +00:00
apfelmaennchen
613c49bc38 YaCy-UI: update to welcome text (change-log and bug tracker) for stable release 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5485 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-01-13 05:59:39 +00:00
orbiter
ac89e8e84d removed unused search interface 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5474 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-01-12 08:45:12 +00:00
orbiter
efe801173c better dht-in cache flush. see also: 
http://forum.yacy-websuche.de/viewtopic.php?p=11936#p11936

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5472 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-01-11 22:39:49 +00:00
apfelmaennchen
cc207a979e - added new unified bookmark api to /xml/bookmarks/ 
the get_bookmarks api currently supports:
  .xml: posts, xbel, rss, flexigrid
  .json: posts, flexigrid
  .html: work in progress
- YaCy-UI: support for new bookmark api 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5467 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-01-11 12:29:19 +00:00
orbiter
6c7e83909b - refactoring of data access methods to be prepared for new cell data structure 
- removed a memory overhead in collections which prevent OOM Exception in low memory configurations

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5443 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2009-01-06 09:38:08 +00:00
orbiter
e004da48d3 - added fast fingerprint computation for files (any). Will be used in new index dump method 
- refactoring

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5415 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-12-29 12:22:13 +00:00
low012
5a89266598 *) new parameters for future use (better blacklist handling for im- and export) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5403 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-12-19 19:33:08 +00:00
orbiter
2802138787 - refactoring of CrawlStacker (to prepare it for new multi-Threading to remove DNS lookup bottleneck) 
- fix of shallBeOwnWord target computation heuristic


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5392 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-12-15 00:02:58 +00:00
orbiter
47292e696a more performance hacks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5379 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-12-04 12:54:16 +00:00
lotus
1951d30a62 addendum to last commit 
handle words with length < 3 correctly

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5369 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-11-26 19:43:40 +00:00
orbiter
3f746be5d4 - consolidation and refactoring of many DHT target - computing methods 
- implemented vertical DHT acceptance ("my own DHT") to accept new targets
- added new target computation for global search: addresses vertical targets also
- enhanced remote crawling: collection of remote crawl urls if queue has less than 100 entries (was: 0 entries)
- better performance value computations for PPM selection in network configuration

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5319 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-11-06 10:07:53 +00:00
orbiter
d014b2728a Design-check, Extension and Refactoring of DHT target position computation: 
- two different computations (but mathematical equivalent) of the DHT distance had been consolidated
- moved from 0.0 .. 1.0 double-range position computation to 0 .. Long.Max range for DHT targets
- added fast Long - to - hash computation
- high-precision target computation of gaps for new peers
- added new target computation for horizontal and vertical DHT targets (not yet in use)
- old horizontal-only DHT targets will be upwards compatible to new horizontal and vertical DHT positions

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5318 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-11-03 00:27:23 +00:00
orbiter
0edec2b760 FULL redesign of algorithms in htmlTools to encode/decode strings from/to unicode and html. 
The old process used a not really efficient way to detect html encoding strings in texts.
All calling methods had been adoped to call the new class in an enhanced way with less parameters.

Many classes in interfaces used a XML encoding only (instead of full html conversion from unicode to html); this behavior was not changed with this commit but should be controlled again since it points out possible XSS leaks

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5295 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-10-22 18:59:04 +00:00
orbiter
ff68f394dd fix for problem with balancer and lost crawl profiles: 
if crawl profile ist lost, no robots.txt is loaded any more

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5258 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-10-08 18:26:36 +00:00
apfelmaennchen
3717d2057a YaCy-UI: fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1483 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5251 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-10-05 18:50:43 +00:00
orbiter
d0bdcdd57c small changes to attributes of DoS attack protection parameters 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5246 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-10-03 19:44:42 +00:00
apfelmaennchen
2c23e6ad34 YaCy-UI: added some features for Admin Console 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5241 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-10-03 05:32:31 +00:00
apfelmaennchen
efcae14714 YaCy-UI: 
- added 'Open' button to search result toolbar
- lets you open all selected search results in new window/tab
- added 'any language' filter as default

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5240 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-10-02 17:38:03 +00:00
orbiter
820a03f9d6 - removed some warnings 
- used fix in SVN 5233 for ysearch.java and search.java

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5237 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-10-01 20:20:39 +00:00
apfelmaennchen
7f88dadf87 YaCy-UI: added a notice to a bug in bookmarks folder recursion! 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5221 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-29 06:02:48 +00:00
apfelmaennchen
cc9f1e9dd9 YaCy-UI: enabled edit bookmarks in toolbar 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5215 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-27 21:15:30 +00:00
apfelmaennchen
00c843f7d9 YaCy-UI: 
- added scrollbar to folder widget
- slightly improved Ajax handling for folder and tag widget

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5214 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-27 11:38:20 +00:00
apfelmaennchen
848dbbd280 YaCy-UI: 
- added bookmark folder widget to sidebar (it uses the xbel.xml with XSLT!

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5206 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-24 21:09:12 +00:00
apfelmaennchen
8b6a0376db YaCy-UI: 
- another XSS fix (TagCloud)
- upload of missing icons for toolbar

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5198 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-22 05:19:37 +00:00
apfelmaennchen
2ff6f0360e YaCy-UI: small fixes to accomodate Safari & Opera 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5196 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-21 14:26:35 +00:00
apfelmaennchen
a28af9d4c4 YaCy-UI: enabled language selection - special thanks to orbiter 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5195 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-21 10:49:05 +00:00
apfelmaennchen
1d9f0234c8 YaCy-UI: 
- added basic functionallity to add bookmark(s) from search results
- re-design of sidebar search form
- replaced classic search with compare search 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5194 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-21 10:04:51 +00:00
orbiter
00c1535f84 added ranking and evaluation of language type in a search 
the wanted language is taken from the browser user-agent string

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5192 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-21 00:04:42 +00:00
apfelmaennchen
e201ad0e9f YaCy-UI: 
- moved search result to flexigrid
- added a toolbar similar to YaCyBar

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5191 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-20 22:21:56 +00:00
apfelmaennchen
06c07f2e0f YaCy-UI: fixed broken style for add bookmark dialog 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5190 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-20 19:33:20 +00:00
apfelmaennchen
a5f61e8b1b YaCy-UI: 
- fixed xss vulnurability
- improved add bookmark dialog

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5189 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-20 18:28:58 +00:00
apfelmaennchen
e071f759d2 YaCy-UI: small optical changes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5172 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-16 21:39:14 +00:00
apfelmaennchen
bbacf86fe8 - added /xml/bookmarks/posts/add_p.xml 
- security fix to /xml/bookmarks/posts/delete_p.xml
- YaCy-UI: added 'add' and 'delete' bookmarks

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5171 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-16 21:38:13 +00:00
apfelmaennchen
cd1ac5bb90 - fixed security issue with /xml/util/ynetSearch.xml 
- hopefully fixed YaCy-UI local search with async=false for Ajax-request

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5170 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-16 05:55:31 +00:00
orbiter
c73cf05ddd tried to fix local search in yacy-ui 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5169 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-15 21:56:53 +00:00
apfelmaennchen
99ff478d63 YaCy-UI: small optical fix 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5167 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-15 20:30:33 +00:00
orbiter
7e24c51fd5 - removed alternative search page in main menue in favor of rich client search page 
- added necessary option to search request of yacy-ui to get snippets

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5165 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-15 19:00:22 +00:00
apfelmaennchen
7c5867a832 Major update to YaCy-UI 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5162 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-15 17:18:07 +00:00
orbiter
4fbee21cea - added fetch-ahead again (had been removed in last commit) 
- reverted default query mode to verify=false

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5111 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-03 23:50:13 +00:00
orbiter
05dbba4bab added logging conditions to all fine and finest log line calls 
this will prevent an overhead for the generation of the log lines in case that they then are not printed

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5102 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-03 00:30:21 +00:00
lotus
3fbfd5a78b * fix for non-changing offset on new search term 
* dht-heap doesn't has to be deleted (5097), we simply write a new one on exit
* do not install YaCy in startup because a Windows-shutdown might corrupt something. Installing YaCy as a service would solve this.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5099 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-09-02 15:09:31 +00:00
danielr
cd19d0aee6 - added warnings for failed transferRWI (dht-in) 
- fixed parseMultipart (uncompress gzipped body) (dht-in)
- fixed parseMultipart (using content-length only if uncompressed)
- better gzipped POST (chunked instead of content-length) (dht-out)



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5096 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-08-29 09:42:39 +00:00
orbiter
536e77e8b7 modifications towards a single database operation to read/write http header and cached file at once: 
- removed distinction between header file types for http and ftp; ftp is simulated by using http properties
- removed all old resourceInfo classes that handled this distinction
- introduced a new distinction between http request and http response objects
- unified new response objects with two other object types that had been introduced elsewhere
- changed all servlet call methods to use the new http request header object type
- divided static object keys for http header properties into request and response types
- refactoring here and there (a large number of type changes and many methods merged/moved)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5079 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-08-25 18:11:47 +00:00
apfelmaennchen
296fa2265b YaCy-UI: removed unused Servlet ymarks.java 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5076 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-08-24 08:47:37 +00:00
apfelmaennchen
a9cf3b42c4 YaCy-UI: removed unused JavaScript 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5075 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-08-24 08:43:07 +00:00
apfelmaennchen
08bf3fd235 YaCy-UI: updated to jQuery-ui 1.6b 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5074 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-08-24 08:41:52 +00:00
danielr
753a1ae430 - changed default browser from netscape to firefox 
- fixed "Inefficient use of keySet iterator instead of entrySet iterator" [WMI_WRONG_MAP_ITERATOR, FindBugs]
- fixed some possible null pointer accesses


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5063 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2008-08-20 07:54:56 +00:00