Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
4,018 Commits 1 Branch 20 Tags 836 MiB
e3e4f06be4
Commit Graph

2687 Commits

Author SHA1 Message Date
orbiter
e3e4f06be4 enhanced search result preparation in the case that no result is found (fast abandon of search) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4273 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-12 14:18:42 +00:00
fuchsi
1bd02762de Improve HTTP/ICAP header processing. 
- workaround for illegal line endings (LF only), closes: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=595
- fixed bug where we didn't break the processing immediately on EOS (the loop was run until the buffer was completely filled with -1)
- further performance improvements (one simple loop, avoid double processing of every byte and unnecessary temporary buffers)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4270 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-12 06:37:18 +00:00
orbiter
01554f4012 fixed bug with double-check in crawler 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4269 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-12 01:32:25 +00:00
orbiter
b1e08d354c repaired indexing after search snippet loading 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4268 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-12 00:33:26 +00:00
orbiter
48138952ff added memory measurement for index recreation to avoid OOM during index RAM space extension 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4267 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-11 15:07:03 +00:00
orbiter
9e23acf2d6 introduced new 'authority' ranking property 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4265 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-11 01:32:58 +00:00
orbiter
a1b80017e0 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=594&p=3630#p3630 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4264 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-10 15:00:01 +00:00
orbiter
a3bfd668aa opening of array files at startup time, not when first time the web index is accessed 
this speeds up the first search after startup

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4263 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-10 02:40:16 +00:00
orbiter
ca488e03f5 fixed authorization case 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4262 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-10 02:04:48 +00:00
orbiter
6a3a292015 - smoothed ymage font 
- changed position of status banner

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4261 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-10 01:47:04 +00:00
low012
7397152e04 *) quick hack for antialiasing, works only on borders now => less blurry image 
*) code is not finished, needs refactoring, still thinking about how to do it best


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4260 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-09 19:17:24 +00:00
orbiter
2954f96fae - removed public peer info box on status page, this info can now be seen in the status banner 
- added peer count to banner
- added some values to protected status box

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4257 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-08 01:39:59 +00:00
low012
4eb40c4f61 *) added 2 filters: blur and antialiasing (which in fact is nothing more than a mild blur) to ymageMatrix 
*) antialiasing is used for logo in banner


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4256 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-07 22:51:13 +00:00
orbiter
aeb1cf83a6 - corrected banner link (relative now) 
- changed color mode (replace) for banner
- changed default color (fits to default skin) of banner in status

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4255 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-07 21:25:36 +00:00
orbiter
e22014dc83 some memory enhancements when generating and displaying ymage objects 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4253 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-07 02:15:12 +00:00
orbiter
f243e338cf implemented online caution also for local and remote search 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4252 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-06 21:53:17 +00:00
orbiter
c57eb76b13 removed CMY color model from ymage classes and re-introduced RGB color model 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4249 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-06 01:06:17 +00:00
orbiter
b46bcaa5d8 changed method of profiling 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4248 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-04 20:19:13 +00:00
low012
76cd6ed6f6 *) New methods to insert bitmaps that feature transparencies. 
*) Logo background is transparent now. (Using pixel at (0,0) to determine which color is transparent. Too dirty?)
*) Logo is loaded through filesystem instead of HTTP now.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4247 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-04 19:45:50 +00:00
orbiter
be214e594f - generalized ymage initialization options 
- auto-adoption of performance memory graph to needed dimension

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4246 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-03 02:35:28 +00:00
low012
ee8a177c26 *) Logo is in the middle of free space now. 
*) Fixed bugs in insertBitmap()


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4245 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-02 21:20:11 +00:00
low012
72698fcd36 *) Banner features a logo now. It does not look nice, but at least it works. Banner is not finished yet. 
Which path do I have to set for IMAGE (htroot/env/grafics/yacy.gi) if I want to load it through the file system and not via HTTP?


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4244 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-12-02 20:37:12 +00:00
fuchsi
39d0f10ca1 Fix parsing oof dates in HTTP headers. 
RFC 2616 requires a client to support RFC 1123 (default), RFC 1036 and ANSI C formatted date strings (we only supported 1123 before).

Closes: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=525 (and probably others). There are servers which break the standards, please report those "DATE ERROR" messages if they contain a "sane" date string.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4243 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-30 20:47:27 +00:00
orbiter
aefb3f7765 added memory graph picture to PerformanceMemory_p.html 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4241 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-30 03:22:42 +00:00
orbiter
9b0ae4b989 added referrer to remote crawl url list 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4236 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-29 13:58:00 +00:00
fuchsi
18e516317d Fix problem with buggy HTTP-Servers which send illegal control characters in HTTP-Headers, they are ignored now. 
Thx to celle for the patch and see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=560 for more information.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4235 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-29 06:02:45 +00:00
orbiter
7d5544e9b1 added some security checks to new remote crawl pull method to prevent that indexer is overloaded 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4234 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-29 02:54:59 +00:00
orbiter
89b9b2b02a redesigned remote crawl process: 
- instead of pushing urls to other peers, the urls are actively pulled
  by the peer that wants to do a remote crawl
- the remote crawl push process had been removed
- a process that adds urls from remote peers had been added
- the server-side interface for providing 'limit'-urls exists since 0.55 and works with this version
- the list-interface had been removed
- servlets using the list-interface had been removed (this implementation did not properly manage double-check)
- changes in configuration file to support new pull-process
- fixed a bug in crawl balancer (status was not saved/closed properly)
- the yacy/urls-protocol was extended to support different networks/clusters
- many interface-adoptions to new stack counters

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4232 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-29 02:07:37 +00:00
fuchsi
69521d92e5 Add another external dependency from PDFBox package ("Bouncy Castle"). This is necessary for parsing of some encrypted PDF files. 
bcprov-jdk14-132.jar is the binary jar as it is provided in the PDFBox-0.7.3 package (same as our FontBox, PDFBox packages).

Closes: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=453


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4231 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-27 23:13:26 +00:00
orbiter
90a02990d2 NPE fix, see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=549&hilit=&p=3383#p3383 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4230 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-23 09:26:35 +00:00
orbiter
2fcd18a972 - fixed bad behaviour of search event worker processes 
- fixed export of url lists in xml

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4229 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-23 01:08:16 +00:00
orbiter
445c0b5333 added domain list extraction and html export format 
to URL administration menu http://localhost:8080/IndexControlURLs_p.html

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4228 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-22 20:47:06 +00:00
orbiter
d8d77fc4b2 fix for NPE, see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=549&hilit=&p=3368#p3368 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4227 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-22 18:15:28 +00:00
orbiter
bf6952abe7 - added url export to http://localhost:8080/IndexControlURLs_p.html 
- removed command-line option to export urls

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4226 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-22 16:52:44 +00:00
orbiter
af10f729df fixed image search and favicon loading 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4225 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-22 01:34:29 +00:00
orbiter
c48b73cda2 redesign of ranking data structure 
- the index administration now uses the same code base for url selection and collection
  as the search interface. The index administration is therefore a good test environment for
  ranking order control
- removed old postsorting-algorithms, will be replaced with new one
- fixed many bugs occurred before during ranking; especially the contraint filtering method
  removed too many links
- fixed media search flags; had been attached to too many urls. The effect should be a better
  pre-sorting before media load within snippet fetch

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4223 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-21 23:14:57 +00:00
orbiter
6f1308da2f - some enhancements to IndexControlURLs (shows more links, connects referrer to another query) 
- some refactoring to search process

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4222 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-17 01:53:02 +00:00
orbiter
c527969185 - enhanced monitoring of ranking parameters 
for details, please try http://localhost:8080/IndexControlRWIs_p.html
- fixed computation of ranking ordering in some cases

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4220 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-16 14:48:09 +00:00
orbiter
bd5673efbe added cleaning of search event before opening the index administration 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4219 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-15 12:49:13 +00:00
orbiter
55da871211 preparations for better ranking: better debugging of index properties 
to do this, the index administration interface was extended.
It is now possible to select parts of a index.
See properties shown in interface after a word search for details.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4218 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-15 03:03:18 +00:00
low012
383dc815d2 *) fix for commit 4212 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4217 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-14 19:14:53 +00:00
orbiter
3491531cea - fixed 'appears in url' flag in index generation 
- extended index administration page, shows some properties to the web links now

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4216 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-14 01:15:28 +00:00
orbiter
ec7ba0d3d0 - fixed problem with too small sort fields (sortbound was not set) 
- slightly changed handling of date in indexURLEntry

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4214 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-13 02:24:10 +00:00
orbiter
bc2368e907 fix for problem with remote crawl referrers 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4210 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-12 16:32:50 +00:00
orbiter
875096552f fix for NPE in case that remote search results are empty 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4209 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-12 15:54:50 +00:00
orbiter
64b3b79e44 - fix for termination problem with uniq() 
- addition to seed dna interpretation

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4208 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-12 14:39:30 +00:00
orbiter
0abf33ed03 - tried to remove deadlock 
- enhanced searchtime in kelondroRowSets
- enhanced uniq() - reverse enumeration causes less time in case of mass removal of doubles

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4207 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-12 01:14:51 +00:00
low012
a4010f7dc8 *) fixed bug where dots were added after numbers < 1000: "123" was transformed to "123." which is undesirable 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4206 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-11 21:42:50 +00:00
orbiter
2421127612 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=513&hilit= 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4204 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-11 01:32:54 +00:00
orbiter
d0d2771883 disabled multiprocessoring of rowCollection.sort for testing purpose 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4202 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2007-11-11 00:28:22 +00:00