Commit Graph

4786 Commits

Author SHA1 Message Date
orbiter
d3d41e2ee4 - fixed problem with searching with quotes (still not complete, but not as bad as before)
- fixed parsing of crawl-delay statements when seconds were given with float numbers
- enhanced performance of profiling (not too many loggings; not more than one per second)
- removed some debug output
- fixed wrong return type in logging
- added a logging condition in httpd to prevent that logging statements are generated when they are not written (should be added everywhere!)
- fixed wrong word distance computation in RWI management


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5101 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-09-02 23:49:48 +00:00
f1ori
3a0e96b552 * only create one debian package for all architectures
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5100 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-09-02 21:53:45 +00:00
lotus
3fbfd5a78b * fix for non-changing offset on new search term
* dht-heap doesn't has to be deleted (5097), we simply write a new one on exit
* do not install YaCy in startup because a Windows-shutdown might corrupt something. Installing YaCy as a service would solve this.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5099 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-09-02 15:09:31 +00:00
danielr
219b93df6a - fixed internal error after receiving chunked POST
- removed debug output
- added info for "501 Unknown" messages



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5098 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-29 13:51:22 +00:00
lotus
c245c7a45e delete index.dhtin/out.heap if restore fails
see http://forum.yacy-websuche.de/viewtopic.php?p=9613#p9613

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5097 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-29 13:10:41 +00:00
danielr
cd19d0aee6 - added warnings for failed transferRWI (dht-in)
- fixed parseMultipart (uncompress gzipped body) (dht-in)
- fixed parseMultipart (using content-length only if uncompressed)
- better gzipped POST (chunked instead of content-length) (dht-out)



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5096 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-29 09:42:39 +00:00
lotus
89cf795a5c proper default priority on first start (Windows)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5095 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-29 07:01:38 +00:00
orbiter
016f57d714 fixed a dead link
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5094 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-28 21:45:58 +00:00
orbiter
df4ff423c4 added additional properties to query id's to distinguish search events better
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5093 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-28 21:15:59 +00:00
danielr
d6d9b0f14a fixed transferRWI.html 'Read timed out'
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5092 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-28 08:37:51 +00:00
danielr
e503158527 Proxy: fix for never ending loading after POST
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5091 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-27 20:46:34 +00:00
danielr
73519cbdca fixed pid-file for linux start-script
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5090 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-27 19:18:38 +00:00
danielr
1a1d57e449 Proxy: added binary passthrough for POST
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5089 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-27 08:07:18 +00:00
apfelmaennchen
aa6ae77e5e - autoReCrawl: fix for filter settings
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5088 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-26 21:51:05 +00:00
apfelmaennchen
8ae29bad57 - fix to previous change of Crawl Profile Names
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5087 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-26 20:42:29 +00:00
f1ori
b8ee04daf1 fix for http://www.yacy-forum.org/viewtopic.php?f=2&t=160 (wrong url in form)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5086 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-26 18:45:19 +00:00
apfelmaennchen
434104e4a0 - change Crawl profile name for autoreCrawl
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5085 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-26 18:08:48 +00:00
danielr
9ff4fc11da partial fix (images,audio,video) for proxy and content-type problem http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1374
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5084 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-26 16:34:24 +00:00
lotus
0df2e47012 changed auto recrawl to comply with new date format
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5083 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-26 13:36:10 +00:00
lotus
d9d9c522a1 addendum to last commit
moved recrawl times for standard profiles to constants
calculate new specific dates in cleanup job

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5082 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-26 13:20:18 +00:00
lotus
480497f7c9 changed recrawl
use a specific date to define old documents
this solves an unwanted recrawl-loop during a running crawl

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5081 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-25 20:31:32 +00:00
orbiter
da1b0b2fc6 added two new classes that will be used for the new htcache
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5080 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-25 18:22:23 +00:00
orbiter
536e77e8b7 modifications towards a single database operation to read/write http header and cached file at once:
- removed distinction between header file types for http and ftp; ftp is simulated by using http properties
- removed all old resourceInfo classes that handled this distinction
- introduced a new distinction between http request and http response objects
- unified new response objects with two other object types that had been introduced elsewhere
- changed all servlet call methods to use the new http request header object type
- divided static object keys for http header properties into request and response types
- refactoring here and there (a large number of type changes and many methods merged/moved)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5079 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-25 18:11:47 +00:00
f1ori
04310a7255 * added long options and help-option to linux startscript
* redirect all error messages to /dev/null


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5078 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-25 17:17:01 +00:00
borg-0300
08cdf6db8a fix for wrong "VegaYacyB" peers
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5077 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-24 11:30:00 +00:00
apfelmaennchen
296fa2265b YaCy-UI: removed unused Servlet ymarks.java
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5076 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-24 08:47:37 +00:00
apfelmaennchen
a9cf3b42c4 YaCy-UI: removed unused JavaScript
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5075 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-24 08:43:07 +00:00
apfelmaennchen
08bf3fd235 YaCy-UI: updated to jQuery-ui 1.6b
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5074 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-24 08:41:52 +00:00
danielr
4d937f6b21 fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=1396
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5073 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-22 23:46:32 +00:00
lotus
5192081283 workaround for BindException at restart on Vista
therefore IPv6 support is lost (Windows only)
see http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6286189  and http://forum.yacy-websuche.de/viewtopic.php?t=1385

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5072 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-22 18:46:14 +00:00
apfelmaennchen
58d7e6f1a6 - some small, rather optical changes to bookmarks
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5071 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-22 18:19:21 +00:00
apfelmaennchen
bd931a82f7 - added dynamic filters to autoReCrawl.conf
- Restrict to sub-path: sub
- Restrict to start-domain: dom

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5070 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-22 18:05:05 +00:00
apfelmaennchen
8551e1d106 - added Info link (/xml/util/getpageinfo_p.xml) to bookmarks
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5069 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-21 21:28:57 +00:00
apfelmaennchen
8d1bedfc3a - added bookmarkTitle to CrawlStart_p.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5068 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-21 21:07:21 +00:00
apfelmaennchen
b3fc5e96a3 - removed unused import from bookmarksDB
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5067 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-20 21:26:06 +00:00
apfelmaennchen
bc048db7b6 - bugfix for bookmarksDB's rebuildDates()
- dates are now saved as String.valueOf(TimeStamp)
- it might be a good idea to delete (backup) bookmarkDates.db and restart YaCy to rebuild it 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5066 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-20 21:25:05 +00:00
danielr
3c68905540 remove redundant null checks
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5065 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-20 08:37:39 +00:00
danielr
f9a715dc33 simplified Linux start-script
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5064 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-20 07:57:47 +00:00
danielr
753a1ae430 - changed default browser from netscape to firefox
- fixed "Inefficient use of keySet iterator instead of entrySet iterator" [WMI_WRONG_MAP_ITERATOR, FindBugs]
- fixed some possible null pointer accesses


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5063 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-20 07:54:56 +00:00
orbiter
7989335ed6 Preparations to replace the HTCache with a new storage data structure:
- refactoring of the HTCache (separation of cache entry)
- added new storage class for BLOBs. (not used yet, this is half-way to a new structure)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5062 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-19 14:10:40 +00:00
f1ori
c05edba6b8 * added text/html-URL to opensearchdescription, so one-click-installation in firefox is possible
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5060 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-17 21:13:41 +00:00
danielr
be28af50f5 - fixed "yacy2yacy no proxy"-problem
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5058 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-17 10:16:32 +00:00
f1ori
6450a51473 * fixed quoting problem in initscript
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5057 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-14 21:38:05 +00:00
f1ori
f99c307eff * correct debian build dependencies
* add huge mem page detection in general initscript
* disable logging completely in jmimemagic-library


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5056 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-14 21:01:21 +00:00
orbiter
bdae051d9a - extended new performance graph (better timing)
- added paths for new libraries in classpath for eclipse
- refactoring to remove compiler warnings (static access to finals variables)
- removed some unused import

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5055 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-13 10:37:53 +00:00
danielr
d9cea5ff23 removed annotations which broke the build with java 1.5
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5054 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-13 09:07:23 +00:00
f1ori
7d8f332bbd * build yacy for debianpackage only once
* do not try to sign .changes file
* add some java arguments from ./startYACY.sh to addon/yacyInit.m4


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5053 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-11 20:26:18 +00:00
danielr
2547700de8 relaxed dependencies for use with debian 4.0 etch
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5052 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-11 15:12:49 +00:00
f1ori
c2d49cc01e * add build target "deb" to create debian packages from svn
still needs testing...


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5051 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-10 15:26:13 +00:00
f1ori
6a550c64f1 * clean up svn ignorelist, added yacy.pid and logfiles
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@5050 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-08-10 14:56:02 +00:00