Commit Graph

2766 Commits

Author SHA1 Message Date
orbiter
b033a80750 better control of failure in node seek of kelondroTree
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2686 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-10-02 00:13:19 +00:00
rramthun
ca8ef0ca9f *)Documented the lng-file format
*)Updated language files to the new standard, especially German
*)Wrote language highlighting definition for Notepad++
*)Corrected News.html

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2685 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-10-01 12:23:35 +00:00
orbiter
310f1c41cd added option to see ranking scores in surftipps
and some cleanups

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2684 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-30 23:28:03 +00:00
orbiter
7c0e6de366 bugfix for surftipps votes (wrong page)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2683 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-30 23:06:38 +00:00
orbiter
3ad0709b53 added a delete button to crawl profile list.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2682 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-30 22:35:59 +00:00
allo
971bfc6f15 added ChangeLog based on Rolands Newsletter.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2681 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-30 11:07:32 +00:00
theli
a2e3095044 *) Bugfix. Add missing plasmaParserDocument.close() calls
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2680 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-30 10:09:01 +00:00
theli
cd5f349666 *) Better handling of large files during parsing
Extracted text of files that are larger than 5MB is stored in a temp file instead of keeping it in memory
*) plasmaParserDocument.java; getText now returnes an inputStream instead of a byte array
*) plasmaParserDocument.java: new function getTextBytes returns the parsed content as byte array
   Attention: the caller of this function has to ensure that enough memory is available to do this 
   to avoid OutOfMemory Exceptions
*) httpd.java: better error handling if the soaphander is not installed
*) pdfParser.java: 
   - better handling of documents with exotic charsets
   - better handling of large documents
   - better error logging of encrypted documents
*) rtfParser.java: Bugfix for UTF-8 support
*) tarParser.java: better handling of large documents
*) zipParser.java: better handling of large documents
*) plasmaCrawlEURL.java: new errorcode for encrypted documents
*) plasmaParserDocument.java: the extracted text can now be passed
   to this object as byte array or temp file   

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2679 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-30 09:31:53 +00:00
theli
8b2ceddb91 *) Displaying servere and warning logging messages in different colors on ViewLog_p.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2678 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-30 08:12:22 +00:00
low012
f8ac694e51 *) fixed a bug where searchword in snippets were not displayed bold in front of a punctuation mark (see http://www.yacy-forum.de/viewtopic.php?p=25998)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2677 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-30 00:27:42 +00:00
orbiter
df1629b05a - code cleanup
- version 0.471
- moved surftipps to own web page


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2676 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-29 22:27:20 +00:00
allo
b78d171b1e Windows installer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2675 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-29 21:13:56 +00:00
theli
c665f6cddb *) handling of quotes in charset string
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2674 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-28 06:29:15 +00:00
theli
b73efd5565 *) missing changes needed because of last commit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2673 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-28 05:48:28 +00:00
theli
65c1f13d11 *) migration to newer odt parser lib
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2672 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-28 04:47:39 +00:00
theli
140ddba93f *) adding soap functions to pause and resume the crawler
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2668 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-27 05:22:43 +00:00
theli
ed8227d222 *) Bugfix for NullpoinerException in IndexCreateIndexingQueue_p.java
See: http://www.yacy-forum.de/viewtopic.php?p=25874

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2667 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-27 04:35:02 +00:00
theli
c0f7a4124c *) Bugfix for soap templates
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2666 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-27 04:24:32 +00:00
orbiter
2463e5624a 'quick' release 0.47
- documentation update
- necessary bugfixes (missing css for new peers)
- reduced effect of search result redundancy filter
- removed some debug output, but not all

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2665 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-26 23:41:54 +00:00
theli
3433dfb5e2 *) Bugfix for soap search template: correction for resultCount tags, cdata for snippet tag
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2664 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-26 16:18:04 +00:00
theli
d42dcead1d *) Bugfix renaming snippet tag in soap search template
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2663 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-26 16:11:38 +00:00
theli
49fbb688df *) SOAP: old urlInfo renamed to urlInfoByHash, new urlInfo Function added.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2662 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-26 15:14:33 +00:00
theli
8f143d516b *) make snippet fetcher accessible via soap api
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2661 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-26 15:07:16 +00:00
theli
97615af406 *) Restructuring of YaCy SOAP services
- general functions moved to abstract service class
   - service class splitted into SearchService, CrawlService, StatusService
*) Bugfix for SOAP search services
   - Attention: some xml tages where renamed
   See: http://www.yacy-forum.de/viewtopic.php?p=25877
*) New SOAP service function urlInfo to view the parsed content of an URL
   See: http://www.yacy-forum.de/viewtopic.php?p=25869

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2660 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-26 14:47:44 +00:00
theli
241b881560 *) Redesign of YaCy SOAP handler
- should be more fail-safe now
   - better handling of compressed request bodies
   - better handling of persistent connections
   - better handling of AxisFaults

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2659 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-26 12:24:40 +00:00
theli
009a33170b *) Content-Location header added
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2658 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-26 04:32:01 +00:00
rramthun
b0cab1e731 *)Adapted surftipps to use common 0/1 parameters
*)Added translation of WatchCrawler.html
*)Changed format of German translation. Formal description will probably follow.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2657 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-24 20:15:28 +00:00
rramthun
f17a91313e *)Updated phosphor.css for XHTML
*)New grey skin
Thanks to Philipp Redeker!

*)Renamed old skins until somebody updates them.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2655 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-24 17:55:21 +00:00
theli
1aa07a52cd *) Bugfix for UnsupportedEncodingException if the media type contains multiple parameters
See: http://www.yacy-forum.de/viewtopic.php?p=25832#25826

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2654 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-24 15:50:51 +00:00
allo
4922ab8920 try to fix a nullpointer on snippet generation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2653 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-22 22:51:44 +00:00
hermens
d8fde14c3a Make maximum number of words in DHT-In cache configurable at runtime
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2652 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-22 12:44:58 +00:00
theli
625c2ce6b1 *) bugfix for snippet fetching problem if content but not http header is available in cache
See: http://www.yacy-forum.de/viewtopic.php?p=25748

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2651 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-22 11:55:28 +00:00
theli
813a8a8179 *) migration of mimeTypeParser to jmimemagic 0.1
- better mimetype detection for rss feeds
   - better mimetype detection for odt documents (less memory consuming)
   - two new detector classes implementing MagicDetector interface of jmimemagic

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2650 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-22 11:40:46 +00:00
hermens
3f5a4153a0 Make Peers more receptible to transferred indexes
- Set MaxWordCount for dhtInCache to indexDistribution.dhtReceiptLimit
  so that the inCache gets flushed when the limit is passed
- Modify flushCacheSome to flush enough words to get below MaxWordCount immediately



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2649 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-22 10:58:58 +00:00
hydrox
740696f6c3 *) few fixes XHTML-validation ( there is still much to do)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2648 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-22 08:04:24 +00:00
theli
57415b6889 *) Bugfix for surftipps UTF-8 problem
See: http://www.yacy-forum.de/viewtopic.php?t=2864

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2647 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-22 05:40:29 +00:00
theli
706572f18d *) Bugfix for ArithmeticException caused by setting max crawling thread count was to 0
See: http://www.yacy-forum.de/viewtopic.php?t=2862

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2646 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-22 04:29:31 +00:00
orbiter
2d3b96eeba bugfixes for surftipps
- added missing authorization check for votes
- second vote on same entry was possible after complete publishing of current vote

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2645 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-21 21:37:40 +00:00
hydrox
85f3617835 *) moved HTML from class-file to template-file (please check if it is valid HTML)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2644 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-21 20:36:46 +00:00
hydrox
9434dba8f2 *) corrected title of IndexCleaner_p.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2643 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-21 07:26:41 +00:00
allo
b0a4fcce8c fix from theli
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2642 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-20 18:03:24 +00:00
theli
b6c7b91582 *) Parser now throws an ParserException instead of returning null on parsing errors (e.g. needed by snippet fetcher)
*) better logging of parser failures
*) simplified usage of plasmaparser through switchboard
*) restructuring of crawler
   - crawler now returns an error message if it is used in sync mode (e.g. by snippet fetcher)
*) snippet-fetcher: more verbose error messages
*) serverByteBuffer.java: adding new function append(String,encoding)
*) serverFileUtils.java: adding functions to copy only a given number of bytes between streams


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2641 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-20 12:25:07 +00:00
orbiter
aa38721cf6 new features for surftipps
- new generation with less memory
- removal of doubles
- positive votes can generate entries without original news (so they can live on)
- link deletion on search results are now also negative votes for surftipps (but they may rarely hit any news)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2640 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-20 12:01:51 +00:00
theli
64b2ef5aae *) Trying to bugfix shutdown problem
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2639 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-20 10:13:23 +00:00
orbiter
e03427871e enhanced surftipps:
- added switchh to show or hide surftipps
- more news contribute to surftipps
- added voting system for surftipps

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2638 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-20 07:17:41 +00:00
theli
e745b63c77 *) Bugfix for different behavior of indexDistributeWhileCrawling to other checkboxes on IndexControl_p.html
See: http://www.yacy-forum.de/viewtopic.php?t=2849

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2637 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-20 04:44:56 +00:00
theli
1dc12d6659 *) Bugfix for shutdown problem caused by cacheScan thread
See: http://www.yacy-forum.de/viewtopic.php?p=25729

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2636 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-20 04:36:25 +00:00
borg-0300
42173462f5 rename cutUrlText to shortenURLString;
other little things;

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2635 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-19 20:47:45 +00:00
borg-0300
af1d89e381 check url == null added;
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2634 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-19 20:12:26 +00:00
theli
cc667b0aa5 *) htmlFilterContentScraper.java: adding support for link tag
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2633 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-09-19 16:13:13 +00:00