Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
7,525 Commits 1 Branch 20 Tags 836 MiB
44d74f8f89
Commit Graph

695 Commits

Author SHA1 Message Date
orbiter
4c013d9088 more UTF8 getBytes() performance hacks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7649 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-04-12 05:02:36 +00:00
cominch
9ac02caf00 different initialization of empty variables in alternative constructor. This leads to wrong interpretation of user credentials, resulting in unnecessary "@" in front of host, and different urlhash values. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7646 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-04-06 10:59:31 +00:00
orbiter
57ce1fb491 reverted synchronization from SVN 7641 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7643 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-04-04 20:31:02 +00:00
orbiter
17530ca7b5 fix for bug http://bugs.yacy.net/view.php?id=10 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7642 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-04-04 12:20:20 +00:00
orbiter
7c8e764201 removed synchronization again... 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7641 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-04-04 10:13:30 +00:00
orbiter
96c32e87b0 fixes to crawler and new user-agent crawl-delay handling 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7640 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-04-04 09:47:18 +00:00
orbiter
cb6f709a16 - enhancements in surrogate reading 
- better display of map in location search

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7636 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-04-02 00:11:37 +00:00
low012
1ff9947f91 *) added new user right: extended search right (allows to define users who can query more results than anonymous users) 
*) cleaned up code a little bit

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7635 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-04-01 23:32:40 +00:00
orbiter
564184909a enhanced the surrogate parser: better reading of UTF-8 characters 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7634 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-04-01 11:05:42 +00:00
orbiter
156cf02703 - added an index constraint 'has location' to the condenser 
- added evaluation of the 'has location' constraint to search using the /location operator


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7633 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-31 09:41:30 +00:00
orbiter
41b8d7f655 fix for url normalization (no backpath resolving in post parameters) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7632 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-31 09:40:01 +00:00
orbiter
0430a94eaa the location search shows now not re-evaluated locations but only such locations that are attached as metadata to web pages 
- added parser for in-text appearing geo-locations
- added geo-locations to rss search result
- added evaluation of metadata-attached geo-locations in yacysearch_location to show search results within a map


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7631 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-30 23:26:36 +00:00
orbiter
8412f8787d fix for http://bugs.yacy.net/view.php?id=8 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7630 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-30 08:17:25 +00:00
orbiter
9b25d07295 - added geo information parsing to html parser 
- extended metadata information in index with geolocalisation
- added display of location in yacydoc and ViewFile

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7629 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-30 00:49:47 +00:00
lotus
cbf87fe72f write PID to yacy.running 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7627 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-26 15:11:29 +00:00
orbiter
b1a8d0c020 enhancements to web cache and less strict caching rules 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7620 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-22 10:35:26 +00:00
orbiter
f3baaca920 - enhancements to DNS IP caching and crawler speed 
- bugfixes (NPEs)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7619 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-22 09:34:10 +00:00
f1ori
df71776929 * fix bug #7 
* log requires poison to finish, so Base64Order main-function doesn't finish, when called from debian configure script


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7616 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-21 19:42:22 +00:00
orbiter
78d4c45d09 enhancement during search process: fast fail of search in case that all index feeder have terminated. 
This change should affect filtering and navigators and should cause that search navigation gets faster

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7614 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-21 13:05:51 +00:00
orbiter
a50f28e6e7 - fixed missing save operation for peer name change 
- fixed import of mediawiki dump files
- added script to add mediawiki dump files

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7609 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-19 23:52:09 +00:00
orbiter
2b5f8585bf performance hack for Balancer and ip address parsing 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7608 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-17 21:09:18 +00:00
orbiter
b1d133b69f another anhancement to the ThreadDump function: better multiple dumps and filtering out of not interesting dump parts 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7606 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-17 20:48:39 +00:00
orbiter
a35d513bd8 fix for not-deleted .gap and .idx files 
see also: http://forum.yacy-websuche.de/viewtopic.php?p=22128#p22128

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7605 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-17 17:09:19 +00:00
orbiter
a6935e7dc8 fix for active dns resolving: do not resolve in case that the dns server is not available (offline mode) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7604 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-16 07:05:10 +00:00
orbiter
859c99886c fix for multiple thread dump 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7603 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-15 23:05:51 +00:00
orbiter
61acf55da4 avoided using a synchronized(this) for the hash computation to prevent that the lock on the object is (accidently) stolen by another thread and replaced this synchronization using the protocol object. Made also the protocol object final. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7602 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-15 09:52:39 +00:00
orbiter
c2a968c23f fix for bug in formatting in ThreadDump 
and added hint for linux/Mac users that they may use the LOCKED feature using the start option -l

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7601 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-15 08:39:05 +00:00
orbiter
078ecacf61 avoid synchronization in DigestURI hash requests 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7599 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-15 00:47:30 +00:00
orbiter
1989ebc24b removed more warnings 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7598 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-14 22:52:30 +00:00
orbiter
0324de1467 removed debug line 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7597 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-14 21:34:42 +00:00
orbiter
1aba7869bf patch for Windows: do not use the thread lock feature from previous commit if used on Windows 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7596 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-14 21:33:36 +00:00
orbiter
0a11727374 added new feature for Thread dump: 
"THREADS WITH STATES: LOCK FOR OTHERS"
will show only such threads that lock other threads. This is the 'opposite part' of the blocked threads.
Because that this uses a thread dump that is produced with a kill -3 on the PID of the process and such thread dumps are written by the Java core outside of System.out and Sytem.err it is necessary to read the dump from a log in the file system. Such a log is only written if YaCy is started with startYACY.sh on a linux system. That means:
this feature is only available on linux and Mac OS X if YaCy is started with ./startYACY.sh -l


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7595 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-14 21:32:20 +00:00
orbiter
b62b79675b removed type cast warnings 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7594 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-14 21:08:18 +00:00
orbiter
a07a1a8b1e removed type cast warnings 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7593 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-14 21:07:15 +00:00
orbiter
8edaccfedf removed unused variables 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7592 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-14 21:03:37 +00:00
orbiter
e6c3507b17 disabled some of the previous changes (did not work in openjdk) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7591 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-14 20:48:36 +00:00
orbiter
f9e5c21083 update to thread dump logs 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7590 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-14 20:46:04 +00:00
orbiter
8f11d3a5bb redesigned the ScoreMap classes: 
- new concurrent score map using atom operation from java concurrency classes
- redesigned difference beween StaticScore and Dynamic Score into ScoreMap and ReversibleScoreMap allowed that many classes can now use simple ScoreMap Objects which can be used better in concurrent environments using the ConcurrentScoreMap
- switched from DynamicScore to ConcurrentScoreMap usage wherever possible

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7586 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-13 01:41:44 +00:00
orbiter
a564230c48 more enhancements against blocked threads occurred in seed age evaluation (blocks httpd in some cases) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7585 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-12 22:54:41 +00:00
orbiter
dc0db3550e avoid string conversion 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7584 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-11 00:59:27 +00:00
orbiter
694fa3a2a5 - replaced more direct string-based UTF-8 conversions by predefined UTF-8 conversion 
- changed menu structure slightly

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7583 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-10 23:25:07 +00:00
orbiter
30aed9824a moved getBytes() to UTF8.getBytes() to use a default String encoding 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7580 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-10 12:35:32 +00:00
lotus
cb6d307bba adding extension for parser 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7579 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-09 20:36:01 +00:00
orbiter
3820525464 more memory protection: auto-flush of caches in case of memory shortage 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7575 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-09 16:32:34 +00:00
orbiter
7962d35425 - removed file upload function in crawl start and replaced it with an input field for a file path where the crawl start file is loaded. This was necessary to support the API steering for file crawl starts, for two reasons: 
1) if the file is changed for a re-crawl this is not reflected in the steering because it would take the previously uploaded crawl start file
2) browsers do not submit the full path of the selected file even if this path is shown in the input field because of security reasons. There is no work-around or hack to make the submission of the full path possible

- fixed deletion of crawl start point urls in crawl stack and balancer double-check
- fixed a problem with steering self-call (no resolving of localhost)
- added more logging for the crawler to supervise why crawl urls are not taken by the loader
- added a javascript onload-function to select domain restriction in all cases where a crawl is started from a file or from a url
- fixed the restrict-to-domain pattern computation, added a 'www.'-prefix and added this functionality also to a crawl start from file 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7574 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-09 12:50:39 +00:00
orbiter
96bb33ed9b added default size to StringBuffer in logger (and it is not possible to replace the StringBuffer with a StringBuilder...) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7573 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-09 09:53:57 +00:00
orbiter
e1b6916423 always try to guess the size of a StringBuilder to prevent too many memory re-allocations 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7572 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-09 09:29:05 +00:00
low012
3b40b98256 *) set SVN properties 
*) minor changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7567 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-08 01:51:51 +00:00
orbiter
619b561a4a enhanced secondary search: index abstracts decompression is now much faster and does not cause strong CPU load after several searches with more than one word 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7565 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-07 23:12:39 +00:00
low012
bf27a72d53 *) set SVN properties 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7564 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-07 23:05:23 +00:00
low012
b649ce2dd7 *) minor changes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7563 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-07 22:59:19 +00:00
orbiter
70a996a06c reverted SVN 7557 because these classes are called using reflection. The class declaration is in the log configuration. Without these classes you get errors during runtime and a non-formatted log output, i.e.: 
STARTUP: Trying to load logging configuration from file /Data/workspace1/yacy/DATA/LOG/yacy.logging
Can't load log handler "net.yacy.kelondro.logging.ConsoleOutErrHandler"
java.lang.ClassNotFoundException: net.yacy.kelondro.logging.ConsoleOutErrHandler
java.lang.ClassNotFoundException: net.yacy.kelondro.logging.ConsoleOutErrHandler
	at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
	at java.util.logging.LogManager$3.run(LogManager.java:359)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.util.logging.LogManager.loadLoggerHandlers(LogManager.java:346)
	at java.util.logging.LogManager.initializeGlobalHandlers(LogManager.java:898)
	at java.util.logging.LogManager.access$900(LogManager.java:130)
	at java.util.logging.LogManager$RootLogger.getHandlers(LogManager.java:979)
	at java.util.logging.Logger.log(Logger.java:454)
	at java.util.logging.Logger.doLog(Logger.java:480)
	at java.util.logging.Logger.log(Logger.java:503)
	at net.yacy.kelondro.logging.Log$logRunner.run(Log.java:332)
Can't load log handler "net.yacy.kelondro.logging.LogalizerHandler"
java.lang.ClassNotFoundException: net.yacy.kelondro.logging.LogalizerHandler
java.lang.ClassNotFoundException: net.yacy.kelondro.logging.LogalizerHandler
	at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
	at java.util.logging.LogManager$3.run(LogManager.java:359)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.util.logging.LogManager.loadLoggerHandlers(LogManager.java:346)
	at java.util.logging.LogManager.initializeGlobalHandlers(LogManager.java:898)
	at java.util.logging.LogManager.access$900(LogManager.java:130)
	at java.util.logging.LogManager$RootLogger.getHandlers(LogManager.java:979)
	at java.util.logging.Logger.log(Logger.java:454)
	at java.util.logging.Logger.doLog(Logger.java:480)
	at java.util.logging.Logger.log(Logger.java:503)
	at net.yacy.kelondro.logging.Log$logRunner.run(Log.java:332)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7559 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-07 20:42:19 +00:00
orbiter
cb1f49d0f2 replaced all 'new String' with default encoding (missing) or UTF-8 encoding with a String generation method that uses a pre-defined Charset constant for UTF-8. This avoids a cache-lookup for the Charset object using String hashing of the String 'UTF-8'. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7558 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-07 20:36:40 +00:00
low012
9d366ee9d7 *) removed unused code (I assume that most of the code was really dead, but if you need any of the classes, tell me and I will put it back in.) 
*) minor code cleanup in ViewLog

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7557 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-07 18:55:11 +00:00
orbiter
7138f4036b less synchronization, better thread dump tool 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7556 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-07 15:29:45 +00:00
orbiter
8d14916c74 more patches for a better out-of-memory management 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7555 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-07 01:45:11 +00:00
orbiter
c2c5b12882 - even less memory for circle tool 
- background thread for bookmark initialization: this uses a DNS lookup which may cause long waiting times during startup

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7554 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-06 12:30:22 +00:00
orbiter
6badc5e558 reduce size of static memory usage: use short instead of int in circle coordinates cache 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7553 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-06 11:23:04 +00:00
orbiter
ce0c8247fc removed (most probably!?!) superfluos System.err output 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7552 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-06 01:01:29 +00:00
orbiter
799c534935 one more patch again OOM during secondary remote search 
see also: http://forum.yacy-websuche.de/viewtopic.php?f=6&t=3202

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7551 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-05 19:52:34 +00:00
orbiter
f8d0454c53 small bug fixes and experiments with search speed enhancement 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7549 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-04 14:29:22 +00:00
orbiter
993b9bc1a8 memory/performance hacks, less synchronization, better concurrency 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7544 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-03 11:30:04 +00:00
sixcooler
65bcc60808 stupid me: revert placement of closing connection which caused unclosed connections 
+ reuse sockets

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7543 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-03 04:16:33 +00:00
sixcooler
e3d75d6cd5 Not storing external header in an Header-Array and reduce a loop for its conversion. 
Ensure connection close if a OOM is thrown.
Ensure setting resolved host is set at the request.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7542 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-01 13:50:11 +00:00
orbiter
42d90664f3 - fixed a memory leak in the httpc.post method (no finish) 
- patched some more memory-saving relevant code
- some more minor bug fixes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7541 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-03-01 09:03:33 +00:00
orbiter
38dce547c0 better concurrency (less locking on date formatting) more logging and minor bug fixes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7540 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-28 06:28:29 +00:00
orbiter
89d337841c more logging for OOMs 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7534 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-26 09:27:15 +00:00
orbiter
b1781d7aae some more performance hacks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7533 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-26 01:24:49 +00:00
orbiter
b2f147d28e performance hack: excluded map encoding in many cases from synchronization block, especially when doing an iteration 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7532 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 21:29:55 +00:00
orbiter
5e186e0122 continuing the fight against deadlocks during time formatting: better caching. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7531 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 21:11:53 +00:00
orbiter
dec24244cf added convenience class to generate UTF StringBody objects with a default UTF8 charset. 
Reason: if this is not used in StringBody-Class initialization, a default charset name is parsed.
This is a synchronized process and all classes using default charsets synchronize at that point
Synchronization is omitted if this class is used
 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7530 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 13:26:09 +00:00
orbiter
1110d16af9 performance hack: replaced generic row.getColBytes() call with row.getPrimaryKeyBytes() where the column is 0 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7529 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 12:41:27 +00:00
orbiter
19b2a50578 - enhanced date formatter cache 
- added more instances of formatter objects to different classes to make them independent in case of lockings that may applay during synchronization of the date formatter object (date formatting is not thread-safe and must be synchronized therefore)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7528 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 12:23:00 +00:00
orbiter
48a61c39a3 speed hacks in BLOB ArrayStack: 
- more concurrency if possible
- less threads if no concurrency necessary

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7527 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 11:58:01 +00:00
orbiter
a92d80a545 performance enhancements using an alternative to a insensitive collator (a complex string compare): 
- less synchronizations
- better speed
..at most important and commonly used classes: http headers, url parsing and html parsing

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7526 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 11:23:57 +00:00
sixcooler
bcea497644 next try to fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=3193&start=0&sid=b98aa9a7466397602b436eb45f4a9d39 
tested proxy, crawl, updatedownload - please do further testing!

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7524 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 01:16:15 +00:00
orbiter
ad7fcb9d61 Enhanced Base64Order transformation: less overhead (transformation between StringBuilder and byte[]) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7523 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 00:56:31 +00:00
orbiter
f95e50ec3d more explanation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7522 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 00:06:31 +00:00
orbiter
bb36bf841a emergency commit (sorry sixcooler for not waiting) because without that automatic updating peers would not be able to do the next update. 
Please see http://forum.yacy-websuche.de/viewtopic.php?p=22059#p22059

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7521 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-25 00:02:04 +00:00
sixcooler
8ad4e10491 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=3193&start=0&sid=b98aa9a7466397602b436eb45f4a9d39 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7520 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-24 23:15:27 +00:00
orbiter
0ce17d823a - fixed bug in ordering 
- fixed ConcurrentModificationException in set join


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7519 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-24 10:32:46 +00:00
orbiter
dec4f36700 - fix for missing favicons in search widgets 
- fix for bad digest/hash computation in case of interrupts to class

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7518 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-23 23:04:05 +00:00
orbiter
804ae2275b - do not delete idx and gap files if the heap is not modified 
this change may have bugs in it which may cause damage to your existing data. please use with care.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7516 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-23 11:28:12 +00:00
orbiter
e3ef4e3021 - increased default peer ping time from 2 minutes to 1 minute 
- filtering out too old peers when reading seed lists (limit is now 240 minutes)
- added concurrent host names resolving in front of the http client because the http client uses the java built-in DNS resolve which is not multithreading-safe (i have seen deadlocks in thread dumps showing that this bug in jdk is still there)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7515 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-23 09:42:01 +00:00
orbiter
5e45ded8e2 - removed locks from WordReference 
- refactoring of HeapReader/Writer

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7514 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-23 00:32:16 +00:00
orbiter
cd19d0517e added dns resolve to HTTPClient POST using a dns cache to prevent that that not-thread-safe built-in dns cache inside apache http client is used 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7513 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-22 22:58:19 +00:00
orbiter
af87af0d4c - removed synchronization in serverSwitch which should improve speed 
- fixed wrong assert in network graph
- enhanced double check method in table class

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7511 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-22 12:56:25 +00:00
orbiter
57e6728cb7 - removed usage of /etc/alternatives/www-browser because of problems with lynx, see: 
http://forum.yacy-websuche.de/viewtopic.php?p=21959#p21959
  please look if the browser that is linked with /etc/alternatives/www-browser can be detected and insert call again if
  it can be made sure that this does not call lynx
- replaced severe warnings with just warnings in yacyClient

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7506 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-21 11:53:45 +00:00
orbiter
91eeaf2cff fix in ftp client 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7505 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-21 10:51:15 +00:00
orbiter
e717bf74ba more logging, more care about OOMs 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7503 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-21 09:51:05 +00:00
orbiter
d84b4a072e healing for some OOM problems 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7502 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-21 00:38:49 +00:00
orbiter
4aa406fb0f added log output to find bug in url parser for short hosts 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7501 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-21 00:07:00 +00:00
orbiter
82f262f685 - enhanced circle drawing speed 
- beautified 'moving dot' feature (using smaller and correctly positioned dots)
- added moving dots to DHT transfer

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7500 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-21 00:03:11 +00:00
orbiter
29dc416ac6 more animations in graphics. See network and access picture. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7498 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-19 01:56:12 +00:00
orbiter
93b9c4fbc9 added missing file for latest commit 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7497 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-18 23:25:58 +00:00
orbiter
3e380c51b6 update to browser start with linux 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7486 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-16 14:53:47 +00:00
orbiter
6083f2f171 fix for (false) oom 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7484 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-15 14:26:25 +00:00
orbiter
b35fda43ea more changes to headless mode; now non-headless mode is used when: 
- YaCy runs on Windows
- YaCy is started with the -gui option
in all other cases YaCy runs in headless mode

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7481 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-14 22:17:59 +00:00
orbiter
6c52e31993 new methods to open a browser 
- if YaCy is started with the option -gui, it is not in headless mode. Then the java 1.6 browse method is used if all other methods fail
- in linux, the path /etc/alternatives/www-browser is used if no firefox is installed

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7480 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-14 16:15:14 +00:00
orbiter
5892fff51f introduction of dht-burst modes: this can expand the number of target peers in some cases where a better heuristic is needed. The problematic cases are either when a muti-word search is made (still a hard case for our term-oriented DHT) or when a network operator wants that all robinson peers are asked. We therefore introduced two new network steering values that switch on more peers during the peer selection. Because the number of peers can now be very large, the number of maximum httpc connections was also increased. 
Please see new coments in yacy.network.freeworld.unit for details of the new DHT selection methods.
The number of maximum peers is now not fixed to a specific number but may increase with
- the partition exponent
- the number of redundant peers
- the robinson burst percentage
- the multiword burst percentage
The maximum can then be the number of senior peers (all visible peers).

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7479 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-13 17:37:28 +00:00
orbiter
4588b5a291 - fixed document number limitation for crawls that restrict the number of documents per domain 
- some restructuring of the document counting and logging structures was necessary
- better abstraction of CrawlProfiles
- added deletion of logs to the index deletion option (if the index is deleted using the servlets) which is necessary to reset the domain counters for the page limitation
- more refactoring to get the LibraryProvider more clean
- some refactoring of the Condenser class

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7478 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-12 00:01:40 +00:00
low012
64f32e8f00 *) replaced all IPs in IP filters for proxy with the proper regular expression 
*) some cleanup

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7477 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-11 23:37:13 +00:00
sixcooler
3e8b72be50 update to httpclient-4.1 - sorry forgot some 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7474 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-08 21:53:20 +00:00
low012
74b22dfa24 *) fixed bug which affected blacklist entries which consisted of domain _and_ path parts 
*) minor cleanup

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7471 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-06 18:07:08 +00:00
orbiter
fe93caac5a added flags and administration options to show advanced search and to show search result attributes (for each search result) 
Administration can be done at ConfigPortal.html

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7466 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-02 15:54:13 +00:00
orbiter
431f780f41 patch for bad data in url metadata 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7464 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-02 01:19:25 +00:00
orbiter
5905f912c5 replaced more double types with float 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7462 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-02 00:22:00 +00:00
orbiter
0cdfb82963 replaced more appearance of double values by float values 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7461 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-02 00:06:29 +00:00
orbiter
eb12e15738 moved all Double values to Float values because of 
http://www.exploringbinary.com/java-hangs-when-converting-2-2250738585072012e-308/
YaCy does not really need double-precision floating point computation anywhere, so this should not affect any feature

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7460 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-02-01 23:49:11 +00:00
f1ori
982aa689ef * fix StringIndexOutOfBoundException in WebStructureGraph 
* add better escaping to saveMap and loadMap

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7458 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-01-31 14:25:09 +00:00
orbiter
88773e4daa changed the default port from 8080 to 8090 
see also: http://forum.yacy-websuche.de/viewtopic.php?p=21683#p21683

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7454 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-01-28 10:54:13 +00:00
orbiter
991b92f4ae enhanced network graphic 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7446 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-01-26 13:52:46 +00:00
f1ori
a321c7673d * adminAccountForLocalhost only for localhost 
* yacy crawls local domains also, if no password is set (the interface is already protected)
* it's not required anymore, to set a password in intranet mode

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7436 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-01-17 11:37:30 +00:00
hermens
930cb412dd Let SHORT_MILSEC_FORMATTER make a new formatted String every millisecond 
see http://forum.yacy-websuche.de/viewtopic.php?f=6&t=3103


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7434 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-01-15 13:16:40 +00:00
orbiter
090c73e32e catch a OOM in HeapReader iteration 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7433 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-01-12 12:04:18 +00:00
low012
48463c4507 *) General private License? ;-) 
*) minor code changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7432 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-01-12 00:28:08 +00:00
orbiter
6c1b14c8e1 - more control in access tracker: count number of returned search results (not only info how much is in the index) 
- extended query params for this
- enhanced cora

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7430 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-01-11 22:58:14 +00:00
low012
9f38c0023d *) Minor changes, mainly cleaning up a little bit, no functional changes. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7428 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-01-10 20:24:52 +00:00
orbiter
54e77e6255 refactoring 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7426 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-01-10 08:40:41 +00:00
orbiter
feefe17568 npe assert fix 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7424 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-01-04 12:24:41 +00:00
orbiter
733903f2c9 fix for http://forum.yacy-websuche.de/viewtopic.php?p=21489#p21489 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7422 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-01-03 23:06:33 +00:00
orbiter
24e4126eee added JSON parser code from json.org (added generics to it) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7421 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-01-03 22:53:15 +00:00
orbiter
10ae8d961b - cora package has now no dependencies to other yacy packages and becomes a 'base' package (refactoring) 
- cleaned up (removed special code and documentation for 27c3)
- added remote search functions to be used within cora

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7420 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-01-03 20:52:54 +00:00
lotus
0e54233408 UPnP: map port again if we are not reachable (e.g. when router rebooted) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7419 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-01-02 21:17:21 +00:00
lotus
b1484299b2 same units for memory observer configuration (MiB) 
old setting for DHT (RAM) will be lost after update
can be set on /Performance_p.html

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7418 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-01-02 20:38:01 +00:00
orbiter
387db84087 maybe found bug in non-working index dumper 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7414 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-30 00:02:42 +00:00
orbiter
a4c9d27287 - moved some variables from Stwitchboard to new class AccessTracker 
- added a limitation in access tracking to delete queries which are older than 10 minutes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7410 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-29 01:54:27 +00:00
f1ori
e4aabaa1c3 * fix negative filelength for files >2G 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7408 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-28 17:25:39 +00:00
orbiter
cdfe8afe3f fix for really bad table iteration implementation: reduction of IO 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7407 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-28 16:44:55 +00:00
low012
9eae33f886 *) Ooops... 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7406 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-28 13:04:48 +00:00
low012
a001e8075c *) minor enhancements 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7405 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-28 13:03:49 +00:00
low012
11ea966f9e *) added SID file (Commodore 64) sound file parser 
*) minor changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7403 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-28 12:06:04 +00:00
orbiter
b2ed4cfaf8 more small bugfixes and light refactoring 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7401 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-28 01:57:05 +00:00
orbiter
3ca06d6290 patch for http://forum.yacy-websuche.de/viewtopic.php?p=21460#p21460 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7399 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-27 23:57:29 +00:00
orbiter
903c824c2c - allow only scanned resourced with granted status 
- increased time-out when scanning an ip range

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7398 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-27 23:11:56 +00:00
low012
936e976c23 *) added FreeMind (http://freemind.sourceforge.net/) mindmap parser 
*) minor changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7397 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-27 20:13:31 +00:00
low012
3d95981f7d *) cleaning up the code a little bit 
*) minor changes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7396 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-27 17:07:21 +00:00
low012
2a6499364d *) minor changes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7395 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-27 15:53:41 +00:00
low012
c0274bd123 *) minor changes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7394 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-27 15:37:11 +00:00
orbiter
fe46536f6e enhanced network scanner (less name resolving during scanning and no name resolving during search) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7392 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-26 16:25:17 +00:00
orbiter
e753027c43 fix for http://forum.yacy-websuche.de/viewtopic.php?p=21439#p21439 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7390 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-22 00:04:30 +00:00
orbiter
bf4ef1513e - fix for map view 
- remove some UNRESOLVED PATTERN
- maybe a fix for non-flushing cache

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7389 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-21 23:48:05 +00:00
orbiter
6b70393d1d - new java version 1.6 
- replaced old gif animator by java 1.6 gif animator

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7388 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-20 22:51:50 +00:00
orbiter
e88c428008 fix to ftp loader 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7387 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-18 10:22:54 +00:00
orbiter
59b70a5a92 another fix to the ftp crawler: now correct directory listings according to rfc2640 (path with spaces) and better title names for such files 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7386 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-18 00:44:19 +00:00
orbiter
9b25a33fd9 - fixed numerous bugs 
- better document names
- fixed problem with ftp crawling
- added automatic removal of search results from services that are not online according to the latest network scan: this does not delete the index but just does not show them. after the next network scan when the server is available again, the results are again showed.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7385 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-17 17:30:09 +00:00
orbiter
7bdb13bf7f more fixes to smb crawling: better file names 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7384 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-17 00:52:24 +00:00
orbiter
94c48500cc several fixes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7383 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-17 00:11:42 +00:00
orbiter
0ac7311a62 fix for token parser 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7382 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-16 23:47:36 +00:00
orbiter
58b59f9bc8 - a collection of bug fixes and some redesign of the Scanner class 
- fixed smb crawling
- added smbget to download script generation

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7381 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-12-16 23:37:21 +00:00