Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
2fc8ecee36
yacy_search_server/source/de/anomic/crawler
History
Michael Peter Christen 2fc8ecee36 ConcurrentLinkedQueue has a VERY long return time on the .size() method. 
See
http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/ConcurrentLinkedQueue.html

and the following test programm:

public class QueueLengthTimeTest {


    public static long countTest(Queue<Integer> q, int c) {
        long t = System.currentTimeMillis();
        for (int i = 0; i < c; i++) {
            q.add(q.size());
        }
        return System.currentTimeMillis() - t;
    }

    public static void main(String[] args) {
        int c = 1;
        for (int i = 0; i < 100; i++) {
            Runtime.getRuntime().gc();
            long t1 = countTest(new ArrayBlockingQueue<Integer>(c), c);
            Runtime.getRuntime().gc();
            long t2 = countTest(new LinkedBlockingQueue<Integer>(), c);
            Runtime.getRuntime().gc();
            long t3 = countTest(new ConcurrentLinkedQueue<Integer>(),
c);

            System.out.println("count = " + c + ": ArrayBlockingQueue =
" + t1 + ", LinkedBlockingQueue = " + t2 + ", ConcurrentLinkedQueue = "
+ t3);
            c = c * 2;
        }
    }
}
2012-02-27 00:42:32 +01:00
..
retrieval Fixed a lot CHMOD rights (no need for execute flag on *.java/*.html) and introduced local/remote crawl size ratio based check 2011-12-29 00:33:16 +01:00
Balancer.java enhanced crawler 2012-02-23 01:20:24 +01:00
CrawlProfile.java fix must match filter for full domain crawl 2012-02-07 16:13:13 +01:00
CrawlQueues.java complete redesign of crawl queue monitoring: do not look at a 2012-02-02 21:33:42 +01:00
CrawlStacker.java fix for http://bugs.yacy.net/view.php?id=148 2012-02-24 00:38:57 +01:00
CrawlSwitchboard.java - less automatic indexing after a search (needs to reset the default 2011-12-05 16:22:11 +01:00
ImporterException.java
Latency.java complete redesign of crawl queue monitoring: do not look at a 2012-02-02 21:33:42 +01:00
NoticedURL.java complete redesign of crawl queue monitoring: do not look at a 2012-02-02 21:33:42 +01:00
ResourceObserver.java more logging in resource observer 2012-02-23 01:20:42 +01:00
ResultImages.java ConcurrentLinkedQueue has a VERY long return time on the .size() method. 2012-02-27 00:42:32 +01:00
ResultURLs.java Removed handling of components objects for URIMetadataRows. 2011-12-17 01:27:08 +01:00
RobotsTxt.java produce a bookmark entry from every crawl start. these bookmarks are always private. 2011-11-21 23:10:29 +00:00
RobotsTxtEntry.java - enhanced logging in robots.txt parser for remote debugging 2011-11-16 01:03:49 +00:00
RobotsTxtParser.java fix for robot parser 2011-11-16 13:12:46 +00:00
RSSLoader.java refactoring: 2011-09-25 16:59:06 +00:00
SitemapImporter.java refactoring: 2011-09-25 16:59:06 +00:00
ZURL.java ConcurrentLinkedQueue has a VERY long return time on the .size() method. 2012-02-27 00:42:32 +01:00