orbiter 0241d070bc added concurrency to indexing process: ... - the methods {parsing, semantic analysis (condensing), structure analysis (web structure)} in the serialized indexing path had been made concurrent. - four BlockingQueues handle concurrency and hand-over of the indexing objects, the last object in the queue is stored into a blockingQueue of maximum size 1 to serialize the process for storage (which uses IO and therefore here should not be deserialized) - a concurrency of (CPUs + 1) is default. Single-CPU users will profil from the change because large files cannot block the indexing process any more. - removed the secondary indexing thread, which is superfluous now. Concurrency is default for all users. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4609 6c8d7289-2bf4-0310-a012-ef5d649a1542		2008-03-28 11:56:28 +00:00
..
anomic	added concurrency to indexing process:	2008-03-28 11:56:28 +00:00