39 Commits

Author	SHA1	Message	Date
Michael Christen	3eccdca63c	protection against too long running snippet fetch processes	2011-12-07 00:32:42 +01:00
Michael Christen	86b3385847	fixed a deadlock during secondary remote search	2011-12-07 00:18:34 +01:00
Michael Christen	c715d19c09	fixes for dependency on svn	2011-12-06 22:05:22 +01:00
Michael Christen	0bc5d76bee	ups	2011-12-06 21:55:50 +01:00
Michael Christen	044f83feed	added some pauses into the search process which shall produce ... better-ranked search results. without that pauses the result page will only contain links from the peer that answers first which is not a good average picture of all the peers that provided results	2011-12-06 15:28:48 +01:00
Michael Christen	f14faf503b	better ranking because we wait a very little time during the search ... process more to get better remote sear results into the ranking priority stack	2011-12-06 02:24:51 +01:00
orbiter	f9216e388c	- faster ping to clean up old peers faster ... - clean up more news git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8125 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-30 21:21:16 +00:00
orbiter	d9c066227a	fix for npe ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8122 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-28 13:36:51 +00:00
orbiter	ebd840ebf6	- enhanced description on search front page ... - fixed language and heuristic modifier - added hint to crawl start that we can do also ftp and smb crawls - added a protocol extension to remote crawls to transport all search modifiers to remote peers git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8108 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-26 13:40:33 +00:00
orbiter	e22f8497c9	- tested the ARC methods ... - removed strict authentication (if password is empty; this was buggy and not useful; can be switched on if necessary globally and not for each interface method) - increased speed of CrawlResults page (no dns lookup any more) - increased speed of favicon display (removed dns lookup) git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8104 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-25 14:09:25 +00:00
orbiter	bc5df0eef5	updated ranking tables (fresh computation) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8103 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-25 12:37:00 +00:00
orbiter	5a55397f99	some last-minute performance hacks ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8101 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-25 11:23:52 +00:00
orbiter	c9216d5adf	fixed secondary remote search (the process that finds distributed join situations) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8098 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-24 22:45:31 +00:00
orbiter	64fd20b857	new default ranking profile ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8097 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-24 20:07:52 +00:00
orbiter	0cf9ebc3b0	speed enhancements when parsing RWI rows (makes search slightly faster) ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8096 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-24 17:37:04 +00:00
orbiter	ee8b1d4de1	fixed unresolved pattern and unwanted local/global switch when using votes on search results ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8093 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-24 16:05:09 +00:00
orbiter	c584db991f	creating a bookmark from the search results now works again .. with new YMarks ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8092 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-24 14:57:09 +00:00
orbiter	6cd27473f5	- better default values for caching and cache usage ... - set new caching and verification behavior according to use case automatically git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8087 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-24 10:22:02 +00:00
orbiter	1019c36dad	bug fixes and speed enhancements for search ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8085 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-24 01:30:12 +00:00
orbiter	507c9d478d	much better timing when search globally; less blocking; more results earlier! ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8084 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-23 23:39:34 +00:00
orbiter	8e0b2c5832	fixed cluster search ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8083 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-23 22:21:14 +00:00
orbiter	804e48888b	smaller bug fixes for search behavior; should produce less unnecessary removals and an exact number of results as shown in counter ... should also be a little bit faster git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8057 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-18 13:09:07 +00:00
orbiter	84c3fc9d97	local/global fixes in search, better abstraction ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8054 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-17 01:05:45 +00:00
orbiter	06352b8d6b	more logging ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8047 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-16 14:09:50 +00:00
orbiter	017a01714d	- enhanced logging in robots.txt parser for remote debugging ... - robots.txt is now more robust against database operations git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8043 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-16 01:03:49 +00:00
orbiter	3a15e58e28	- increased stability when opening the robots table ... - increased stability when deleting tables git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8034 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-14 15:33:35 +00:00
orbiter	78ce3b13be	typo ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8027 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-10 11:57:26 +00:00
orbiter	85d6bf4ac4	fixed urls to media content during indexing ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8021 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-09 15:40:14 +00:00
orbiter	0d858d48ec	replaced String with StringBuilder in suggestion process ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8020 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-09 14:42:55 +00:00
orbiter	3a807e10cf	- added a cache for active crawl profiles to the crawl switchboard ... - moved the domain cache for domain counter from the crawl switchboard to the crawl profiles. the crawl domain counter is now therefore relative for each crawl start, not for the whole crawler. git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8018 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-08 15:38:08 +00:00
orbiter	e58438c01c	- added a new retry connector for solr (for cases where solr responses are slow) ... - added a new exist property into the metadataRepository which includes solr entries git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8016 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-11-08 11:49:04 +00:00
orbiter	4ad9fc2bff	new snippet strategy for search hits in metadata: show beginning of text instead of hit position ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7999 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-10-13 00:34:52 +00:00
orbiter	5af9598bd1	enhanced exported row parsing during row import ... this affects the search and dht receive speed git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7994 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-10-10 09:46:38 +00:00
orbiter	a7df70221e	refactoring ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7987 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-10-04 09:06:24 +00:00
orbiter	cf4fd525ee	added directDocByURL attribute in crawl profile ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7985 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-09-30 12:38:28 +00:00
orbiter	035ebfbf3b	- performance hacks (should affect the crawl balancer and reduce CPU load during crawl stack re-fill) ... - this may have also (good) performance side effects on other parts of YaCy git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7982 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-09-30 07:57:50 +00:00
orbiter	b250e6466d	implemented crawl restrictions for IP pattern and country lists ... git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7980 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-09-29 15:17:39 +00:00
orbiter	2c3161b4ac	refactoring: ... RankingProcess -> RWIProcess ResultFetcher -> SnippetProcess git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7974 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-09-26 21:42:28 +00:00
orbiter	d2ea250d99	refactoring: ... - moved many classes from de.anomic to net.yacy - made more sub-packages for search classes git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7973 6c8d7289-2bf4-0310-a012-ef5d649a1542	2011-09-25 16:59:06 +00:00