Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
8,160 Commits 1 Branch 20 Tags 836 MiB
0d148c3353
Commit Graph

4726 Commits

Author SHA1 Message Date
Michael Peter Christen
0d148c3353 more logging in resource observer 2012-02-23 01:20:42 +01:00
Michael Peter Christen
2fa037ae1d enhanced crawler 2012-02-23 01:20:24 +01:00
low012
2120db289a *) Small change which should solve problem with cgitb module in Python CGI scripts. 2012-02-14 20:54:19 +01:00
Lotus
ee89cf5ae5 fix must match filter for full domain crawl 
allow:
http://www.example.com
http://www.example.com/
http://www.example.com/abc.html?xyz=q
block:
http://www.example.com.cn
http://www.example.com.cn/dsf
 2012-02-07 16:13:13 +01:00
Michael Peter Christen
9ad1d8dde2 complete redesign of crawl queue monitoring: do not look at a 
ready-prepared crawl list but at the stacks of the domains that are
stored for balanced crawling. This affects also the balancer since that
does not need to prepare the pre-selected crawl list for monitoring. As
a effect:
- it is no more possible to see the correct order of next to-be-crawled
links, since that depends on the actual state of the balancer stack the
next time another url is requested for loading
- the balancer works better since the next url can be selected according
to the current situation and not according to a pre-selected order.
 2012-02-02 21:33:42 +01:00
Michael Peter Christen
4540174fe0 memory hacks 2012-02-02 07:37:00 +01:00
Michael Peter Christen
9ebcae2fbc enhanced url parser to understand urls with & instead of & in post 
urls
 2012-01-25 02:42:24 +01:00
Michael Peter Christen
1f4f60654a Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 
Conflicts:
	source/net/yacy/document/parser/pdfParser.java
 2012-01-24 20:42:30 +01:00
Michael Peter Christen
e6d26a023f fix for bookmark crash with possible side-effects on crawl start after 
the crash
 2012-01-19 23:06:09 +01:00
Michael Peter Christen
190b77c55e added Ukrainian translation 2012-01-17 17:45:28 +01:00
Marek Otahal
72adbeae90 !Important: move from Hashtable to HashMap 
Hashtable is an obsolete collection v1, now since v2 offers HashMap with same or better
functionality. Please review, almost all code was already moved, so only a few changes. That is not the issue,
but I found notices that some (ugly big) helper classes had to be created in past
to compensate missing Hashtable's functionality. I'd like input if we can remove some of them.
look for //FIX: if these commits

Signed-off-by: Marek Otahal <markotahal@gmail.com>
 2012-01-09 01:29:18 +01:00
Marek Otahal
c1af123ddd just a little faster toString 
Signed-off-by: Marek Otahal <markotahal@gmail.com>
 2012-01-09 01:26:39 +01:00
Marek Otahal
64e4bcee82 serverSwitch get(App/Data)Path() use common helper method 
Signed-off-by: Marek Otahal <markotahal@gmail.com>
 2012-01-09 01:26:03 +01:00
Marek Otahal
371fbb4deb just comment + shorter code in serverSwitch 
Signed-off-by: Marek Otahal <markotahal@gmail.com>
 2012-01-09 01:24:51 +01:00
Marek Otahal
ed253b7aff update javadoc, does not throw IOException 
Signed-off-by: Marek Otahal <markotahal@gmail.com>
 2012-01-09 01:19:24 +01:00
Michael Peter Christen
2ee8cbeb2c Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 
Conflicts:
	source/net/yacy/search/Switchboard.java
 2012-01-05 18:37:46 +01:00
Michael Peter Christen
992dbdf4bb added noload statistic to servlets 2012-01-05 18:33:05 +01:00
Michael Christen
354b976110 fix for concurrency problem and endless loop in /suggest.json 2012-01-05 08:35:44 +01:00
Michael Christen
c21966bb43 fix 2012-01-04 23:02:12 +01:00
Michael Christen
13b05f9c08 fix 2012-01-04 23:01:04 +01:00
Michael Christen
e5d878c59e Merge branch 'master' of ssh://gitorious.org/yacy/rc1 
Conflicts:
	source/de/anomic/crawler/CrawlQueues.java
 2012-01-04 22:08:17 +01:00
Michael Christen
ec26b2bea4 Merge commit 'fa08ed5ae5d72bddc3cc6a662b23103579e86109' into quix0r 
Conflicts:
	source/de/anomic/crawler/CrawlQueues.java
 2012-01-04 20:32:42 +01:00
Michael Christen
216a287a85 Merge commit '6d4e08ed06c5cd28c45981b2ebe31c7f7ec6fd83' into quix0r 
Conflicts:
	source/de/anomic/crawler/CrawlQueues.java
 2012-01-04 20:16:37 +01:00
stbrumm
d18095dc48 Patch fuer Issue 0000102 
and fixes to Patch (private peer status is a property of a peer, not a
status)
 2012-01-03 17:49:37 +01:00
Roland 'Quix0r' Haeder
901f37d608 Also this ... :( #2 2011-12-29 00:36:56 +01:00
Roland 'Quix0r' Haeder
a985717ed2 Also this ... :( 2011-12-29 00:35:51 +01:00
Roland 'Quix0r' Haeder
5f490de554 Fix for ported fix from my old days ... 2011-12-29 00:34:46 +01:00
Roland 'Quix0r' Haeder
fa08ed5ae5 Fixed a lot CHMOD rights (no need for execute flag on *.java/*.html) and introduced local/remote crawl size ratio based check 2011-12-29 00:33:16 +01:00
Michael Christen
9e5894c784 Removed handling of components objects for URIMetadataRows. 
This is a preparation to replace this rows with nodes from the node
store.
 2011-12-17 01:27:08 +01:00
Michael Christen
c04bfaa51b refactoring 2011-12-16 23:59:29 +01:00
Michael Christen
17f962fceb translator updates: 
- config string for chinese
- do not copy the language file to DATA/LOCALE any more (and do not use
them there, this is really confusing for new translators)
 2011-12-08 10:25:26 +01:00
Michael Christen
752b092b8a Merge branch 'master' of ssh://git@gitorious.org/yacy/rc1.git 2011-12-07 01:10:37 +01:00
admin
23afee58fe Merge branch 'master' of git://github.com/f1ori/yacy 2011-12-07 01:01:04 +01:00
Michael Christen
3eccdca63c protection against too long running snippet fetch processes 2011-12-07 00:32:42 +01:00
apfelmaennchen
ff19fcdb28 bugfix for YMarks XBEL import and export; thanks to Dominic 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8138 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-12-06 23:17:57 +00:00
Michael Christen
044f83feed added some pauses into the search process which shall produce 
better-ranked search results. without that pauses the result page will
only contain links from the peer that answers first which is not a good
average picture of all the peers that provided results
 2011-12-06 15:28:48 +01:00
Michael Christen
6e66c9d7f1 fix for http://bugs.yacy.net/view.php?id=87 2011-12-05 23:46:42 +01:00
Michael Christen
e7e429705a - less automatic indexing after a search (needs to reset the default 
crawl profiles)
- fix for concurrency problem in storage of serverSwitch Properties
- markup update
 2011-12-05 16:22:11 +01:00
admin
a4ac051029 Merge branch 'master' of git://github.com/f1ori/yacy 2011-12-04 15:01:03 +01:00
low012
7cfdc2c092 Improved CGI capabilities: 
*) CGI respects shebang now (should solve problems with MS Windows)
*) better error handling (more correct HTTP error codes)
*) logging

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8136 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-12-04 13:43:35 +00:00
Michael Christen
9cd469e6d6 added pull request from als plus an NPE fix 2011-12-04 12:15:03 +01:00
orbiter
11729061f2 added an option in the bookmark import process to put everything into the crawler 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8134 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-12-03 00:27:01 +00:00
apfelmaennchen
70bcfc150a - small bug fix to ymarks html importer 
- import of delicious.com exports has successfully been tested

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8132 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-12-02 23:24:40 +00:00
apfelmaennchen
b5d9f631e3 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8128 6c8d7289-2bf4-0310-a012-ef5d649a1542 2011-12-01 22:25:23 +00:00
orbiter
35a9e8f307 - fixed network graphic 
- debuged evaluation tables
- changed cache settings in template engine
- some speed hacks
- changed int angles for peer positions in network graphic to double angles

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8124 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-30 20:50:41 +00:00
Al Sutton
8993cac4d8 Initial performance improvements 2011-11-30 11:15:54 +00:00
orbiter
8895d8c1cd removed unnecessary log entries 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8117 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-27 16:54:48 +00:00
apfelmaennchen
77a080ced9 smaller fixes for YMarks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8105 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-25 17:33:03 +00:00
orbiter
5a55397f99 some last-minute performance hacks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8101 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-25 11:23:52 +00:00
apfelmaennchen
dd1482aaf5 further update to YMarks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8100 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-25 00:32:18 +00:00