Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
1,635 Commits 1 Branch 20 Tags 836 MiB
60dac4325e
Commit Graph

1635 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
orbiter
60dac4325e serialized indexing with dht selection 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1719 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-20 23:57:50 +00:00
orbiter
a840755964 moved parts of index transfer logic back to switchboard 
this is needed to merge the dht selection with the indexing thread

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1718 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-20 23:27:11 +00:00
orbiter
134253a603 fixed bug with cache flush 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1717 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-20 23:25:48 +00:00
borg-0300
149409ba5c move description -> javadoc 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1716 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-20 23:15:39 +00:00
borg-0300
00975359c5 added keywords, properties 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1715 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-20 22:57:42 +00:00
allo
3fd1641893 queuesizes in queues_p.xml 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1714 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-20 22:48:39 +00:00
orbiter
c2d863855d different flush limit 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1713 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-20 22:21:53 +00:00
borg-0300
17334f7bdd 'via' removed 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1712 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-20 20:59:05 +00:00
borg-0300
64441b1f78 ADDED: yacy.badwords list to filter the topwords 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1711 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-20 17:50:42 +00:00
allo
d7da273d7e using ArrayList instead of Vector 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1710 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-20 07:42:41 +00:00
allo
d6f7bd44ae fix for public tags 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1709 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-20 06:34:44 +00:00
orbiter
f9063e2040 added some synchronization to avoid that several tasks can trigger a cache flush simultanously 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1708 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-20 00:06:03 +00:00
orbiter
2c4e4ae6a2 further refactoring of dht selection, transfer and flushing 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1707 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-19 23:47:45 +00:00
orbiter
73dad68cf1 outsourced thelis DHT flush class into own file 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1706 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-19 21:54:46 +00:00
allo
aa4b04e3dd reverted last change 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1705 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-19 19:05:46 +00:00
allo
467216d4c7 bugfix for umlaute and html-specialchars in wikiCode 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1704 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-19 18:54:38 +00:00
allo
4b0dae8fcf added a possiblity to get the ranking values for an url. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1703 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-19 11:08:06 +00:00
orbiter
85ac7d8386 * moved DHT transfer thread to own class file, needed for further modularization 
* changed status handling
* added forced cache flush when cache has containers with too high number of index entries


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1702 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-19 01:36:42 +00:00
orbiter
34341a868e code cleanup 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1701 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-19 00:39:16 +00:00
orbiter
7df2e6e571 bugfix for last commit 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1700 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-19 00:35:23 +00:00
allo
6b20ccaa0f count to the next update 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1699 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-18 20:05:55 +00:00
orbiter
cd41e9a0eb moved DHT index selection to new object that holds indexes to be send away to other peer. 
This was made to make it possible that RWI selections can be serialized with indexing.
Serialization will be implemented in another step.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1698 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-18 19:34:16 +00:00
allo
e3dd67bba0 bookmarks import. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1697 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-18 17:49:24 +00:00
allo
dda143df79 We NEED this here for the XML-Files 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1696 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-18 14:57:25 +00:00
theli
42a5f56723 *) Bugfix for broken dht thread configuration 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1695 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-18 14:51:01 +00:00
allo
91b78d9f04 missing File for IndexCreate 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1694 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-18 12:01:52 +00:00
allo
bfd37e34aa using other XML Parser 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1693 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-18 12:00:13 +00:00
theli
aa87df35e1 *) To avoid confusion location will now also be displayed for own peer 
See: http://www.yacy-forum.de/viewtopic.php?p=17283#17283

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1692 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-18 10:27:56 +00:00
rramthun
851493624f MIME-Type setzen 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1691 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-17 21:38:46 +00:00
rramthun
15ed57f9b7 Updated German language, by VT100, NN, rramthun 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1690 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-17 21:23:45 +00:00
allo
62664d7252 AJAX Check for robots.txt before crawling. 
Icons from herrlich
TODO: Style it nicely ;-)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1689 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-17 20:55:31 +00:00
allo
0b5a736280 bugfix for nullpointer on unparsable date. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1688 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-17 19:40:01 +00:00
borg-0300
cf0993f516 added newline after excluded stopwords 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1687 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-17 18:46:04 +00:00
allo
f503df04f8 sort the Bookmarks with newest First. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1686 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-17 18:14:11 +00:00
allo
3b4a99ff6a fix for java 1.4.x 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1685 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-17 17:55:13 +00:00
low012
1d14958417 *) added link to profile of peer that sent a message (requested in http://www.yacy-forum.de/viewtopic.php?t=1957) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1684 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-17 17:41:21 +00:00
low012
8cf55537d7 *) changed name of COPYING and removed email address as suggested in the forum 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1683 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-17 17:08:59 +00:00
low012
d884796c7f *) new bookmark icons and file COPYING added 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1682 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-17 16:48:44 +00:00
theli
3dbaca03c7 *) correcting classpath for target compileHtdocsWWW 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1681 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-17 11:05:20 +00:00
theli
651bce8e2f *) adding missing function to transmit url chunks for crawl-order jobs 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1680 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-17 11:04:35 +00:00
theli
f95d98142f *) displaying amount of items in the existsIndex caches 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1679 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-17 11:02:01 +00:00
theli
9b941fb773 *) bugfix for usage of yacy with extended port binding (e.g. #eth0:8080, 192.168.0.1:8080, etc.) 
- port was reported incorrectly to other peers


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1678 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-17 10:53:20 +00:00
allo
26d7e8dd0d more escapes 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1677 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-16 22:19:04 +00:00
allo
2d4e1325cf UTF-8 fix 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1676 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-16 21:33:41 +00:00
allo
a4b3981f33 all queues on WatchCrawler.html 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1675 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-16 21:08:34 +00:00
allo
127396436f more queues in the xml backend 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1674 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-16 20:26:10 +00:00
hydrox
e2af2a3f45 *) it's now possible to run more then one indexDistribution-Thread 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1673 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-16 15:22:25 +00:00
rramthun
4fe572164c My punishment for criticizing theli ;-) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1672 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-16 14:28:39 +00:00
theli
40dd6ec4fd *) experimental restructuring of db import function 
- trying to reduce IO load by avoiding  unnecessary db access
   - trying to presort url list

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1671 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-16 13:07:01 +00:00
theli
0b9f2d785d *) showing RemoteCrawlJobQueue and RemoteCrawlTriggerQueue in status table 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1670 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2006-02-16 11:56:42 +00:00