Commit Graph

1255 Commits

Author SHA1 Message Date
theli
ca26aab9b1 *) More debugging output for migrateWords
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1085 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-15 11:55:09 +00:00
theli
9b35ae9027 *) Correcting wrong % values on IndexTransfer_p page
See: http://www.yacy-forum.de/viewtopic.php?p=12646 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1084 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-15 09:52:50 +00:00
theli
e6bf9d90a5 *) Fixing Problems with MalformedURLs during Word Selection
- removing (lurl.toString() == null) comparison because toString() is never null
   - adding (lurl.url() == null) condition because url() is null if we have selected a word entry with
     a malformed URL

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1083 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-15 09:07:00 +00:00
theli
0610ff4fe9 *) small changes to crawlReceipt.java
- we do not know if the URL was stored in the noticeURL-DB with the old or new hash.
     therefore we now try to remove the URL from the noticeURL-DB using both hash values

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1082 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-15 08:40:49 +00:00
theli
86a9210264 *) indexing queue slots are now configurable via config file
See: http://www.yacy-forum.de/viewtopic.php?t=1480

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1081 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-15 08:25:46 +00:00
theli
3c11d7b81c *) Bugfix for minimizeUrlDB
- function didn't work correctly because of new url hash structure
   See: http://www.yacy-forum.de/viewtopic.php?p=12753#12753

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1080 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-15 07:35:04 +00:00
orbiter
9913049009 fixed outOfMemory bug caused by loops in kelondroTree during enumeration
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1079 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-15 01:20:05 +00:00
allo
f8f9d509d5 removed dead Code
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1078 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-14 12:48:14 +00:00
theli
47a2a8885d *) Display MimeType on URL Info page
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1077 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-14 12:05:57 +00:00
allo
5918d3985e removed Debug Statements
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1076 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-14 11:47:46 +00:00
theli
bbb936b9ea *) Bugfix for not human readable content of PDFs while viewing the URL Content via GUI
- This Bug also affects the snippet generation on non html/text documents
   See: http://www.yacy-forum.de/viewtopic.php?t=1472

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1075 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-14 10:25:43 +00:00
theli
445e3a620f *) Avoid rejecting of html content by the crawler when the file extension is not set properly
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1074 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-14 10:19:40 +00:00
orbiter
e9d6defce6 qquickfix for http://www.yacy-forum.de/viewtopic.php?p=12638#12638
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1073 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-14 09:48:39 +00:00
orbiter
a3fd0069f5 fixed bug in kelondroTree node iteration (rotation did not work)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1072 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-14 00:23:20 +00:00
theli
fd58d5f8e6 *) Adding possibility to specify the interface / IP-Address where YaCy should bind to.
- e.g. Port = 192.168.0.1:8080
          Port = #eth0:8080
          Port = 8080

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1071 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-13 17:03:52 +00:00
allo
889de6686c Migration in yacyVersion
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1070 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-13 15:12:48 +00:00
theli
444a5a9368 *) Bugfix for Entries with null url in GlobalQueue
See: http://www.yacy-forum.de/viewtopic.php?p=12675#12675

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1069 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-13 14:59:38 +00:00
allo
22da652d4f more options on Userpage.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1068 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-13 14:17:23 +00:00
allo
3bbb932fa2 Bugfix for nullpointerexception.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1067 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-13 09:55:14 +00:00
rramthun
a6f19d94b1 Updated language file...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1066 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-12 18:36:20 +00:00
allo
4976f97956 Logout Function on User.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1065 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-12 15:29:18 +00:00
allo
b74e990c74 better Logout function
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1064 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-12 12:09:45 +00:00
borg-0300
ebac51df52 restore defaultRemoteProfile
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1063 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-12 11:38:35 +00:00
allo
4daa4b00e9 logout function
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1062 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-11 22:41:36 +00:00
borg-0300
d51c0b1f48 changed properties
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1061 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-11 13:58:56 +00:00
borg-0300
5778428455 move cutUrlText to nxTools,
max length from URLs(title) on searchpage now 120 chars


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1060 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-11 13:40:53 +00:00
borg-0300
9158845c3b bugfix for snippet text null bytes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1059 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-11 13:27:36 +00:00
orbiter
6e81f2580d try to fix bug with storage of settings
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1058 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-11 08:41:13 +00:00
orbiter
f763923e0a added missing files for last commit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1057 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-11 08:02:46 +00:00
theli
9649d08171 *) More tolerant robots parser
- converting tabs to spaces
   - cutting of '*' in the disallow section

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1056 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-11 07:49:54 +00:00
orbiter
79818a320f introduced citation-rank transmission protocol and activate transport for anonymisation
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1055 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-10 23:48:20 +00:00
borg-0300
9a441e8e77 new Ranking Images
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1054 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-09 13:24:41 +00:00
theli
6f8d7d3bcd *) Adding first version of YaCy bookmarklet
- this can be used to easily crawl a webpage which is currently opened in the browser
   - to get the bookmarklet javascript simply call http://localhost:8000/QuickCrawlLink_p.html
     and drag and drop the link shown to your Browsers Toolbar/Link-Bar.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1053 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-08 12:14:51 +00:00
theli
7e0647f692 *) Bugfix for userDB usage during authentication
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1052 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-08 10:17:12 +00:00
hydrox
86c74d209d *) fix for Settings_p.html (wrong variablename in link)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1051 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-08 09:02:26 +00:00
hydrox
886955e38c *) fix for last commit (wrong filename)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1050 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-08 08:58:18 +00:00
hydrox
88669ce008 *) cleaned up Settings_p.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1049 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-08 08:40:18 +00:00
theli
93cadb47b9 *) More tolerant robots parser for robots-files which missing empty lines between rule blocks
See: http://www.yacy-forum.de/viewtopic.php?p=12471

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1048 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-08 07:41:25 +00:00
orbiter
02f8013013 auto-delete of corrupted word files during word-migration
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1047 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-07 14:57:37 +00:00
orbiter
d2731418bf added creation of global ranking files and changed url normal form usage
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1046 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-07 12:33:02 +00:00
theli
6f9f8ed8f8 *) Automatic Reset of Stack Crawler DB on startup errors
See: http://www.yacy-forum.de/viewtopic.php?t=1432

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1045 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-07 12:19:05 +00:00
theli
fb766413d1 *) Changes on httpc dns caching
- Bugfix: old dns cache did not handle case insensitive hostnames correctly. 
   - adding a possibility to set domain name patterns defining hostnames that should not be cached by the httpc dns cache
     e.g. borg-300.dyndns.org
     This can be done by setting the new httpc.nameCacheNoCachingPatterns property
   - using httpc.dnsResolve wherever possible within the sourcecode
     [httpd.java,plasmaCrawlStacker.java]

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1044 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-07 10:57:54 +00:00
allo
89a4cca4df max. num of Entries
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1043 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-07 06:53:44 +00:00
allo
8d8d866494 Bugfix for catch up late Peer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1042 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-07 06:52:30 +00:00
orbiter
bc420c62f6 fixed htcache path generation (never change a running system)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1041 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-07 01:31:11 +00:00
borg-0300
795f488222 new urlNormalform version
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1040 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-06 22:35:56 +00:00
orbiter
c86d801b0f removed dyndns domains from dns caching
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1039 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-06 22:12:08 +00:00
orbiter
6dc42a2392 detecting of loops in kelondroTree during last/first-Node search
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1038 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-06 21:06:55 +00:00
borg-0300
17d2830394 see: http://www.yacy-forum.de/viewtopic.php?t=1416
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1037 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-06 17:24:57 +00:00
theli
dd24f0252f *) Searchword highlighting for info page
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1036 6c8d7289-2bf4-0310-a012-ef5d649a1542
2005-11-06 06:27:17 +00:00