theli
0db237467f
*) bugfix for URL generation from file
...
see: http://www.yacy-forum.de/viewtopic.php?p=24116
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2326 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-25 16:18:45 +00:00
orbiter
ad692fc6c7
implemented option to extract nurls from the database
...
(plus some iteration enhancements for nurls)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2325 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-24 16:40:59 +00:00
orbiter
7fd90ca7c8
* strict handling of NURL entry element generation, storage and stacking
...
* more space for EURL reason strings (you must delete the EURL db to use this)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2324 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-24 16:04:14 +00:00
orbiter
5f72be2a95
some redesign of EURL storage
...
* store() is now called explicitely
* more urls are written to the EURL table
* the EURL stack does not store the complete entry any more, now only the URL hash
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2323 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-24 15:25:47 +00:00
orbiter
1ed3e2daef
added option to extract domains and/or urls from the eurl database
...
when extracting from eurl, the html output format is recommended, since
this format adds also the fail reason to the domain/url.
The complete syntax for domain extraction is now
java -Xmx<megabytes>m -classpath classes yacy -domlist [ -source { lurl | eurl } ] [ -format { text | zip | gzip | html } ] [ <path to DATA folder> ]
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2322 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-24 08:08:33 +00:00
orbiter
7e0a130fb5
new indexURLEntry class 'indexURLEntryNew', to replace old class
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2321 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-23 22:43:36 +00:00
orbiter
58df8b7bbf
a large collection of different changes
...
* mainly for the transition to the new indexing database structure
* a bugfix for an endless loop inside kelondroTree iteration
* a bugfix for bulk read inside a kelondroTree iteration; the bug caused that some elements had been iterated twice
* very strong speed enhancement for url/domain extraction
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2320 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-23 22:39:41 +00:00
orbiter
493b1cd2bf
better logging for domain extraction
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2319 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-22 11:43:56 +00:00
orbiter
e20ff77c10
another bugfix in new url class
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2318 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-22 11:37:22 +00:00
orbiter
685430a1b5
bugfix in new URL class, better loggin for domain extraction
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2317 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-22 11:33:01 +00:00
orbiter
c57b78722b
added some more logging to domain extraction
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2316 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-22 10:56:40 +00:00
orbiter
79af283f6c
better debugging in new URL class for wrong port numbers
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2315 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-22 10:21:24 +00:00
orbiter
cc2be7fb43
fix for genurllist in case of bad urls
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2314 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-22 10:00:21 +00:00
allo
1b2ea58ee9
wrong substring invocation.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2313 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-20 13:49:38 +00:00
orbiter
e4f1820b58
protection against too long authentication strings in switchboard
...
see also: http://www.yacy-forum.de/viewtopic.php?p=23943#23943
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2312 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-20 11:30:10 +00:00
orbiter
b3f7e62e03
better handling of whitespace
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2311 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-19 23:53:27 +00:00
orbiter
4149939c02
better handling of whitespace for gettext quotation
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2310 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-19 23:18:06 +00:00
orbiter
97fa6788a1
added gettext support:
...
automatic replacement of string appearances in html files by
gettext quotes.
see also: http://www.yacy-forum.de/viewtopic.php?p=23901#23901
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2309 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-19 22:35:36 +00:00
theli
b3c569f706
*) renaming of function getTransferedEntitySpeed to getTransferedEntrySpeed to avoid confusion
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2308 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-19 13:52:33 +00:00
allo
01363bc7f6
missing file for the supertemplate demonstration.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2307 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-19 11:59:55 +00:00
allo
ff3f174a2d
case insentive commandline options
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2306 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-19 11:20:22 +00:00
orbiter
67edd80884
removed tabs
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2305 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-19 11:13:14 +00:00
allo
67c486a023
some example Code, how supertemplates can be used.
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2304 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-19 07:08:15 +00:00
orbiter
5214f571cd
simplified method call in balancer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2303 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-19 00:42:20 +00:00
allo
7b0e2521bb
Support for a supertemplate, which can do all thing, a normal template can do.
...
Its a layer under the servlets, this means, #[page]# will be replaced by serverletcode, the rest can be set by you.
(TODO: if we use this for layout, we need to read "TITLE" from the servlet's tp, to set it outside of the servlet.)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2302 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-18 15:51:19 +00:00
orbiter
4bd626572b
added hashCode and compareTo to new URL class
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2301 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-18 12:00:54 +00:00
orbiter
abb5264929
fix for
...
http://www.yacy-forum.de/viewtopic.php?p=23868#23868
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2300 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-18 11:52:56 +00:00
allo
ff39a7a0d1
Overlay for welcome.*
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2299 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-18 11:07:30 +00:00
theli
a70cbd959b
*) further improvements for the anomic.net.url class
...
- relpath starting with javascript: are ignored now
- bugfix for concatenation of relpath starting with # or ?
in this case no slash should be added to the baseURL, otherwise
we get URLs of the form http://test.de/index.html/?param=value
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2298 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-18 05:12:08 +00:00
theli
8a1f1d96b3
*) Bugfix for url concatenation. Relative urls with / or http:// at the beginning
...
were not handled correctly on url concatenation via new URL(URL,relPath).
See: http://www.yacy-forum.de/viewtopic.php?t=2623
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2297 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-18 04:48:18 +00:00
rramthun
b5ec7de936
Correction to last commit + spelling
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2296 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-17 16:23:47 +00:00
rramthun
ca33eaa442
- Some spelling
...
- Removed unused init value
- Set default upload value to "none", which avoids an warning which says, upload method '' would be unknown, on new installations
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2295 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-17 16:16:41 +00:00
allo
8795875800
dirlisting for all empty directories.
...
no problem to update dir.java anymore, because its only in htroot/htdocsdefault needed.
migration to delete old dir.* files in the fileshare
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2294 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-17 15:49:42 +00:00
low012
45cefedd35
*) removed outdated seedlist
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2293 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-14 21:49:50 +00:00
allo
8e08125657
security fix (known path on yacy host / known path on remote ftp server)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2292 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-13 19:15:28 +00:00
orbiter
7935f27038
enhanced synchronization in balancer
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2291 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-13 01:31:00 +00:00
orbiter
3879a0ecd0
replaced java.net.URL usage by use of new class de.anomic.net.URL
...
This shall be seen as an experiment to exclude all cases where
there could be a DNS lookup during URL comparisment.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2290 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-13 01:21:53 +00:00
orbiter
07900366ac
deactivated cache-initialization for file-indexes (files in WORDS)
...
see also: http://www.yacy-forum.de/viewtopic.php?p=23801#23801
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2289 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-12 09:45:31 +00:00
orbiter
40aa735520
fixe timing problem causing too long delay during initialization of kelondroTree objects
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2288 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-11 23:44:44 +00:00
orbiter
d2bb3f442e
fixed timing problem causing a division by zero exception
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2287 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-11 23:43:25 +00:00
allo
379acb4915
-show stats command
...
-statusbar for irssi (use /statusbar window add yacy(Links, Words, PPM) to insert the items)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2286 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-11 17:35:20 +00:00
allo
6acb6a4d8f
tiny performance optimization
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2285 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-09 15:37:45 +00:00
allo
933a9e02ab
fix for broken build
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2284 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-09 14:35:20 +00:00
allo
360056b30c
fix ajax bug (no valid xml)
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2283 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-09 10:59:55 +00:00
allo
2bdf1fc360
totalPPM
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2282 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-08 22:41:48 +00:00
allo
f4d200ffa2
typo
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2281 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-07 20:44:45 +00:00
theli
24a02cbeef
*) Bugfix for not parsable application/xhtml+xml resources if
...
an URL has no extension
See: http://www.yacy-forum.de/viewtopic.php?p=23687
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2280 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-07 05:36:19 +00:00
orbiter
b0ca5fa784
some correction algorithm for preload time computation during assortment open
...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2279 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-05 09:20:59 +00:00
orbiter
e22cbaee97
- extended logging for preload
...
- reduced preload-time for IndexImport_p.java
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2278 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-05 09:02:58 +00:00
orbiter
671fd9a5c9
work towards new indexing database structure
...
(no effect on current functionality yet)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@2277 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-07-04 14:47:27 +00:00