Commit Graph

1324 Commits

Author SHA1 Message Date
hermens
474379ae63 remove TABs from plasmaDbImporter.java
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1916 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-17 21:52:36 +00:00
orbiter
dba02f399f starting of re-design of kelondroTree iterator
- new access to iterator
- added many IOException handling in other Classes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1914 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-17 20:52:43 +00:00
orbiter
f02b426073 made kelondroTree.nodeIterator private
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1910 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-17 18:10:48 +00:00
borg-0300
5f6fdf1786 Bugfix for getCachePath(URL url)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1909 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-17 16:39:29 +00:00
orbiter
303b6463a8 added debug line to URL storage for testing
see http://www.yacy-forum.de/viewtopic.php?p=19129#19129

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1908 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-17 16:30:31 +00:00
orbiter
431a4f3609 eliminated correcting iterator in kelondroTree
VERY EXPERIMENTAL! NOT TESTED!

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1907 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-17 14:20:04 +00:00
orbiter
91dca2cd8d fixed a bug in last commit: LURL entries cannot be written,
because a stored property was not set to false (but true)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1906 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-17 13:07:34 +00:00
orbiter
3286b1f498 re-organisation of lurl-creation and -stacking
this was necessary to prevent useless write to the database
in case of blacklist appearance of the url

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1905 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-17 10:16:07 +00:00
orbiter
9cca36a107 no more strict comparator checking in map exclude method if not needed
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1901 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-16 22:46:32 +00:00
orbiter
0b903c5317 removed usage of kelondroNaturalOrder from plasmaCondenser to experimental
exclude cause of a 100% bug.
see http://www.yacy-forum.de/viewtopic.php?p=19076#19076

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1900 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-16 22:26:04 +00:00
orbiter
4239db0d1c fixed new ordering for backup iterator TreeSet
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1899 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-16 22:20:28 +00:00
borg-0300
399538b7de Bugfix: wrongly compared
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1898 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-16 20:57:06 +00:00
orbiter
33eba5ecb8 temporary disabling last change, does not work (cannot debug right now)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1896 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-16 16:50:56 +00:00
orbiter
f0464042fc fix for latest iterator-replacement-fix:
iterator generated TreeSet which did not resprect rotations
this has now be implemented using kelondroOrder Objects
and by adding this rotation-rules to the ording

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1895 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-16 16:44:29 +00:00
borg-0300
ec21c585cb try to fix path too long
see http://www.yacy-forum.de/viewtopic.php?p=19079

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1893 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-15 20:16:15 +00:00
orbiter
a6a3f4b694 fix for svn 1888
this is a redesign of the no-iterator solution

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1892 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-15 16:01:42 +00:00
hydrox
8da13088e9 *)removed multiple DHT_Distribution_Threads
*)boosted DHT_Distribution sending chunk parallel to multiple peers

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1890 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-15 11:27:43 +00:00
orbiter
283a7181c6 try to fix new 100% cpu bug, possibly caused by iterator method
see http://www.yacy-forum.de/viewtopic.php?p=18900#18900

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1888 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-14 23:22:49 +00:00
theli
34c075c1c7 testcommit with subversive
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1886 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-14 08:54:44 +00:00
orbiter
f588c0724f removed cache flush in case of DHT receive
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1885 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-13 23:23:45 +00:00
orbiter
e94b374d56 update to cache flush method
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1884 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-13 23:06:06 +00:00
orbiter
bcd99fe83e introduced a second RAM cache for DHT transfer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1880 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-13 10:43:12 +00:00
hydrox
360a460da8 *)URL-Cleaner: moved logging-statement to correct position
*)plasmaURLPattern: host is now added to the hashset in lowercase

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1879 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-12 18:08:48 +00:00
orbiter
02f9765013 quickfix for time problem during cache restore
see http://www.yacy-forum.de/viewtopic.php?p=18810#18810

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1878 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-11 21:55:32 +00:00
orbiter
69d26045eb quick-fix for http://www.yacy-forum.de/viewtopic.php?p=18805#18805
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1876 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-10 20:45:14 +00:00
allo
b3df5ce2b8 another local iprange
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1875 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-10 18:34:53 +00:00
hermens
ad119f06af *) Don't overwrite new entries with older ones
see: http://www.yacy-forum.de/viewtopic.php?t=2015



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1874 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-10 16:28:01 +00:00
orbiter
be88687d8c fixed some problems with new cache flush karenz
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1873 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-10 13:57:30 +00:00
theli
d3da7c9a08 *) Adding support for robots Allow directive
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1872 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-09 14:03:54 +00:00
theli
734d18f283 *) more correct robots.txt validation
- isDisallowed now uses getFile instead of getPath

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1870 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-09 13:04:57 +00:00
hydrox
f046e1814a *fix or last commit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1869 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-09 12:35:50 +00:00
hydrox
c55c51e2a8 *)added keywords to IndexCleaner_p.java
*)updated Logging

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1868 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-09 12:23:05 +00:00
theli
f0ad0d2b2b *) better robots.txt support
- previously rules for all crawlers and special rules for yacy where 
     combined using AND. Now the general rule will be ignored if there is
     a special rule for yacy (according to rfc)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1867 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-09 12:19:46 +00:00
orbiter
ddbeda738e added minimum age of word in cache to performance menu
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1866 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-09 11:31:17 +00:00
orbiter
f188611fc6 apply blacklist on rwis during dht receive
very experimental!

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1865 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-09 10:46:02 +00:00
theli
915812f597 *) Undoing robots parser policy changes from svn rev. 1421
- crawling is not allowed if server returned a 403 statuscode (according to rfc)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1864 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-09 10:17:37 +00:00
orbiter
0ec28b8f8e added DBCleaner from Hydrox
see http://www.yacy-forum.de/viewtopic.php?p=18093#18093
The servlet is now named IndexCleaner_p.
See http://localhost:8080/IndexCleaner_p.html
The Servlet was adopted to fit in the overall architecture

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1863 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-08 22:06:11 +00:00
theli
5ee0125046 *) adding possibility to configure the server port for seed uploading via scp.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1861 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-08 16:34:05 +00:00
allo
2065138a41 reverted last Commit.
@Orbiter: kannst du dich dann mal drum kümmern, wenn ich versuche die ganze Sache ans Laufen zu bringen, hast du jedesmal was dagegen. Dann mach du es bitte, du wirst ja wissen was du willst...


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1860 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-08 13:35:38 +00:00
allo
7afa5c1b8e staticIP fix
tried to solve http://www.yacy-forum.de/viewtopic.php?p=18663#18663
D 2006/03/08 07:08:20 YACY yacyClient.publishMySeed mySeed error - not proper: IP is not proper: -UNRESOLVED_PATTERN-


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1859 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-08 12:23:26 +00:00
theli
fb4100d47b *) undoing last commit.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1856 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-08 07:59:32 +00:00
theli
a84cc71218 *) removing getTotalRuntime
- not needed anymore

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1855 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-08 07:37:21 +00:00
low012
1965a8cf70 *) bugfix for previous commit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1853 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-08 00:38:39 +00:00
low012
1c3b415c35 *) nicer anchors (TOC)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1852 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-08 00:24:49 +00:00
low012
6000a38a05 *) changed anchors (TOC) according to thelis request in http://www.yacy-forum.de/viewtopic.php?t=1987
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1851 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-07 23:58:30 +00:00
auron_x
dce08771d1 *) Fix for wrong estimated and elapsed times when import was paused
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1850 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-07 22:51:18 +00:00
hermens
b34713324a DBImport: remove words from source index even if nothing has been added to home index
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1849 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-07 22:19:39 +00:00
orbiter
3237fe1cc7 added IOException for httpc client error
see also http://www.yacy-forum.de/viewtopic.php?p=18615#18615

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1842 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-07 19:38:10 +00:00
orbiter
520b60f15b fix for http://www.yacy-forum.de/viewtopic.php?p=18610#18610
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1841 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-07 19:28:06 +00:00
theli
c7ececbfb2 *) httpd.mime: adding jar mimetype
*) httpd.java: charset is only appended to mimetype for text mimetypes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1839 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-07 15:58:50 +00:00
orbiter
ae536c4608 fixed soap service: index.soap -> yacysearch.soap
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1827 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-05 15:28:40 +00:00
orbiter
bae3783d38 added a snippet marking
(search words are now bold in snippets)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1823 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-05 01:11:06 +00:00
orbiter
f0a38873eb * added yacysearch page with better view on search results
the old search page is obsolete and will be removed
* ConfigBasic.html is now the default page instead of index.html
  as long as no password is set

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1815 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-04 18:52:04 +00:00
orbiter
f0041d504d remove of several results from a single domain is stopped if the result set is smaller than the wanted number of results
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1811 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-04 00:10:49 +00:00
auron_x
b92a00172e *) modified Blog
- added delete-option
- fixed displaying of admin options

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1805 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-03 16:47:41 +00:00
allo
68ec572df0 bugfix for staticIP
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1803 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-03 16:24:21 +00:00
theli
89286478e7 *) removing thread pool eviction for now. Not needed at the moment
See: http://www.yacy-forum.de/viewtopic.php?p=18290#18290

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1801 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-03 09:09:36 +00:00
theli
759800f543 *) Bugfix for storeHTCache problem
- content was not indexed if storeHTCache was off
   See: http://www.yacy-forum.de/viewtopic.php?p=18269
   See: http://www.yacy-forum.de/viewtopic.php?t=1882
   See: http://www.yacy-forum.de/viewtopic.php?t=241

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1800 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-03 08:30:08 +00:00
orbiter
a4682e2810 fixed problems in basic config and added language setting
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1799 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-02 22:25:46 +00:00
orbiter
a8548c0484 * several bugfixes regarding basic configuration
* extended number of search target peers

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1794 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-01 23:24:57 +00:00
orbiter
1b9b8922d9 * fixed problems with new basic 1-2-3 configuration (now authentication required)
* fixed graphics problem
* fixed some other problems with default values
* 1-2-3 config now appears automatically on start-up if no password is set
* added new config menu
* moved profile to new config menu


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1792 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-01 22:27:20 +00:00
auron_x
8c6f38fe70 *) added Blog to YaCy (atm not reachable through interface) -> Blog.html
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1790 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-01 07:40:25 +00:00
orbiter
c51603a405 added two new kelondro classes that will handle the new index data structures (not used yet)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1789 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-03-01 00:25:02 +00:00
orbiter
ce5274c194 yacybot user agent
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1786 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-28 19:08:58 +00:00
allo
3b7e66ab48 staticIP should now work
(with resolved Conflict)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1785 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-28 12:31:57 +00:00
theli
8558000bd9 *) better handling of port forwarding
- button is disabled if libx is not installed
   - bugfix for disconnect

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1782 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-28 09:50:56 +00:00
allo
32d4c3e4c6 static IP is proper
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1779 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-28 08:57:20 +00:00
orbiter
f96238e9be cosmetic
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1778 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-27 22:25:27 +00:00
low012
d6cc0ec3d8 *) Fixed a bug: Images in tables were not displayed if alt or align attributes were added.
*) Align for images only accepts valid parameters (bottom, top, center, middle, left , right) now.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1775 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-27 12:33:34 +00:00
hermens
351bd0a678 *) dbImport: convert cacheSize to kb when creating plasma* objects
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1773 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-27 01:58:37 +00:00
orbiter
eaffcfefe2 * added more ranking attributes (without function; this will be added later)
* added ranking coefficient transmission to remote peer (without evaluation on server side, will be added later)
* changed ranking coefficients slightly

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1770 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-26 11:30:37 +00:00
orbiter
87e90b9d8c refinements in ram cache flush procedure and default timing
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1768 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-25 21:05:19 +00:00
orbiter
d31a4e0b4f some small enhancements with cache flushing parameters and data structures
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1767 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-25 16:10:31 +00:00
orbiter
3703f76866 - fixed re-search bug: after a search with several words, a second search could not
find the same words as before. This was caused because indexContaines stored the url references
  with a hashtable. A tree was needed to work with the index conjunction-by-numeration
- added permanent ram cache flush (again)
- removed direct flush of ram cache after a large container is added.
  this happens especially during DHT transmission and therefore this fix should
  speed up DHT transmission on server side.
- removed unused and out-dated methods

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1765 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-25 08:42:45 +00:00
theli
fbbbf5f411 *) remote trigger for proxy-crawl
- remote crawling can now be enabled for the proxy crawling profile
   See: http://www.yacy-forum.de/viewtopic.php?p=17753#17753
   

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1758 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-24 09:35:54 +00:00
low012
f31b633b2e *) Fixed yet another small bug in the wiki. (no blank between image URLs, align and alt)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1755 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-24 01:50:29 +00:00
low012
8bdbf0dfff *) Fixed a bug in code for tables and hopefully eliminated XSS vulnarability which Allo pointed out to me.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1754 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-24 01:14:38 +00:00
theli
dc9174c809 *) Implementing snippet fetching via ajax
Snippets that are not available on page load time will be fetched using ajax requests.
   see: http://www.yacy-forum.de/viewtopic.php?p=16479

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1748 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-23 13:29:07 +00:00
theli
468ca5b0e6 *) Bugfix for url.toString problem in yacyClient crawlOrder
Thanks to Stephan for the advice

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1737 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-22 07:18:27 +00:00
low012
62d0f7a334 *) added TOC for pages in wiki as requested in http://www.yacy-forum.de/viewtopic.php?t=1987
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1734 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-22 00:53:14 +00:00
orbiter
1d8ca6e082 serialized dhtChunk deletion with indexing
The dht selection, transmission and deletion is now completely serialized with indexing


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1731 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-21 23:08:07 +00:00
allo
ff476c19a5 tuning User.html
FIXED: Now admin can logout
ADDED: Nice Progressbar, representing your remaining onlinetime.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1727 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-21 15:51:09 +00:00
low012
2e8221050a *) removed redundancy, no changes in functionality
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1725 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-21 12:24:44 +00:00
theli
2336f0f013 *) allow pausing/resuming of crawlJob Threads separately
- pausing/resuming localCrawls
   - pausing/resuming remoteTriggeredCrawls
   - pausing/resuming globalCrawlTrigger
   See: http://www.yacy-forum.de/viewtopic.php?t=1591

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1723 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-21 11:18:48 +00:00
allo
af1da090bb blogRight in UserDB.
Look at the Fileshare, for how to use it.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1722 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-21 09:45:09 +00:00
borg-0300
2ee5796e76 added keywords, properties;
change '\n' -> '\r\n'

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1720 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-21 00:30:38 +00:00
orbiter
60dac4325e serialized indexing with dht selection
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1719 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-20 23:57:50 +00:00
orbiter
a840755964 moved parts of index transfer logic back to switchboard
this is needed to merge the dht selection with the indexing thread

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1718 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-20 23:27:11 +00:00
orbiter
134253a603 fixed bug with cache flush
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1717 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-20 23:25:48 +00:00
borg-0300
149409ba5c move description -> javadoc
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1716 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-20 23:15:39 +00:00
borg-0300
00975359c5 added keywords, properties
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1715 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-20 22:57:42 +00:00
orbiter
c2d863855d different flush limit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1713 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-20 22:21:53 +00:00
borg-0300
64441b1f78 ADDED: yacy.badwords list to filter the topwords
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1711 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-20 17:50:42 +00:00
allo
d7da273d7e using ArrayList instead of Vector
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1710 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-20 07:42:41 +00:00
allo
d6f7bd44ae fix for public tags
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1709 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-20 06:34:44 +00:00
orbiter
f9063e2040 added some synchronization to avoid that several tasks can trigger a cache flush simultanously
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1708 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-20 00:06:03 +00:00
orbiter
2c4e4ae6a2 further refactoring of dht selection, transfer and flushing
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1707 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-19 23:47:45 +00:00
orbiter
73dad68cf1 outsourced thelis DHT flush class into own file
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1706 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-19 21:54:46 +00:00
allo
aa4b04e3dd reverted last change
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1705 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-19 19:05:46 +00:00
allo
467216d4c7 bugfix for umlaute and html-specialchars in wikiCode
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1704 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-19 18:54:38 +00:00
allo
4b0dae8fcf added a possiblity to get the ranking values for an url.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1703 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-19 11:08:06 +00:00
orbiter
85ac7d8386 * moved DHT transfer thread to own class file, needed for further modularization
* changed status handling
* added forced cache flush when cache has containers with too high number of index entries


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1702 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-19 01:36:42 +00:00
orbiter
34341a868e code cleanup
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1701 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-19 00:39:16 +00:00
orbiter
7df2e6e571 bugfix for last commit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1700 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-19 00:35:23 +00:00
orbiter
cd41e9a0eb moved DHT index selection to new object that holds indexes to be send away to other peer.
This was made to make it possible that RWI selections can be serialized with indexing.
Serialization will be implemented in another step.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1698 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-18 19:34:16 +00:00
allo
e3dd67bba0 bookmarks import.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1697 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-18 17:49:24 +00:00
allo
dda143df79 We NEED this here for the XML-Files
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1696 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-18 14:57:25 +00:00
theli
42a5f56723 *) Bugfix for broken dht thread configuration
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1695 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-18 14:51:01 +00:00
allo
bfd37e34aa using other XML Parser
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1693 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-18 12:00:13 +00:00
theli
aa87df35e1 *) To avoid confusion location will now also be displayed for own peer
See: http://www.yacy-forum.de/viewtopic.php?p=17283#17283

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1692 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-18 10:27:56 +00:00
rramthun
15ed57f9b7 Updated German language, by VT100, NN, rramthun
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1690 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-17 21:23:45 +00:00
allo
0b5a736280 bugfix for nullpointer on unparsable date.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1688 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-17 19:40:01 +00:00
allo
f503df04f8 sort the Bookmarks with newest First.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1686 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-17 18:14:11 +00:00
allo
3b4a99ff6a fix for java 1.4.x
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1685 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-17 17:55:13 +00:00
theli
651bce8e2f *) adding missing function to transmit url chunks for crawl-order jobs
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1680 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-17 11:04:35 +00:00
theli
f95d98142f *) displaying amount of items in the existsIndex caches
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1679 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-17 11:02:01 +00:00
theli
9b941fb773 *) bugfix for usage of yacy with extended port binding (e.g. #eth0:8080, 192.168.0.1:8080, etc.)
- port was reported incorrectly to other peers


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1678 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-17 10:53:20 +00:00
allo
2d4e1325cf UTF-8 fix
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1676 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-16 21:33:41 +00:00
hydrox
e2af2a3f45 *) it's now possible to run more then one indexDistribution-Thread
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1673 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-16 15:22:25 +00:00
theli
40dd6ec4fd *) experimental restructuring of db import function
- trying to reduce IO load by avoiding  unnecessary db access
   - trying to presort url list

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1671 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-16 13:07:01 +00:00
hermens
c8f5adea4d - don't send Message Body on HEAD requests, even in the case of an error
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1669 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-16 11:45:32 +00:00
theli
a7248fbb0a *) bugfix for http/0.9 responses
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1668 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-16 11:07:17 +00:00
theli
2da18ab359 *) correcting logging output
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1667 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-16 10:56:17 +00:00
theli
a354bc2ec1 *) Bugfix for content length check
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1666 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-16 10:54:47 +00:00
theli
8ffc6e35ad *) correcting logging output
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1665 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-16 10:28:54 +00:00
hermens
e974d0cb99 Improve compliance to rfc
*) There is no status line in HTTP/0.9
*) Answers to HEAD requests should return the same headers as a GET request



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1664 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-16 10:27:21 +00:00
theli
556d242be0 *) Limited support of content-range requests
- a simple continue download request should work now

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1663 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-16 09:23:27 +00:00
theli
8fcb25f9f9 *) Setting via header according to rfc
- can be disabled via settings dialog

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1662 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-16 09:20:57 +00:00
theli
980e986b64 *) Re enabling short cycle for already removed nurl entries
See: http://www.yacy-forum.de/viewtopic.php?p=17147#17147 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1660 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-16 06:59:34 +00:00
orbiter
218cd6561c fixed problem with wrong hash length in file share
see: http://www.yacy-forum.de/viewtopic.php?p=16565#16565

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1658 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-15 22:12:53 +00:00
orbiter
56516fd8e6 added refresh parser to htmlFilterContentScraper
* getRefreshSeconds() - number of seconds until refresh
* getRefreshPath() - url path
See also: http://www.yacy-forum.de/viewtopic.php?p=16851#16851

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1657 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-15 21:55:15 +00:00
orbiter
ba5fe0b287 fixed (again) nullPointerException in kelondroTree
see: http://www.yacy-forum.de/viewtopic.php?p=17091#17091

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1656 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-15 21:33:10 +00:00
allo
e6c2f700b1 public Tagview
IMPORTANT: Fix for new(empty) Tags when adding/editing bookmarks.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1651 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-15 14:35:30 +00:00
allo
672b384fc7 refactoring setTagsTable and setBookmarksTable.
preparation for remote Bookmarks. (and Bookmark/Tag Object, which do not interfere with local bookmarks/tags)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1649 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-15 13:16:10 +00:00
theli
040624e361 *) better support for http head requests of servlets
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1648 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-15 12:51:24 +00:00
theli
62ffb5ece0 *) httpdFileHandler.java: adding real streaming support for lage files
- avoid to read the whole file into memory
   - support of chunked transfer-encoding for http/1.1 clients
   - support of gzip content-encoding suitable clients
   See: http://www.yacy-forum.de/viewtopic.php?p=17058#17058
*) MessageSend_p.html: better highlighting of peer response/status messages

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1646 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-15 12:31:52 +00:00
hermens
3b6328ad02 *) Consistent use of minCount for index transfer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1645 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-14 22:50:13 +00:00
orbiter
0017812b2e bugfix for null compare
see http://www.yacy-forum.de/viewtopic.php?p=16980#16980

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1644 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-14 22:18:45 +00:00
allo
b137c90015 BugFix for too long Usernames.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1642 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-14 20:19:40 +00:00
allo
9d6bae77ee the best PPM ever! ;-p
Now stopped indexingstop is recognized as 0 PPM.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1641 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-14 17:29:46 +00:00
hermens
0b60b9bf51 *) Remove entries from AssortmentCluster before reinserting the rest into the ramCache
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1640 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-14 13:06:32 +00:00
hydrox
8ab1d6ff4b *) fixed NullPointerException in plasmaWordIndexEntity
See: http://www.yacy-forum.de/viewtopic.php?t=1921

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1638 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-14 11:40:36 +00:00
allo
a26574c894 Migration from tagName as key to wordhash(tagName) as key for bookmarkTags.db
(just deleting the old db, rebuildTags does the rest)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1637 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-14 10:00:12 +00:00
theli
eeba8b055e *) guessing, testing and suggesting alternative hostnames on "unknown host" error
See: http://www.yacy-forum.de/viewtopic.php?t=1879

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1636 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-14 09:55:09 +00:00
orbiter
7eb10675b3 re-organization of index management
this was done to be prepared for new storage algorithms


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1635 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-14 00:12:07 +00:00
allo
a8a5ad959a using TAG_NAME for getFriendlyName() and the lowercaseversion for getTagName()
Removed TAG_FRIENDLY_NAME

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1634 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-13 21:27:50 +00:00
allo
73f18ed5b2 git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1627 6c8d7289-2bf4-0310-a012-ef5d649a1542 2006-02-13 14:42:19 +00:00
allo
bc6d400b03 bookmarktags with wordhashs.
to Come: Migration
please test this...1) backup bookmarks.db 2) delete bookmarksTags 3) test it...


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1626 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-13 08:38:32 +00:00
allo
addc802cf3 workaround for kelondro Umlaut-Problem.
his will only fix öäüÖÄÜß ...just do not use other special chars ;-)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1621 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-12 15:18:46 +00:00
allo
fb5c387368 XML import works.
Please backup your bookmarks.db before trying it, i had many kelondro errors while importing del.icio.us xml-Files

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1620 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-12 14:51:04 +00:00
allo
a4513523d6 hide add/edit/import bookmarks per default.
xml-bookmark import (this does not work, yet)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1619 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-12 14:03:56 +00:00
orbiter
1e4578aab6 VERY EXPERIMENTAL removal of index ram cache flushing thread.
The cache will fill up and flushed explicitely when it is full.
This shall remove double-access of assortments (indexing and flush)
during indexing process. Hopefully this should reduce IO.
The main idea is: the cache shall mainly be flushed by DHT transfer, and
only indexes that shall be hosted by the own peer are flushed to the
assortments. This needs further work.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1617 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-11 23:19:01 +00:00
theli
44996afd79 *) Bugfix for handling of http/0.9 clients.
- nothing was send as response

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1610 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-11 15:21:04 +00:00
orbiter
01c668fe1b experimental disabling of db write buffer
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1605 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-11 11:31:44 +00:00
hermens
954f02d22e *) Bugfix: Prevent wordIndex.getContainer() from returning and even manipulating
the containers from the ram cache. Return a new container instead.
*) Speedup flushFromMem by reducing the number of searches in the TreeMap



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1604 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-11 03:15:14 +00:00
orbiter
4438a96b87 tried to fix "Object type or Object type combination not supported" bug
see http://www.yacy-forum.de/viewtopic.php?p=16588#16588

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1592 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-09 22:32:18 +00:00
orbiter
c70befbe9c added xml read/write example file: kelondroXMLTables
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1591 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-09 22:16:46 +00:00
allo
27b6b3d714 public Tags.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1589 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-09 20:08:45 +00:00
hydrox
a627162f13 *)fixed logginglevel for Debugmsg
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1585 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-09 09:10:49 +00:00
orbiter
fe39493145 changed default ranking parameters
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1582 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-09 00:23:54 +00:00
allo
5ef57211a6 trim was just too easy...
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1579 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-08 18:48:38 +00:00
allo
38e79aec94 Fix for wrong(?) httpheader:
" pragma: no-cache" (IIS specific?)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1578 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-08 18:06:26 +00:00
allo
a8eff9a0ae xml/bookmarks/posts/all.xml to list all public Bookmarks
bookmarksIterator now accepts an option, if you want all(with private) or only public bookmarks.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1577 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-08 17:13:27 +00:00
allo
d04e7d724e fix for deleting tags.
string2vector should return an empty vector on ""

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1576 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-08 16:37:05 +00:00
orbiter
365a3fff8e fixings for ranking attributes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1569 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-07 23:16:46 +00:00
orbiter
785763d070 added debug output for "Object type or Object type combination not supported" bug.
See also: http://www.yacy-forum.de/viewtopic.php?p=16306#16306

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1568 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-07 22:48:22 +00:00
theli
2a88232cee *) Bugfix for httpd security but
- authentication was only required for html files.
   See: http://www.yacy-forum.de/viewtopic.php?p=16510

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1563 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-07 06:54:23 +00:00
orbiter
8e55098b74 fixed detailed search
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1562 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-06 23:13:54 +00:00
theli
ebc5b1eafb *) adding a servlet that can be used to generate a Firefox search-plugin for yacy.logging
- You can access this servlet via YaCySearchPluginFF.html
   - The generated search plugin has the name YaCySearchPluginFF.src

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1555 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-06 14:41:59 +00:00
orbiter
0cb940a8e5 added detailed search.
ranking profiles do not work properly yet

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1551 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-05 23:38:00 +00:00
orbiter
c695928f7c adopted search page to new detailed search (to be commited later)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1550 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-05 22:51:37 +00:00
orbiter
45323e7b76 fixed null pointer exception during search
see http://www.yacy-forum.de/viewtopic.php?p=16429#16429


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1547 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-05 14:59:30 +00:00
allo
1fce0ea263 xml api for deleting bookmarks.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1546 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-05 14:54:07 +00:00
allo
6b16c13b20 showing the public/privat flag.
TODO: Some nice icons.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1545 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-05 14:03:33 +00:00
rramthun
49647583b2 language update
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1544 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-05 13:55:18 +00:00
allo
670356fa4e code cleanup
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1543 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-05 13:27:04 +00:00
orbiter
fb7411d7bb re-structuring of ranking application:
concentration of all ranking attributes in the
plasmaSearchRankingProfile

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1541 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-05 01:47:51 +00:00
hermens
5f5eee1ae9 *) replace System.out.println with log
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1540 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-05 01:17:08 +00:00
orbiter
d98418390b - introduced rankingProfile Class
- selection of ranking and timing profiles for each search


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1539 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-04 23:51:00 +00:00
orbiter
eab1805bca refactoring: plasmaSearchProfile -> plasmaSearchTimingProfile
This was made to distiguish this profile from the
(to-be-implemented) plasmaSeachOrderProfile

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1538 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-04 23:11:31 +00:00
orbiter
6eef848954 re-design of post-ranking process
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1537 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-04 23:00:31 +00:00
allo
af2127f3ad allow multiple #File: Sektions for one File in translations.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1534 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-04 18:08:01 +00:00
allo
1bb768178f simple check for unused Translations.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1533 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-04 17:49:10 +00:00
orbiter
be77fe1a88 code clean-up
@Martin: bitte schaue mal warum die Variablenzuweisung
         in plasmaCrawlNURLImporter war. So wie sie waren, waren sie überflüssig.
         Das hattest du dir bestimmt nicht so gedacht.
         Sollten es ggf. globale Variablen sein?

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1529 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-04 15:25:48 +00:00
orbiter
0bc2aaeb42 added normalization to search attributes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1528 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-04 14:13:54 +00:00
allo
7bd61ab0e5 Locales will now be in DATA/HTDOCS. So it works with readonly htroot.
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1527 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-04 10:50:22 +00:00
theli
008bcb7fb8 *) simplifying code by moving closeTransferIndexes into final block
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1522 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-03 13:14:30 +00:00
theli
50d85657b8 *) new import function for IndexImport_p.html
- can be used to import the crawling queue (noticeUrlDB + stacks)
   

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1518 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-02 16:46:58 +00:00
theli
214302284e *) undoing last commit because of problems with getUpdateTime
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1514 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-02 10:05:16 +00:00
theli
408de3beee *) avoiding to search in the treemap two times for the same key
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1513 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-02 10:03:47 +00:00
borg-0300
139ba4e0c8 Bugfix for getCachePath(URL url)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1510 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-02-01 19:07:49 +00:00
theli
442807cb29 *) Bugfix for last commit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1506 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-31 15:26:11 +00:00
theli
22fd1ca9aa *) minor changes
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1505 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-31 12:39:10 +00:00
theli
6a99304b2b *) Redesign of db import functionality
- restructuring to allow different import tasks to be controlled via one gui 
   - adding possibility to import a single assortment file
   - adding possibility to set the cache size that should be used

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1504 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-31 12:30:24 +00:00
hermens
56936139ae Fix logging in kelondroTree:
*) Use java.util.logging.* (via kelondroRecords) without de.anomic.server.logging.serverLog
*) Log "CORRECTING ITERATOR" only in debug mode
   (happens way too often to be a useful warning and is corrected anyway)
*) Log when correctedNodeIterator stumbles on incorrectly ordered data
   (now this one deserves throwing a warning at the user)



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1503 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-31 02:01:32 +00:00
orbiter
3834675084 fixed bug that caused wrong behavior of search result preparation
(second search on same topic resulted in less links)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1502 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-31 01:20:28 +00:00
hermens
31c8476b5d plasmaWordIndexCache.getContainer:
*) Also get entries from cache
*) calculate available remaining time for backend.getContainer correctly



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1501 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-31 01:04:23 +00:00
hermens
bb1664b63e *) Remove workaround from SVN 1472: It is not needed anymore
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1500 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-31 00:45:48 +00:00
orbiter
ec5d88664a tried too fix serverSwitch synchronization problems
see also: http://www.yacy-forum.de/viewtopic.php?p=16110#16110

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1499 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-30 23:07:20 +00:00
orbiter
3419b3bcdd fix for bug that caused the peer-counter problem.
See http://www.yacy-forum.de/viewtopic.php?p=16016#16016
The kelondroDyn now uses a generic fill character.
kelondroDyn-Tables containing peer/word/url-hashes must not use '_'
as fill character.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1498 6c8d7289-2bf4-0310-a012-ef5d649a1542
2006-01-30 22:16:55 +00:00