Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
996 Commits 1 Branch 20 Tags 836 MiB
00ab4d8723
Commit Graph

662 Commits

Author SHA1 Message Date
borg-0300
00ab4d8723 cleaned, small change, Properties 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1026 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-11-04 13:41:51 +00:00
theli
b8ceb1ffde *) Adding better https support for crawler 
- solving problems with unkown certificates by implementing a dummy trust Manager
   - adding https support to robots-parser 
   - Seed File can now be downloaded from https resources
   - adapting plasmaHTCache.java to support https URLs properly

*) URL Normalization
   - sub URLs are now normalized properly during indexing
   - pointing urlNormalForm function of plasmaParser to htmlFilterContentScraper function
   - normalizing URLs which were received by a crawlOrder request

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1024 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-11-03 15:28:37 +00:00
borg-0300
d2507c6081 rename setJunior()... to orJunior()..., 
added javadoc, 
added getPeerType(), setIP(), setPort(String port)


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1023 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-11-03 14:16:16 +00:00
borg-0300
e3179a6394 added getOwnSeedFile() 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1022 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-11-03 14:07:58 +00:00
borg-0300
a803a509ae bugfix: port handling in HTCache 
grogram flow, cleared up


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1021 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-11-03 12:39:24 +00:00
theli
3d0dfd4df4 *) Using StringBuffer instead of String concatenation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1018 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-11-03 08:08:37 +00:00
low012
452db479cd *) bugfix: "21" was displayed as "21" in yacyWiki 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1017 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-11-02 23:28:54 +00:00
hydrox
cb69047b91 *)cleanup access static methods and fields 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1016 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-11-02 17:56:26 +00:00
hydrox
56b9f34411 *)removed unused imports 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1015 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-11-02 16:30:45 +00:00
orbiter
5f68b6886b introduced new url-hashes for better ranking computation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1013 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-11-02 00:54:55 +00:00
orbiter
aadace1285 fixed network image in search performance monitor 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1012 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-11-01 00:49:13 +00:00
orbiter
bb369c98de fixed search result ordering by date 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1011 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-31 17:17:48 +00:00
hydrox
295aff52a3 *)added offline-browsing-support (onlineMode=0) 
*)online-mode now can be changed in Status.html

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1010 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-31 12:25:40 +00:00
orbiter
4d1e56e4d9 fixed intermission-bug (removed 'break for intermission' of httpd-thread) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1009 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-31 10:46:13 +00:00
orbiter
b058ecf0bc refactoring of image-generation; added experimental PNG encoder (not active now) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1008 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-31 02:43:55 +00:00
orbiter
7a9fc653e9 fixed 100% cpu bug in wikiCode 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1007 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-30 14:27:09 +00:00
orbiter
1aa707d357 fix for http://www.yacy-forum.de/viewtopic.php?p=11866#11866 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1005 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-29 15:30:06 +00:00
orbiter
d42531e1b2 added auto-reset for NURL-DBs 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1004 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-29 13:53:32 +00:00
allo
5526d2ff73 Basic WatchCrawler.html, which loades the Queues using AJAX. 
ajax.js: generic Ajax-Functions
xml/queues/indexing_p*: the Backend


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1002 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-29 09:47:30 +00:00
allo
92c49b406b adminAuth with userDB and adminAuthenticated (fix for statuspage) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1001 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-28 18:06:36 +00:00
rramthun
27f180f24b Update of YaWoStat to 0.2. 
Now does not try to make 400000! operations to load a 4MB textfile :-/

Program is not finished yet.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@1000 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-28 16:27:49 +00:00
theli
3b5d0eb053 *) Synchronizing robots.txt downloads to avoid parallel downloads of the same file by separate threads 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@998 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-28 09:10:48 +00:00
theli
723e056c48 *) Bugfix for ClassCastException during SessionPool.close 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@996 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-28 07:25:22 +00:00
theli
e58e85363d *) Bugfix for ConcurrentModificationException while operating on seed properties 
*) Bugfix for YACY database inconsistency (no more elements available in db '...seed.new.db'), re-set of db.
   See: http://www.yacy-forum.de/viewtopic.php?p=11836#11836
        http://www.yacy-forum.de/viewtopic.php?p=11814#11814

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@995 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-28 07:15:00 +00:00
orbiter
d656e2b433 added a memory-profile chart generation to database performance testing 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@993 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-28 01:22:11 +00:00
theli
ec3af327f7 *) Bugfix for Proxy-Authentication against remote proxy 
See: http://www.yacy-forum.de/viewtopic.php?p=11804#11804

*) Adding first version of db test for mysql
   NOTES:
   - db user + db + db table must be created before starting the test
   - db table must be empty. Entries can not be updated at the moment
   - db connection properties must be changed in the sourcecode at the moment
   TODOs:
   - accepting connection properties via command line
   - implementing update + remove + read operations
   - 'maybe' adding code to create db + table if it doesn't exists

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@991 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-27 11:28:37 +00:00
orbiter
5b0911d7ea added new performance menu for search sequence configuration and monitoring 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@990 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-27 00:42:08 +00:00
orbiter
4fa942511b de-serialized read and write access 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@989 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-26 23:05:20 +00:00
orbiter
1ff0ced515 integration of an interface class for abstract access of kelondro indexed structures like kelondroTree and kelondroHashtable 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@987 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-26 12:37:22 +00:00
allo
ada06b0674 bugfix for Networkimage from Hydrox 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@986 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-26 10:08:37 +00:00
allo
e3f87f9dab Bugfix for not parsable Numbers (NumberFormatException when loading users) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@985 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-25 12:30:07 +00:00
theli
d3ad712418 *) Bugfix for Seed file upload problem via ftpc 
See: http://www.yacy-forum.de/viewtopic.php?p=11662#11662

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@984 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-25 10:45:14 +00:00
theli
9a2afe88d4 *) Deactivating unlimited timeout for persistent connections because this 
could cause problems with clients which do not shutdown persistent 
   connections properly.
   - Setting timeout for idle persistent connections to 30 minutes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@983 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-25 10:21:00 +00:00
orbiter
1aa4ba8b62 added post-search filtering of redundant urls (longer than existing cited) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@982 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-25 09:06:00 +00:00
low012
8e7b214e41 *) 5 inverted commas are needed instead of only 4 to strongly emphasise a text (closer to Wikipedia's syntax now) 
*) Tried to fix nested <pre> and [= issues, but did not succeed due to sleepyness. Will continue work when I'm awake and have some time.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@981 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-24 22:37:10 +00:00
orbiter
8d827cdb30 tried to fix problems with order of network list by last-seen (which could also improve the network picture) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@980 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-24 14:07:43 +00:00
low012
1e6cf9fd41 *) indented texts between escape symbols behave like indented preformatted texts now 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@979 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-24 13:21:57 +00:00
theli
446e7e8bef *) Bugfix for Seed-Upload - Permission denied problem 
See: http://www.yacy-forum.de/viewtopic.php?p=11648#11648

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@978 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-24 08:54:46 +00:00
orbiter
097009d910 experimental visualization of DHT access during global search (temporary) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@977 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-24 00:34:15 +00:00
orbiter
4dcbc26ef1 introduction of search profiles; very experimental 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@976 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-23 17:50:27 +00:00
theli
6c48c3ce39 *) Bugfix for ArithmeticException during IndexTransfer 
See: http://www.yacy-forum.de/viewtopic.php?t=1362

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@974 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-23 16:07:44 +00:00
low012
fd1b08303e fixed a bug theli pointed out (indented <pre>-blocks) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@973 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-23 14:04:21 +00:00
theli
525c8dcbd4 *) Adding Traffic Statistic for Crawler 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@972 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-23 12:35:48 +00:00
theli
9a5ab62928 *) Adding yacy specific X-YACY-Index-Control header which can be used by clients 
to disallow yacy to index the response that belongs to the request where 
   X-YACY-Index-Contro is set to "no-index"

*) Bugfix for Seed-List download via Remote Proxy.
   Now the pragma and cache-control http headers of the request are properly set to "no-cache" 
   See: http://www.yacy-forum.de/viewtopic.php?p=11639#11639

*) Bugfix for http-Proxy
   yacy has ignored "no-cache"- pragma and cache-control http headers that were send in requests.
   Now, these request headers are evaluated properly

TODO: Missing evaluation of "no-store" request headers

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@971 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-23 10:35:05 +00:00
theli
539f9e57aa *) Disabling RemoteProxy if remoteProxyName is null or empty 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@970 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-23 09:03:40 +00:00
theli
5a25ad9109 *) Bugfix for useRemoteProxy4YACY and useRemoteProxy4SSL check 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@969 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-23 08:59:11 +00:00
theli
02d9af1a70 *) Restructuring and extending of Remote Proxy Support 
- remote proxy configuration can now be "really" changed on the fly and takes effect immediately
   - adding possibility to disable remote proxy usage for yacy->yacy communication
   - adding possibility to disable remote proxy usage for ssl
   - restructuring proxy configuration so that it is stored in a single place now

*) Adding possibility to import a foreign word DB (or even more of them in parallel) 
   at runtime into the peers DB
   - this can be done by calling IndexImport_p.html 
   - ATTENTION: please not that at the moment this thread must be aborted via gui
     before a normal server shutdown is done. 
   - TODO: integrating IndexImport Thread into normal server shutdown
   - TODO: Adding posibility to import crawl-queues, etc. from foreign peers
   - TODO: removing old import function from yacy.java and calling the new routines instead

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@968 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-22 13:28:04 +00:00
low012
222607ef0f *) added <pre>-tag for preformatted text to yacyWiki (as requested by theli) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@967 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-22 13:00:36 +00:00
allo
117a424d00 bugfix for sharing png/gif files in WWW/SHARE 
http://www.yacy-forum.de/viewtopic.php?p=11565


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@966 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-20 14:58:15 +00:00
theli
af3060938b *) Bugfix for manual peer ping functionality 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@965 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2005-10-20 11:43:43 +00:00