Commit Graph

4539 Commits

Author SHA1 Message Date
orbiter
78087da287 - changed seed file storage to clear text
- fixed kill script
- fixed saving of seed file (had been corrupted by latest changes)
- some refactoring

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4799 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-14 20:30:44 +00:00
orbiter
5fde679acb - fixed problem in performance configuration
- extended rss fetch size for rssTerminal


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4798 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-13 15:28:55 +00:00
orbiter
db032fb6de - added RWI transmissions to the event terminal
- fixed bug in Collage
- added 'embedded mode' to collage
- integrated Collage to terminal_p as iframe in embedded mode (Pictures now visible in terminal_p)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4797 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-13 11:46:20 +00:00
orbiter
239cc4428d - better domain graph, faster when more links exist, looks better
- new authorization rule: localhost is always authorized for administration. This solves many problems with ajax, and also fixed a problem in rssTerminal
- fix bug in RSSFeed which prevented that entries had been recognized as individual, new entries
- added reloading/updating of status image on status page

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4796 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-12 22:23:29 +00:00
orbiter
415b92bb07 fix for http://forum.yacy-websuche.de/viewtopic.php?f=6&t=1148&hilit=&p=7711#p7711
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4795 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-12 15:06:04 +00:00
apfelmaennchen
2113672bf2 small fix on tag comporator functions
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4794 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-12 15:05:27 +00:00
orbiter
dd75b3cabc - patch for bad profiles
- time-out when deleting profiles

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4793 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-12 14:58:56 +00:00
apfelmaennchen
0f7449840e - minor changes on YaCy-UI
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4792 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-12 09:42:26 +00:00
apfelmaennchen
5c6ff5ebea - added attributes to /xml/bookmarks/xbel/xbel.xml
- tag=tagName returns only bookmarks which match tagName in a single XBEL-folder tagName
- added metadata for public=true/false to XBEL output
- added the attribute 'added' (date) to XBEL output

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4791 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-12 09:36:21 +00:00
apfelmaennchen
6c79edcd56 - added attributes to /xml/bookmarks/tags/get.xml
- top=number returns the number most frequent tags
- sort=comporator lets you choose between "size" and "alpha" as comporator function
- tag=tagName returns all tags associated with bookmarks tagged with tagName

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4790 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-12 09:31:53 +00:00
danielr
2080ff72b7 ftpc fix for npe
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4789 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-10 22:03:51 +00:00
lotus
e021278bf0 unescape link display in search results
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4788 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-10 11:25:54 +00:00
danielr
3f1721b827 informational comment
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4787 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-10 11:21:51 +00:00
danielr
04a51b775a changed .org/.net back to America
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4786 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-10 11:10:10 +00:00
apfelmaennchen
bbda1a45a9 - added box800.png (quick resize of box600, needs to be fine tuned)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4785 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-10 08:55:57 +00:00
danielr
74b1a60043 fixed "java.lang.NoClassDefFoundError: org/a"
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4784 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-10 08:42:31 +00:00
apfelmaennchen
fa3ed2888d - bookmarks are now retrieved from /xml/bookmarks/xbel/xbel.xml and do no longer require a seperate servlet
- minor bug fixes
- refactoring (obsolete items)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4783 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-10 08:37:18 +00:00
danielr
f44d5d302b updated TLDs
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4782 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-09 12:46:10 +00:00
hermens
5bfc02ccfb Repair publishThread
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4781 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-09 11:14:32 +00:00
orbiter
f42c8cf69c updated terminal and dynamic webstructure applet: can now change when crawl is running
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4780 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-09 00:01:47 +00:00
orbiter
906c144799 - design update to new terminal and rssTerminal
- added terminal to main menu
- removed transfer size limitation in server

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4779 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-08 21:33:16 +00:00
orbiter
7ec01d444a fix for npe
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4778 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-08 20:25:11 +00:00
orbiter
ad0f905124 fix for npe in crawler
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4777 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-08 20:16:19 +00:00
danielr
ae03a54d23 pdfParser: updated lib, fixed ClassNotFoundException: CMSError
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4776 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-08 16:55:45 +00:00
danielr
0d3808bd9e minor refactoring
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4775 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-08 16:51:01 +00:00
orbiter
719f5defb1 updated some grafics at new terminal_p
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4774 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-07 23:42:14 +00:00
orbiter
58830e9b28 added new terminal servlet using current visualization methods and a new one: a processing (processing.org) applet.
the new servlet can be found at
http://localhost:8080/terminal_p.html
..to be enhanced..

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4773 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-07 22:49:07 +00:00
lotus
9bc56a9edc xss protection
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4772 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-07 16:37:13 +00:00
orbiter
b32736762c enhanced rssTerminal
- 3 lines possible
- distinguishing of private and public data, if not authorized only public data is shown
- shows now more events, including local searches in clear text if user is logged in
- simplyfied peer events
- better recognition of 'real' new peers
- presentation of peer pings from other peers

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4771 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-06 23:05:48 +00:00
orbiter
fbb712c669 refactoring:
moved importer classes to crawler and plasma package

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4770 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-06 13:44:38 +00:00
orbiter
ee81ff4ef4 added crawler target directory for build
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4769 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-06 00:37:51 +00:00
orbiter
1689030ee8 refactoring: moved all crawler classes into their own package
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4768 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-06 00:32:41 +00:00
orbiter
fe4871ac02 removed empty package
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4767 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-05 23:54:00 +00:00
orbiter
3082edfdbc ups
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4766 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-05 23:19:49 +00:00
orbiter
d2ba1fd2ab major step forward to network switching (target is easy switch to intranet or other networks .. and back)
This change is inspired by the need to see a network connected to the index it creates in a indexing team.
It is not possible to divide the network and the index. Therefore all control files for the network was moved to the network within the INDEX/<network-name> subfolder.
The remaining YACYDB is superfluous and can be deleted.
The yacyDB and yacyNews data structures are now part of plasmaWordIndex. Therefore all methods, using static access to yacySeedDB had to be rewritten. A special problem had been all the port forwarding methods which had been tightly mixed with seed construction. It was not possible to move the port forwarding functions to the place, meaning and usage of plasmaWordIndex. Therefore the port forwarding had been deleted (I guess nobody used it and it can be simulated by methods outside of YaCy).
The mySeed.txt is automatically moved to the current network position. A new effect causes that every network will create a different local seed file, which is ok, since the seed identifies the peer only against the network (it is the purpose of the seed hash to give a peer a location within the DHT).
No other functional change has been made. The next steps to enable network switcing are:
- shift of crawler tables from PLASMADB into the network (crawls are also network-specific)
- possibly shift of plasmaWordIndex code into yacy package (index management is network-specific)
- servlet to switch networks 

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4765 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-05 23:13:47 +00:00
danielr
d70a472460 added file for previous commit
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4764 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-05 05:19:01 +00:00
danielr
d32fe84472 added default User-Agent
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4763 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-04 17:26:19 +00:00
danielr
8c5f062e0b corrected YaCy version in HTTP User-Agent
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4762 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-04 12:18:00 +00:00
danielr
d7b21bc90c re-added gzip POST for transferRWI/URL (HTTP/1.1 compliant)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4761 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-04 10:53:04 +00:00
f1ori
a5a1f19368 * allow to force login for xbel, needed for yacybar
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4757 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-03 11:13:27 +00:00
low012
8d83febb95 *) BlacklistCleaner_p.java reports exception to log instead of System.err
*) changes in formatting for better readability in BlacklistCleaner_p.java
*) replaced  test for necessary Java version (was 1.4.2, is 1.5 now)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4756 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-03 10:16:04 +00:00
danielr
d4bce6affd refactoring (initialized static fields, removed empty if/else, serialized some fields in serializable classes)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4755 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-03 09:06:00 +00:00
daburna
19ca452666 updated language file
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4754 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-02 20:19:05 +00:00
danielr
be2c9c07ff escape some unescaped characers in URLs (fixes problems with proxy)
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4753 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-02 08:19:47 +00:00
orbiter
d0678f7ab9 refactoring as result of
http://forum.yacy-websuche.de/viewtopic.php?f=6&t=959&p=7560#p7560

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4752 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-05-01 22:40:42 +00:00
orbiter
483e9a2066 - shifted tld recognition methods from yacyURL to serverDomains
- changed isLocal Property in such a way that it is possible to see if a domain is in the internet (and not intranet)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4751 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-04-30 23:06:42 +00:00
orbiter
a3df23659c re-implementation of charset checking
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4750 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-04-30 13:23:05 +00:00
orbiter
75a1702133 - fix for ConcurrentModificationException during shutdown
- fix for Ranking distribution problem (suma-lab peer does not exist any more)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4749 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-04-30 11:19:52 +00:00
orbiter
27ab0a5f89 fixed XSS problem in ConfigProperties
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4748 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-04-29 22:47:00 +00:00
orbiter
32b5b057b9 - modified, simplified old kelondroHTCache object; I believe it should be replaced by something completely new
- removed tree data type in kelondroHTCache
- added new class kelondroHeap; may be the core for a storage object that will once replace the many-files strategy of kelondroHTCache
- removed compatibility mode in indexRAMRI


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@4747 6c8d7289-2bf4-0310-a012-ef5d649a1542
2008-04-29 22:31:05 +00:00