Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-19 00:01:41 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
6,389 Commits 1 Branch 20 Tags 836 MiB
1bbe14d23f
Commit Graph

4231 Commits

Author SHA1 Message Date
orbiter
1bbe14d23f SVN 6716 unfortunately contained parts of the unfinished SMB integration. To fix compile errors the remaining parts of the SMB implementation stub is added with this commit. 
This adds the jcifs smb library.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6717 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-05 21:46:22 +00:00
orbiter
884b262130 - added a new Wiki Namespace Navigator 
- some redesign of Navigator data structures

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6716 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-05 21:25:49 +00:00
orbiter
617dfbbd06 allo 'authorization by encoded password' also if requesting client is not from localhost but from the same host as yacy is running on. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6714 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-05 16:03:55 +00:00
orbiter
270fb38674 - fixed some bugs in Table viewer 
- added 'select all' feature in Tables_p
- enhanced ViewFile.html: has now an input field to load arbitrary resources from the web and analyze them (!!!)
- included the ViewFile servlet into the Index Administration menu
- show in ViewFile if ressource is in url-db and/or in Web cache
- bugfixes to BEncodedHeap and Tables management

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6713 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-05 15:41:15 +00:00
orbiter
599c3766c4 added authentication to automated API call 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6711 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-04 14:10:03 +00:00
orbiter
727dd9b193 - fixed a bug in robots.txt parser 
- moved storage of robots.txt entries to WorkTables, so it is now possible to browse the robots entries with the table browser

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6710 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-04 11:58:07 +00:00
orbiter
54af9e6b49 - added parsing of robots meta-tag in html headers to detect a noindexing request 
- added evaluation and indexing prevention in case that a noindexing is given in a html file

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6709 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-03-03 23:32:56 +00:00
orbiter
46c4f8b68a better look-ahead into the crawl queue: show more on crawl monitor 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6699 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-24 23:11:58 +00:00
lotus
7b546415dc added svn6695 for windows 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6697 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-24 14:58:53 +00:00
orbiter
f175f9a2d3 changed way how number of search requests are counted: 
so far only search requests at the remote search interface had been counted.
This was done to protect the privacy of searchers, because counting was not done and published at the own search interface.
This caused that no search requests of robinson peers had been counted, becuase they cannot be counted at remote peer.
This change introduces a distinction of locally done search requests at the local search interface from search requests that are on the local interface but had been submitted from a remote IP without authentication.
Now 3 counters are maintained:
- partial count of remote searches
- total count of local searches on robinson peers from non-authenticated clients
- total count of local searches on robinson peers from localhost or authenticated clients
In the global statistic of search requests now the first two counters of the three cases are added
Because we habe a large number of robinson peers with a large number of remote non-authenticated requests the statistic should show at least three times of the number of search requests.


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6696 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-24 13:53:55 +00:00
orbiter
84222e3b4f fix for auto-updater: delete old libraries before copy of new one 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6695 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-24 13:46:50 +00:00
sixcooler
cd6de83905 next try for for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2703 
(reverted 6692)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6694 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-23 15:59:58 +00:00
sixcooler
bfe4693e9a fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2703 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6693 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-23 13:46:56 +00:00
orbiter
93b7ddc27d fix for http://forum.yacy-websuche.de/viewtopic.php?p=19376#p19376 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6684 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-21 22:49:35 +00:00
orbiter
8030ed3319 self-healing for lost crawl profile handles 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6680 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-18 21:55:45 +00:00
orbiter
e3e5e05ec2 fix for problem in ranking setting which was caused by the introduction of a toString() method in serverObjects 
see also: http://forum.yacy-websuche.de/viewtopic.php?p=19310#p19310

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6678 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-17 21:31:08 +00:00
orbiter
e3ccfb54aa fix for display problem in Firefox on MacOS X 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6677 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-17 09:08:16 +00:00
orbiter
564927ce72 redesign of CrawlResult data structures because of OOM occurrences during URL deletion processes. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6675 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-16 23:06:04 +00:00
orbiter
30c8185139 fix for sid check 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6673 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-15 23:31:32 +00:00
orbiter
ef62d017e5 integrated session id filtering for crawler 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6672 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-15 23:15:17 +00:00
orbiter
d8d9984913 added framework for session id filtering (not ready yet) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6671 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-15 22:30:41 +00:00
orbiter
2bc36de336 - fix for bug in svn 6669 
- cleanup

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6670 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-15 22:06:13 +00:00
orbiter
d378ca4604 better handling of concurrency in seed 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6669 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-15 15:57:35 +00:00
orbiter
6538043d89 fix for http://forum.yacy-websuche.de/viewtopic.php?p=19189#p19189 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6668 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-15 15:45:31 +00:00
sixcooler
e071d71f19 fix for yacy-banner-network-values 
http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2521

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6659 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-09 18:22:36 +00:00
lotus
945e0ba5a5 allow global search if res. observer disabled index transmission 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6658 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-09 17:14:16 +00:00
lotus
8faeedd99a not a fix! for: 
http://forum.yacy-websuche.de/viewtopic.php?f=5&t=2679

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6657 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-09 09:33:30 +00:00
sixcooler
787b588c33 reverted a part of svn6636: 
- didn't work on blobs >2GB
- should be obsolete since svn6651
http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2652&sid=7fa98fd3edfc2a03f26394d545e3e3c1&p=19172#p19172

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6655 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-07 19:32:46 +00:00
lotus
11188cd7eb resource observer now uses the Java 6 method to check for free space. thus, disk observing now needs Java 6 installed. 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6652 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-06 18:48:06 +00:00
sixcooler
089877f32c my first commit - hopefully fix for merge problem 
- http://forum.yacy-websuche.de/viewtopic.php?f=6&t=2652

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6651 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-05 19:38:00 +00:00
orbiter
be18b5d8cd fix for 'cannot switch back to default language'-bug 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6649 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-04 23:53:02 +00:00
orbiter
d6391f2537 better handling of rewrite cases where the resulting rewrite blob entry is equal in size 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6648 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-04 23:17:47 +00:00
orbiter
ef9473d92c added another sixcooler suggestion: recycle corrupted records 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6647 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-04 16:25:05 +00:00
orbiter
fe78edac32 - view API calls in correct date-order 
- execute recorded API calls in date-order

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6646 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-04 15:51:54 +00:00
orbiter
74e736c903 missing file for last commit 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6645 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-04 14:52:58 +00:00
orbiter
308a973503 refactoring of tables data organisation 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6644 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-04 11:26:23 +00:00
lotus
85ca96227f fix for re-enable parser 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6643 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-03 19:33:59 +00:00
orbiter
8a76f38d26 Added a new steering servlet that can be used to repeat actions that had been made on the yacy interface. This can be used to: 
- start again a previously started crawl
- submit settings (again). This option will be used to transmit
  all settings of one peer to another peer if the remote-peer
  steering function is ready
This steering framework will also be used for a 'schedule-everything'
which will also include a new scheduler for crawling.
  

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6642 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-03 09:31:12 +00:00
orbiter
840527689b more simplification of bookmark class 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6639 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-01 23:04:52 +00:00
orbiter
d77782a8d5 removed bookmark tags file, tags are now stored only in RAM 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6638 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-01 22:44:59 +00:00
orbiter
ada0ce9de3 refactoring of bookmarks: there is a big performance problem in the bookmarks code and furthermore the bookmarks 
will loose its leading role for the re-crawl funtion when the new api tables will work. To be prepared for a replacement
of such functions the bookmark class is re-organised.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6637 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-01 22:18:56 +00:00
orbiter
3751ab4ae2 added sixcoolers patch and more checks/removed unnecessary code 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6636 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-01 16:11:00 +00:00
orbiter
d8d8562c59 fill key with zeros during normalization 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6635 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-01 15:40:16 +00:00
orbiter
a131ebbcb5 one more fix for NPE, see 
http://forum.yacy-websuche.de/viewtopic.php?p=19010#p19010

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6634 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-02-01 11:28:37 +00:00
orbiter
24060885b6 - added Tables abstraction in data.Tables.java 
fix for
http://forum.yacy-websuche.de/viewtopic.php?p=18910#p18910
http://forum.yacy-websuche.de/viewtopic.php?p=18894#p18894
http://forum.yacy-websuche.de/viewtopic.php?p=18814#p18814


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6631 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-29 18:02:09 +00:00
orbiter
7fdf59a77f misc NPE check 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6630 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-29 15:59:24 +00:00
orbiter
a512aef6ad fix for http://forum.yacy-websuche.de/viewtopic.php?p=18918#p18918 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6629 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-29 10:33:20 +00:00
lotus
38a3d55afd added more possible php extensions for html 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6621 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-24 20:04:31 +00:00
orbiter
4403304957 bugfix for list() 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6616 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-22 16:00:56 +00:00
orbiter
3889438db6 fix for bookmarks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@6615 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2010-01-22 14:20:24 +00:00