Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
7,771 Commits 1 Branch 20 Tags 836 MiB
ebd840ebf6
Commit Graph

5194 Commits

Author SHA1 Message Date
orbiter
ebd840ebf6 - enhanced description on search front page 
- fixed language and heuristic modifier
- added hint to crawl start that we can do also ftp and smb crawls
- added a protocol extension to remote crawls to transport all search modifiers to remote peers

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8108 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-26 13:40:33 +00:00
apfelmaennchen
77a080ced9 smaller fixes for YMarks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8105 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-25 17:33:03 +00:00
orbiter
e22f8497c9 - tested the ARC methods 
- removed strict authentication (if password is empty; this was buggy and not useful; can be switched on if necessary globally and not for each interface method)
- increased speed of CrawlResults page (no dns lookup any more)
- increased speed of favicon display (removed dns lookup)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8104 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-25 14:09:25 +00:00
orbiter
bc5df0eef5 updated ranking tables (fresh computation) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8103 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-25 12:37:00 +00:00
orbiter
5a55397f99 some last-minute performance hacks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8101 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-25 11:23:52 +00:00
apfelmaennchen
dd1482aaf5 further update to YMarks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8100 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-25 00:32:18 +00:00
orbiter
c9216d5adf fixed secondary remote search (the process that finds distributed join situations) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8098 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-24 22:45:31 +00:00
orbiter
64fd20b857 new default ranking profile 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8097 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-24 20:07:52 +00:00
orbiter
0cf9ebc3b0 speed enhancements when parsing RWI rows (makes search slightly faster) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8096 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-24 17:37:04 +00:00
orbiter
c9a0dbd25a added a security check 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8094 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-24 16:29:35 +00:00
orbiter
ee8b1d4de1 fixed unresolved pattern and unwanted local/global switch when using votes on search results 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8093 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-24 16:05:09 +00:00
orbiter
c584db991f creating a bookmark from the search results now works again .. with new YMarks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8092 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-24 14:57:09 +00:00
orbiter
1120f0c93c update to network graphics: slightly less crawling activity, slightly stronger color for query activity 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8089 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-24 12:55:09 +00:00
orbiter
6cd27473f5 - better default values for caching and cache usage 
- set new caching and verification behavior according to use case automatically

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8087 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-24 10:22:02 +00:00
orbiter
709013385a fix for language fix 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8086 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-24 09:50:30 +00:00
orbiter
1019c36dad bug fixes and speed enhancements for search 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8085 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-24 01:30:12 +00:00
orbiter
507c9d478d much better timing when search globally; less blocking; more results earlier! 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8084 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-23 23:39:34 +00:00
orbiter
8e0b2c5832 fixed cluster search 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8083 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-23 22:21:14 +00:00
orbiter
c0c6e9e7a5 fix for bad language encoding 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8082 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-23 22:03:08 +00:00
apfelmaennchen
564374d1fe - included YMarks in addition to old bookmarks in yacysearchitem.html; don't get confused by the old bookmark dialog, the ymark is automatically added silently beforehand. 
- reworked bookmark creation on crawlstart
- many smaller adjustments to ymarks


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8072 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-22 23:50:49 +00:00
orbiter
05f34a3fa7 added a full, complete, database insert, update and delete API for the tables. 
Please see this example:

list all database tables:
http://localhost:8090/api/table_p.xml

now create a new table and insert some values into 'mytable'
http://localhost:8090/api/table_p.xml?table=mytable&pk=&commitrow=&col_termin=Release%20Machen&col_datum=24.11.2011&col_status=ongoing

list the table content:
http://localhost:8090/api/table_p.xml?table=mytable&pk=

update the table and change a single value inside. You must refer to the row using a primary key 'pk'
http://localhost:8090/api/table_p.xml?table=mytable&pk=000000000001&commitrow=&col_datum=29.11.2011

you can also select rows using a search operator
http://localhost:8090/api/table_p.xml?table=mytable&pk=&count=10&search=

now lets delete the row:
http://localhost:8090/api/table_p.xml?table=mytable&pk=&deleterows=pk_000000000001

and we can also delete the complete table:
http://localhost:8090/api/table_p.xml?table=mytable&deletetable=

You can use this to administrate the robots, bookmarks and API steering using an outside application!

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8071 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-22 12:31:07 +00:00
lotus
3cc93325f0 temporary remove compare search from tray 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8070 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-22 12:12:25 +00:00
orbiter
c93f10417a add a bookmark automatically each time a new crawl is started 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8063 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-22 00:03:20 +00:00
orbiter
e4a82ddd8b produce a bookmark entry from every crawl start. these bookmarks are always private. 
these bookmarks will be used to get a source reference for the search in case of intranet or portal searches.

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8062 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-21 23:10:29 +00:00
apfelmaennchen
6287c2b4a9 YMarks: 
- introduced tag manager - a quite powerful tool (still not 100% stable, so be careful)

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8060 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-20 22:42:15 +00:00
cominch
2236e01137 Minor correction to prevent useless comma at beginning of string, created from list 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8059 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-20 15:20:57 +00:00
apfelmaennchen
5581be12fb YMarks: 
- added backend and api for tag management


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8058 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-19 20:59:21 +00:00
orbiter
804e48888b smaller bug fixes for search behavior; should produce less unnecessary removals and an exact number of results as shown in counter 
should also be a little bit faster

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8057 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-18 13:09:07 +00:00
apfelmaennchen
a3eebfdcba YMarks: 
- show active/running crawls
- execute crawls (works currently only if API entry is available)
- various smaller fixes

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8056 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-17 23:11:27 +00:00
orbiter
c50f8f9a06 code cleanup 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8055 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-17 13:40:22 +00:00
orbiter
84c3fc9d97 local/global fixes in search, better abstraction 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8054 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-17 01:05:45 +00:00
apfelmaennchen
4f95f72124 YMarks: 
- working direct importer for YaCy Crawl Starts
- working direct import for old bookmarks.db

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8052 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-16 23:10:53 +00:00
orbiter
aa322bc6d0 fix 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8050 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-16 15:36:30 +00:00
orbiter
97d1347adb added also a default accept field to robots.txt downloads 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8049 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-16 15:33:55 +00:00
orbiter
f183d3822c added a default accept header in http requests since some http fraud detection functions check that this header field exist 
see also: http://bad-behavior.ioerror.us/ in source file browser.inc.php

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8048 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-16 15:27:43 +00:00
orbiter
06352b8d6b more logging 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8047 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-16 14:09:50 +00:00
orbiter
a99934226e more logging for debugging of robots.txt 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8046 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-16 13:56:31 +00:00
orbiter
7a5841e061 fix for robot parser 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8045 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-16 13:12:46 +00:00
orbiter
458c20ff72 fix for robot parser 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8044 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-16 13:06:46 +00:00
orbiter
017a01714d - enhanced logging in robots.txt parser for remote debugging 
- robots.txt is now more robust against database operations

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8043 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-16 01:03:49 +00:00
apfelmaennchen
a8dfe787ed - updated to jquery flexigrid 1.1 
- YMarks.html automatically  recognizes if a bookmark is a crawl start


git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8040 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-15 21:45:17 +00:00
orbiter
eb1c7c041d write info about robots.txt evaluation into getpageinfo_p.xml 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8038 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-15 00:33:54 +00:00
apfelmaennchen
abba31f02e - bugfix for correctly sorting ymarks 
- some tuning for the autotagger (still not perfect)
- /api/ymarks/get_metadata.xml now provides info for crawlstarts
- removed unused code

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8036 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-14 22:00:44 +00:00
orbiter
3a15e58e28 - increased stability when opening the robots table 
- increased stability when deleting tables

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8034 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-14 15:33:35 +00:00
orbiter
775b44017e refactoring 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8033 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-14 15:11:57 +00:00
orbiter
e914a30099 fix for npe 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8032 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-14 12:32:15 +00:00
apfelmaennchen
5f7dbe1c42 - some refactoring (ymarks) 
- improvement for autotagger (is now able to create/detect  multi word tags e.g. 'open source')



git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8031 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-13 23:19:47 +00:00
orbiter
78ce3b13be typo 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8027 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-10 11:57:26 +00:00
orbiter
85d6bf4ac4 fixed urls to media content during indexing 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8021 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-09 15:40:14 +00:00
orbiter
0d858d48ec replaced String with StringBuilder in suggestion process 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8020 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-09 14:42:55 +00:00