Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
8,070 Commits 1 Branch 20 Tags 836 MiB
411aab02e3
Commit Graph

473 Commits

Author SHA1 Message Date
Michael Peter Christen
87f0210480 enriched log output to find NPE in HeapReader 2012-01-15 12:08:46 +01:00
Michael Peter Christen
254adea51c small fixes 2012-01-13 11:24:08 +01:00
Michael Peter Christen
49be60a7c8 WorkflowProcess is forced to make small pauses if shortMemoryStatus is 
reached.
 2012-01-10 03:03:12 +01:00
Michael Peter Christen
b7bb84c0bb set a limit to CharBuffer object size to fight against bad/too large 
content
 2012-01-10 03:02:17 +01:00
Marek Otahal
72adbeae90 !Important: move from Hashtable to HashMap 
Hashtable is an obsolete collection v1, now since v2 offers HashMap with same or better
functionality. Please review, almost all code was already moved, so only a few changes. That is not the issue,
but I found notices that some (ugly big) helper classes had to be created in past
to compensate missing Hashtable's functionality. I'd like input if we can remove some of them.
look for //FIX: if these commits

Signed-off-by: Marek Otahal <markotahal@gmail.com>
 2012-01-09 01:29:18 +01:00
Marek Otahal
f75b5e40e0 little fix in copy() 
Signed-off-by: Marek Otahal <markotahal@gmail.com>
 2012-01-09 01:16:46 +01:00
Michael Christen
216a287a85 Merge commit '6d4e08ed06c5cd28c45981b2ebe31c7f7ec6fd83' into quix0r 
Conflicts:
	source/de/anomic/crawler/CrawlQueues.java
 2012-01-04 20:16:37 +01:00
Michael Christen
20962a4ed7 added metadata node stub for metadata from blobs 2012-01-03 14:38:03 +01:00
Michael Christen
575dbbaa93 enhancements in Blob retrieval: try to use less CPU resources by testing 
a blog first that most certainly has wanted entries.
 2012-01-02 02:14:05 +01:00
Roland 'Quix0r' Haeder
6d4e08ed06 Rewrote filesize() to (hopefully) avoid a NPE, rewrote Blacklist class to concurrent classes to avoid a CME 2011-12-29 03:42:38 +01:00
Roland 'Quix0r' Haeder
fa08ed5ae5 Fixed a lot CHMOD rights (no need for execute flag on *.java/*.html) and introduced local/remote crawl size ratio based check 2011-12-29 00:33:16 +01:00
Michael Christen
9e5894c784 Removed handling of components objects for URIMetadataRows. 
This is a preparation to replace this rows with nodes from the node
store.
 2011-12-17 01:27:08 +01:00
Michael Christen
c04bfaa51b refactoring 2011-12-16 23:59:29 +01:00
Michael Peter Christen
613ab6a69d added BEncodedHeapBag and BEncodedHeapShard which are storage container 
for a new metadata store. An abstraction of the content for this storage
is defined with MapStore. A MapStore is an abstraction of a RDF Node
store.
 2011-12-16 23:00:50 +01:00
Michael Christen
6fecd0db88 one more performance hack to prevent costly md5 computation 2011-12-15 23:33:41 +01:00
Michael Christen
e13441b069 better digest pool size (smaller by default but unlimited) 2011-12-15 17:45:46 +01:00
Michael Christen
1f4afb4dc0 performance hacks 2011-12-15 15:15:53 +01:00
Michael Christen
e9dc99fe15 added rules to set specific RWIs as private RWIs which are not 
transmitted to remote peers. This will be used for private index copies
and phonetic indexes.
 2011-12-14 22:15:51 +01:00
Michael Peter Christen
0bcef2d156 added feature as requested in 
http://forum.yacy-websuche.de/viewtopic.php?f=18&t=3461
The search can now be configured with a non-display host list.
the search will always exlude the given list of host unless they are
requested directly using the host navigation
 2011-12-13 00:16:05 +01:00
Michael Christen
204c29f010 small bugfixes for search result display and cache display 2011-12-10 01:35:38 +01:00
Michael Christen
078fcde0dd bad initialization 2011-12-07 01:02:23 +01:00
Michael Christen
14e45e90fd patch for a bug that I don't understand by now. 2011-12-07 00:52:04 +01:00
Michael Christen
86b3385847 fixed a deadlock during secondary remote search 2011-12-07 00:18:34 +01:00
Michael Christen
404758698a less io operations 2011-12-06 22:04:34 +01:00
Michael Christen
044f83feed added some pauses into the search process which shall produce 
better-ranked search results. without that pauses the result page will
only contain links from the peer that answers first which is not a good
average picture of all the peers that provided results
 2011-12-06 15:28:48 +01:00
sixcooler
448656087a probably fix for http://bugs.yacy.net/view.php?id=94 
(don't know how to force this exception)
 2011-12-06 02:49:58 +01:00
Michael Christen
d35bdc2df6 removed npe 2011-12-05 23:37:49 +01:00
Michael Christen
e7e429705a - less automatic indexing after a search (needs to reset the default 
crawl profiles)
- fix for concurrency problem in storage of serverSwitch Properties
- markup update
 2011-12-05 16:22:11 +01:00
Michael Christen
9cd469e6d6 added pull request from als plus an NPE fix 2011-12-04 12:15:03 +01:00
orbiter
83335c3b09 fix for http://bugs.yacy.net/view.php?id=78 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8127 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-12-01 20:57:22 +00:00
orbiter
35a9e8f307 - fixed network graphic 
- debuged evaluation tables
- changed cache settings in template engine
- some speed hacks
- changed int angles for peer positions in network graphic to double angles

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8124 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-30 20:50:41 +00:00
Al Sutton
8993cac4d8 Initial performance improvements 2011-11-30 11:15:54 +00:00
orbiter
8895d8c1cd removed unnecessary log entries 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8117 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-27 16:54:48 +00:00
orbiter
bc5df0eef5 updated ranking tables (fresh computation) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8103 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-25 12:37:00 +00:00
orbiter
5a55397f99 some last-minute performance hacks 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8101 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-25 11:23:52 +00:00
orbiter
c9216d5adf fixed secondary remote search (the process that finds distributed join situations) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8098 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-24 22:45:31 +00:00
orbiter
0cf9ebc3b0 speed enhancements when parsing RWI rows (makes search slightly faster) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8096 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-24 17:37:04 +00:00
orbiter
709013385a fix for language fix 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8086 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-24 09:50:30 +00:00
orbiter
c0c6e9e7a5 fix for bad language encoding 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8082 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-23 22:03:08 +00:00
orbiter
05f34a3fa7 added a full, complete, database insert, update and delete API for the tables. 
Please see this example:

list all database tables:
http://localhost:8090/api/table_p.xml

now create a new table and insert some values into 'mytable'
http://localhost:8090/api/table_p.xml?table=mytable&pk=&commitrow=&col_termin=Release%20Machen&col_datum=24.11.2011&col_status=ongoing

list the table content:
http://localhost:8090/api/table_p.xml?table=mytable&pk=

update the table and change a single value inside. You must refer to the row using a primary key 'pk'
http://localhost:8090/api/table_p.xml?table=mytable&pk=000000000001&commitrow=&col_datum=29.11.2011

you can also select rows using a search operator
http://localhost:8090/api/table_p.xml?table=mytable&pk=&count=10&search=

now lets delete the row:
http://localhost:8090/api/table_p.xml?table=mytable&pk=&deleterows=pk_000000000001

and we can also delete the complete table:
http://localhost:8090/api/table_p.xml?table=mytable&deletetable=

You can use this to administrate the robots, bookmarks and API steering using an outside application!

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8071 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-22 12:31:07 +00:00
orbiter
3a15e58e28 - increased stability when opening the robots table 
- increased stability when deleting tables

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8034 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-14 15:33:35 +00:00
orbiter
775b44017e refactoring 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8033 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-14 15:11:57 +00:00
orbiter
e914a30099 fix for npe 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8032 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-14 12:32:15 +00:00
orbiter
0d858d48ec replaced String with StringBuilder in suggestion process 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8020 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-09 14:42:55 +00:00
orbiter
e58438c01c - added a new retry connector for solr (for cases where solr responses are slow) 
- added a new exist property into the metadataRepository which includes solr entries

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8016 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-08 11:49:04 +00:00
orbiter
d8d9735b4f stability bugfix 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8012 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-03 14:41:38 +00:00
orbiter
f121f4bb45 fix for link in Supporter and Suftipps page 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8010 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-11-01 22:49:14 +00:00
orbiter
1b86d06d1e fix for http://bugs.yacy.net/view.php?id=62 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8004 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-10-26 10:07:16 +00:00
orbiter
eb9c9edb01 enhanced table method (used by almost all yacy api interfaces) 
git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@8000 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-10-18 23:38:19 +00:00
orbiter
5af9598bd1 enhanced exported row parsing during row import 
this affects the search and dht receive speed

git-svn-id: https://svn.berlios.de/svnroot/repos/yacy/trunk@7994 6c8d7289-2bf4-0310-a012-ef5d649a1542
 2011-10-10 09:46:38 +00:00