Marown/yacy_search_server
1
0
Fork 0
mirror of https://github.com/yacy/yacy_search_server.git synced 2024-09-21 00:00:13 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity
6e1dc444c3
yacy_search_server/source/net/yacy/http/servlets/YaCyDefaultServlet.java

1132 lines
51 KiB
Java
Raw Normal View History

implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
// YaCyDefaultServlet
// Copyright 2013 by Michael Peter Christen; mc@yacy.net, Frankfurt a. M., Germany
// First released 2013 at http://yacy.net
//
// This library is free software; you can redistribute it and/or
// modify it under the terms of the GNU Lesser General Public
// License as published by the Free Software Foundation; either
// version 2.1 of the License, or (at your option) any later version.
//
// This library is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
// Lesser General Public License for more details.
//
// You should have received a copy of the GNU Lesser General Public License
// along with this program in the file lgpl21.txt
// If not, see <http://www.gnu.org/licenses/>.
//
refactoring: moving all servlets to the same package (the solr servlet is currently actually a filter which should be changed somehow)
2013-12-23 01:32:29 +01:00
package net.yacy.http.servlets;
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
import java.awt.Image;
import java.awt.image.BufferedImage;
import java.io.BufferedInputStream;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileInputStream;
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
import java.io.FileNotFoundException;
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.lang.ref.SoftReference;
fixed generics warnings for generic array instantiation that appeared after migration to Java 7
2014-05-20 21:50:16 +02:00
import java.lang.reflect.Array;
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.net.URL;
using concurrency to do base64 encoding in file POST commands
2014-06-20 13:55:15 +02:00
import java.util.AbstractMap;
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
import java.util.Enumeration;
import java.util.Iterator;
import java.util.List;
using concurrency to do base64 encoding in file POST commands
2014-06-20 13:55:15 +02:00
import java.util.Map;
import java.util.concurrent.BlockingQueue;
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
import java.util.concurrent.ConcurrentHashMap;
using concurrency to do base64 encoding in file POST commands
2014-06-20 13:55:15 +02:00
import java.util.concurrent.LinkedBlockingQueue;
implement gzip input handling directly in defaultservlet (making reference to legacy httpdemon obsolete)
2014-04-17 03:20:29 +02:00
import java.util.zip.GZIPInputStream;
re-implementation of file push option in the context of POST http requests. The internal representation of post-arguments is String and therefore not appropriate for byte[] object as submitted by file pushes. Therefore all pushed files are encoded to base64 _after_ uploading with an http form (you do not need to do that encoding yourself) to hand-over the byte[] as string in the post argument. Servlets which read such files must decode the base64 data to get the original byte[] array. This is considered as a temporary solution for file uploads and a proper implementations would need to consider all attributes as handed over as Objects with either String or byte[] Object instances. This would be a major code change and is not done at this time here now. The feature was submitted to realize a feature as pushed with the next commit.
2014-06-12 18:06:22 +02:00
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
import javax.servlet.ReadListener;
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
import javax.servlet.RequestDispatcher;
import javax.servlet.ServletContext;
import javax.servlet.ServletException;
implement gzip input handling directly in defaultservlet (making reference to legacy httpdemon obsolete)
2014-04-17 03:20:29 +02:00
import javax.servlet.ServletInputStream;
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
import javax.servlet.UnavailableException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
implement gzip input handling directly in defaultservlet (making reference to legacy httpdemon obsolete)
2014-04-17 03:20:29 +02:00
import javax.servlet.http.HttpServletRequestWrapper;
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
import javax.servlet.http.HttpServletResponse;
re-implementation of file push option in the context of POST http requests. The internal representation of post-arguments is String and therefore not appropriate for byte[] object as submitted by file pushes. Therefore all pushed files are encoded to base64 _after_ uploading with an http form (you do not need to do that encoding yourself) to hand-over the byte[] as string in the post argument. Servlets which read such files must decode the base64 data to get the original byte[] array. This is considered as a temporary solution for file uploads and a proper implementations would need to consider all attributes as handed over as Objects with either String or byte[] Object instances. This would be a major code change and is not done at this time here now. The feature was submitted to realize a feature as pushed with the next commit.
2014-06-12 18:06:22 +02:00
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
import net.yacy.cora.date.GenericFormatter;
import net.yacy.cora.document.analysis.Classification;
re-implementation of file push option in the context of POST http requests. The internal representation of post-arguments is String and therefore not appropriate for byte[] object as submitted by file pushes. Therefore all pushed files are encoded to base64 _after_ uploading with an http form (you do not need to do that encoding yourself) to hand-over the byte[] as string in the post argument. Servlets which read such files must decode the base64 data to get the original byte[] array. This is considered as a temporary solution for file uploads and a proper implementations would need to consider all attributes as handed over as Objects with either String or byte[] Object instances. This would be a major code change and is not done at this time here now. The feature was submitted to realize a feature as pushed with the next commit.
2014-06-12 18:06:22 +02:00
import net.yacy.cora.order.Base64Order;
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
import net.yacy.cora.protocol.HeaderFramework;
import net.yacy.cora.protocol.RequestHeader;
optimize parse ServerSideInclude
2014-01-22 21:23:32 +01:00
import net.yacy.cora.util.ByteBuffer;
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
import net.yacy.cora.util.ConcurrentLog;
implemented DIGEST authentication, which is for remote login more secure as BASIC were pwd is transmitted near clear text (B64enc). This has some implication as RFC 2617 requires and recommends a password hash MD5(user:realm:pwd) for DIGEST. !!! before activating DIGEST you have to reassign all passwords !!! to allow new calculation of the hash - default authentication is still BASIC - configuration at this time only manually in (DATA/settings) or defaults/web.xml (<auth-method> - the realmname is in defaults/yacy.init adminRealm=YaCy-AdminUI - fyi: the realmname is shown on login screen - changing the realm name invalidates all passwords - but for security you are encouraged to do so (as localhostadmin) - implemented to support both, old hashes for BASIC and new hashes for BASIC and DIGEST - to differentiate old / new hash the in Jetty used hash-prefix "MD5:" is used for new pwd-hashes ( "MD5:hash" )
2014-01-17 00:02:23 +01:00
import net.yacy.data.UserDB.AccessRight;
import net.yacy.data.UserDB.Entry;
refactoring: moving all servlets to the same package (the solr servlet is currently actually a filter which should be changed somehow)
2013-12-23 01:32:29 +01:00
import net.yacy.http.ProxyHandler;
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
import net.yacy.kelondro.util.FileUtils;
import net.yacy.kelondro.util.MemoryControl;
import net.yacy.peers.Seed;
import net.yacy.peers.graphics.EncodedImage;
import net.yacy.peers.operation.yacyBuildProperties;
import net.yacy.search.Switchboard;
import net.yacy.search.SwitchboardConstants;
added html retrieval from alternative DATA/HTDOCS path
2013-12-23 02:06:33 +01:00
import net.yacy.server.http.HTTPDFileHandler;
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
import net.yacy.server.http.TemplateEngine;
import net.yacy.server.serverClassLoader;
import net.yacy.server.serverObjects;
import net.yacy.server.serverSwitch;
import net.yacy.server.servletProperties;
import net.yacy.visualization.RasterPlotter;
re-implementation of file push option in the context of POST http requests. The internal representation of post-arguments is String and therefore not appropriate for byte[] object as submitted by file pushes. Therefore all pushed files are encoded to base64 _after_ uploading with an http form (you do not need to do that encoding yourself) to hand-over the byte[] as string in the post argument. Servlets which read such files must decode the base64 data to get the original byte[] array. This is considered as a temporary solution for file uploads and a proper implementations would need to consider all attributes as handed over as Objects with either String or byte[] Object instances. This would be a major code change and is not done at this time here now. The feature was submitted to realize a feature as pushed with the next commit.
2014-06-12 18:06:22 +02:00
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
import org.apache.commons.fileupload.FileItem;
- add size check to multipart form data handling of YaCyDefaultServlet (same as in HTTPDemon.parseMultipart) - reduce Jetty logging - give build.run a bit more memory (set to YaCy.default 600m from 512m)
2013-10-13 20:56:03 +02:00
import org.apache.commons.fileupload.FileItemFactory;
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
import org.apache.commons.fileupload.disk.DiskFileItemFactory;
import org.apache.commons.fileupload.servlet.ServletFileUpload;
re-implementation of file push option in the context of POST http requests. The internal representation of post-arguments is String and therefore not appropriate for byte[] object as submitted by file pushes. Therefore all pushed files are encoded to base64 _after_ uploading with an http form (you do not need to do that encoding yourself) to hand-over the byte[] as string in the post argument. Servlets which read such files must decode the base64 data to get the original byte[] array. This is considered as a temporary solution for file uploads and a proper implementations would need to consider all attributes as handed over as Objects with either String or byte[] Object instances. This would be a major code change and is not done at this time here now. The feature was submitted to realize a feature as pushed with the next commit.
2014-06-12 18:06:22 +02:00
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
import static org.eclipse.jetty.http.HttpHeader.CONTENT_RANGE;
import static org.eclipse.jetty.http.HttpHeader.IF_MODIFIED_SINCE;
import static org.eclipse.jetty.http.HttpHeader.IF_UNMODIFIED_SINCE;
import static org.eclipse.jetty.http.HttpHeader.REQUEST_RANGE;
import static org.eclipse.jetty.http.HttpMethod.HEAD;
re-implementation of file push option in the context of POST http requests. The internal representation of post-arguments is String and therefore not appropriate for byte[] object as submitted by file pushes. Therefore all pushed files are encoded to base64 _after_ uploading with an http form (you do not need to do that encoding yourself) to hand-over the byte[] as string in the post argument. Servlets which read such files must decode the base64 data to get the original byte[] array. This is considered as a temporary solution for file uploads and a proper implementations would need to consider all attributes as handed over as Objects with either String or byte[] Object instances. This would be a major code change and is not done at this time here now. The feature was submitted to realize a feature as pushed with the next commit.
2014-06-12 18:06:22 +02:00
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
import org.eclipse.jetty.http.MimeTypes;
re-implementation of file push option in the context of POST http requests. The internal representation of post-arguments is String and therefore not appropriate for byte[] object as submitted by file pushes. Therefore all pushed files are encoded to base64 _after_ uploading with an http form (you do not need to do that encoding yourself) to hand-over the byte[] as string in the post argument. Servlets which read such files must decode the base64 data to get the original byte[] array. This is considered as a temporary solution for file uploads and a proper implementations would need to consider all attributes as handed over as Objects with either String or byte[] Object instances. This would be a major code change and is not done at this time here now. The feature was submitted to realize a feature as pushed with the next commit.
2014-06-12 18:06:22 +02:00
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
import static org.eclipse.jetty.http.MimeTypes.Type.TEXT_HTML;
import static org.eclipse.jetty.http.MimeTypes.Type.TEXT_HTML_UTF_8;
re-implementation of file push option in the context of POST http requests. The internal representation of post-arguments is String and therefore not appropriate for byte[] object as submitted by file pushes. Therefore all pushed files are encoded to base64 _after_ uploading with an http form (you do not need to do that encoding yourself) to hand-over the byte[] as string in the post argument. Servlets which read such files must decode the base64 data to get the original byte[] array. This is considered as a temporary solution for file uploads and a proper implementations would need to consider all attributes as handed over as Objects with either String or byte[] Object instances. This would be a major code change and is not done at this time here now. The feature was submitted to realize a feature as pushed with the next commit.
2014-06-12 18:06:22 +02:00
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
import org.eclipse.jetty.io.WriterOutputStream;
import org.eclipse.jetty.server.InclusiveByteRange;
import org.eclipse.jetty.util.MultiPartOutputStream;
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
import org.eclipse.jetty.util.URIUtil;
import org.eclipse.jetty.util.resource.Resource;
/**
downgrade to Jetty 8 to assure support of JRE 1.6 - introduce a YaCyHttp interface to modulize/separate http server - adjust the Jetty version specific implementation part (in package net.yacy.http) - putting the version specific code in classes starting with Jetty8xxxx - moved existing Jetty9xxx implementation into a test class (to keep the code) - adjust build to the changed jars - make use of the introduced YaCyHttpServer interface in related htroot servlets - adjust other test cases/classes
2013-10-09 00:40:48 +02:00
* YaCyDefaultServlet based on Jetty DefaultServlet.java
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
* handles static files and the YaCy servlets.
downgrade to Jetty 8 to assure support of JRE 1.6 - introduce a YaCyHttp interface to modulize/separate http server - adjust the Jetty version specific implementation part (in package net.yacy.http) - putting the version specific code in classes starting with Jetty8xxxx - moved existing Jetty9xxx implementation into a test class (to keep the code) - adjust build to the changed jars - make use of the introduced YaCyHttpServer interface in related htroot servlets - adjust other test cases/classes
2013-10-09 00:40:48 +02:00
*
* This interface impements the YaCy specific and standard Servlet routines
* which should not have a dependency on the implemented Jetty version.
* The Jetty version specific code is moved to the Jetty8HttpServerImpl.java implementation
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
*/
/**
* The default servlet. This servlet, normally mapped to /, provides the
* handling for static content, OPTION and TRACE methods for the context. The
* following initParameters are supported, these can be set either on the
* servlet itself or as ServletContext initParameters :
* <PRE>
* acceptRanges If true, range requests and responses are
* supported
*
* dirAllowed If true, directory listings are returned if no
* welcome file is found. Else 403 Forbidden.
- add welcome file (index.html) support to YaCyDefaultServlet - change SolrServlet default search field (&df) to text_t
2013-09-29 03:34:00 +02:00
*
* welcomeFile name of the welcome file (default is "index.html", "welcome.html")
update Info.plist small DefaultServlet refactoring
2013-10-12 23:01:14 +02:00
*
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
* resourceBase Set to replace the context resource base
*
* pathInfoOnly If true, only the path info will be applied to the resourceBase
*
* </PRE>
*/
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
public class YaCyDefaultServlet extends HttpServlet {
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
fix not necessary use of DigestURL
2013-09-23 03:05:09 +02:00
private static final long serialVersionUID = 4900000000000001110L;
downgrade to Jetty 8 to assure support of JRE 1.6 - introduce a YaCyHttp interface to modulize/separate http server - adjust the Jetty version specific implementation part (in package net.yacy.http) - putting the version specific code in classes starting with Jetty8xxxx - moved existing Jetty9xxx implementation into a test class (to keep the code) - adjust build to the changed jars - make use of the introduced YaCyHttpServer interface in related htroot servlets - adjust other test cases/classes
2013-10-09 00:40:48 +02:00
protected ServletContext _servletContext;
protected boolean _acceptRanges = true;
protected boolean _dirAllowed = true;
protected boolean _pathInfoOnly = false;
protected Resource _resourceBase;
protected MimeTypes _mimeTypes;
protected String[] _welcomes;
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
downgrade to Jetty 8 to assure support of JRE 1.6 - introduce a YaCyHttp interface to modulize/separate http server - adjust the Jetty version specific implementation part (in package net.yacy.http) - putting the version specific code in classes starting with Jetty8xxxx - moved existing Jetty9xxx implementation into a test class (to keep the code) - adjust build to the changed jars - make use of the introduced YaCyHttpServer interface in related htroot servlets - adjust other test cases/classes
2013-10-09 00:40:48 +02:00
protected File _htLocalePath;
protected File _htDocsPath;
protected static final serverClassLoader provider = new serverClassLoader(/*this.getClass().getClassLoader()*/);
protected ConcurrentHashMap<File, SoftReference<Method>> templateMethodCache = null;
- add size check to multipart form data handling of YaCyDefaultServlet (same as in HTTPDemon.parseMultipart) - reduce Jetty logging - give build.run a bit more memory (set to YaCy.default 600m from 512m)
2013-10-13 20:56:03 +02:00
// settings for multipart/form-data
protected static final File TMPDIR = new File(System.getProperty("java.io.tmpdir"));
increased fixed limit for http POST request sizes to 100MB
2014-06-26 11:58:07 +02:00
protected static final int SIZE_FILE_THRESHOLD = 100 * 1024 * 1024; // 100 MB is a lot but appropriate for multi-document pushed using the push_p.json servlet
- add size check to multipart form data handling of YaCyDefaultServlet (same as in HTTPDemon.parseMultipart) - reduce Jetty logging - give build.run a bit more memory (set to YaCy.default 600m from 512m)
2013-10-13 20:56:03 +02:00
protected static final FileItemFactory DISK_FILE_ITEM_FACTORY = new DiskFileItemFactory(SIZE_FILE_THRESHOLD, TMPDIR);
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
/* ------------------------------------------------------------ */
@Override
use more YaCy HeaderFramework constants (instead of Jetty's)
2013-09-22 04:23:42 +02:00
public void init() throws UnavailableException {
added tutorial mode - some menu items will only appear if you 'qualify' for them. Thus, the first-time user will only see four menu items. The other items will unfold as the user interacts.
2014-04-02 02:33:17 +02:00
Switchboard sb = Switchboard.getSwitchboard();
_htDocsPath = sb.htDocsPath;
_htLocalePath = sb.getDataPath("locale.translated_html", "DATA/LOCALE/htroot");
use more YaCy HeaderFramework constants (instead of Jetty's)
2013-09-22 04:23:42 +02:00
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
_servletContext = getServletContext();
_mimeTypes = new MimeTypes();
- add welcome file (index.html) support to YaCyDefaultServlet - change SolrServlet default search field (&df) to text_t
2013-09-29 03:34:00 +02:00
String tmpstr = this.getServletContext().getInitParameter("welcomeFile");
if (tmpstr == null) {
- use default files from yacy.init property "defaultFiles" if no jetty-configuration is given for default files. - fix a problem with default paths if no path is given (i.e. http://localhost:8090 instead of http://localhost:8090/). Without this patch the path was resolved automatically to http://localhost:8090//
2013-12-26 23:59:04 +01:00
_welcomes = HTTPDFileHandler.defaultFiles;
- add welcome file (index.html) support to YaCyDefaultServlet - change SolrServlet default search field (&df) to text_t
2013-09-29 03:34:00 +02:00
} else {
_welcomes = new String[]{tmpstr,"index.html"};
}
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
_acceptRanges = getInitBoolean("acceptRanges", _acceptRanges);
_dirAllowed = getInitBoolean("dirAllowed", _dirAllowed);
_pathInfoOnly = getInitBoolean("pathInfoOnly", _pathInfoOnly);
- disable Jetty servlet defaultUseCache (prevent double caching) - include short memory status check for class cache in DefaultServlet - remove obsolete Resource interface for Jetty8YaCyDefaultServlet
2013-11-18 03:15:45 +01:00
Resource.setDefaultUseCaches(false); // caching is handled internally (prevent double caching)
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
String rb = getInitParameter("resourceBase");
remove unused init parameter from DefaultServlet - remove "RelativeResourceBase" parameter
2013-12-15 23:39:19 +01:00
try {
if (rb != null) {
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
_resourceBase = Resource.newResource(rb);
remove unused init parameter from DefaultServlet - remove "RelativeResourceBase" parameter
2013-12-15 23:39:19 +01:00
} else {
added tutorial mode - some menu items will only appear if you 'qualify' for them. Thus, the first-time user will only see four menu items. The other items will unfold as the user interacts.
2014-04-02 02:33:17 +02:00
_resourceBase = Resource.newResource(sb.getConfig(SwitchboardConstants.HTROOT_PATH, SwitchboardConstants.HTROOT_PATH_DEFAULT)); //default
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
}
remove unused init parameter from DefaultServlet - remove "RelativeResourceBase" parameter
2013-12-15 23:39:19 +01:00
} catch (IOException e) {
ConcurrentLog.severe("FILEHANDLER", "YaCyDefaultServlet: resource base (htRootPath) missing");
ConcurrentLog.logException(e);
throw new UnavailableException(e.toString());
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
}
reduce logging (by assigning logger to existing logger) - small additional cleanups
2013-10-30 00:51:04 +01:00
if (ConcurrentLog.isFine("FILEHANDLER")) {
ConcurrentLog.fine("FILEHANDLER","YaCyDefaultServlet: resource base = " + _resourceBase);
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
}
templateMethodCache = new ConcurrentHashMap<File, SoftReference<Method>>();
}
/* ------------------------------------------------------------ */
downgrade to Jetty 8 to assure support of JRE 1.6 - introduce a YaCyHttp interface to modulize/separate http server - adjust the Jetty version specific implementation part (in package net.yacy.http) - putting the version specific code in classes starting with Jetty8xxxx - moved existing Jetty9xxx implementation into a test class (to keep the code) - adjust build to the changed jars - make use of the introduced YaCyHttpServer interface in related htroot servlets - adjust other test cases/classes
2013-10-09 00:40:48 +02:00
protected boolean getInitBoolean(String name, boolean dft) {
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
String value = getInitParameter(name);
if (value == null || value.length() == 0) {
return dft;
}
return (value.startsWith("t")
|| value.startsWith("T")
|| value.startsWith("y")
|| value.startsWith("Y")
|| value.startsWith("1"));
}
/* ------------------------------------------------------------ */
/**
* get Resource to serve. Map a path to a resource. The default
* implementation calls HttpContext.getResource but derived servlets may
* provide their own mapping.
*
* @param pathInContext The path to find a resource for.
* @return The resource to serve.
*/
public Resource getResource(String pathInContext) {
Resource r = null;
try {
if (_resourceBase != null) {
r = _resourceBase.addPath(pathInContext);
} else {
URL u = _servletContext.getResource(pathInContext);
r = Resource.newResource(u);
}
reduce logging (by assigning logger to existing logger) - small additional cleanups
2013-10-30 00:51:04 +01:00
if (ConcurrentLog.isFine("FILEHANDLER")) {
ConcurrentLog.fine("FILEHANDLER","YaCyDefaultServlet: Resource " + pathInContext + "=" + r);
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
}
} catch (IOException e) {
// ConcurrentLog.logException(e);
}
return r;
}
/* ------------------------------------------------------------ */
downgrade to Jetty 8 to assure support of JRE 1.6 - introduce a YaCyHttp interface to modulize/separate http server - adjust the Jetty version specific implementation part (in package net.yacy.http) - putting the version specific code in classes starting with Jetty8xxxx - moved existing Jetty9xxx implementation into a test class (to keep the code) - adjust build to the changed jars - make use of the introduced YaCyHttpServer interface in related htroot servlets - adjust other test cases/classes
2013-10-09 00:40:48 +02:00
protected boolean hasDefinedRange(Enumeration<String> reqRanges) {
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
return (reqRanges != null && reqRanges.hasMoreElements());
}
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
/* ------------------------------------------------------------ */
@Override
protected void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
String servletPath;
simulate Authorization cookie for yacy servlet header
2014-01-10 19:31:36 +01:00
String pathInfo;
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
Enumeration<String> reqRanges = null;
boolean included = request.getAttribute(RequestDispatcher.INCLUDE_REQUEST_URI) != null;
if (included) {
servletPath = (String) request.getAttribute(RequestDispatcher.INCLUDE_SERVLET_PATH);
pathInfo = (String) request.getAttribute(RequestDispatcher.INCLUDE_PATH_INFO);
if (servletPath == null) {
servletPath = request.getServletPath();
pathInfo = request.getPathInfo();
}
} else {
servletPath = _pathInfoOnly ? "/" : request.getServletPath();
pathInfo = request.getPathInfo();
// Is this a Range request?
reqRanges = request.getHeaders(HeaderFramework.RANGE);
if (!hasDefinedRange(reqRanges)) {
reqRanges = null;
}
}
String pathInContext = URIUtil.addPaths(servletPath, pathInfo);
boolean endsWithSlash = (pathInfo == null ? request.getServletPath() : pathInfo).endsWith(URIUtil.SLASH);
// Find the resource
Resource resource = null;
try {
// Look for a class resource
boolean hasClass = false;
if (reqRanges == null && !endsWithSlash) {
final int p = pathInContext.lastIndexOf('.');
if (p >= 0) {
String pathofClass = pathInContext.substring(0, p) + ".class";
Resource classresource = _resourceBase.addPath(pathofClass);
// Does a class resource exist?
if (classresource != null && classresource.exists() && !classresource.isDirectory()) {
hasClass = true;
}
}
}
added html retrieval from alternative DATA/HTDOCS path
2013-12-23 02:06:33 +01:00
// find resource
resource = getResource(pathInContext);
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
added html retrieval from alternative DATA/HTDOCS path
2013-12-23 02:06:33 +01:00
if (!hasClass && (resource == null || !resource.exists()) && !pathInContext.contains("..")) {
// try to get this in the alternative htDocsPath
resource = Resource.newResource(new File(HTTPDFileHandler.htDocsPath, pathInContext));
}
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
if (ConcurrentLog.isFine("FILEHANDLER")) {
ConcurrentLog.fine("FILEHANDLER","YaCyDefaultServlet: uri=" + request.getRequestURI() + " resource=" + resource);
}
added html retrieval from alternative DATA/HTDOCS path
2013-12-23 02:06:33 +01:00
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
// Handle resource
if (!hasClass && (resource == null || !resource.exists())) {
if (included) {
throw new FileNotFoundException("!" + pathInContext);
}
response.sendError(HttpServletResponse.SC_NOT_FOUND);
} else if (!resource.isDirectory()) {
if (endsWithSlash && pathInContext.length() > 1) {
String q = request.getQueryString();
pathInContext = pathInContext.substring(0, pathInContext.length() - 1);
if (q != null && q.length() != 0) {
pathInContext += "?" + q;
}
response.sendRedirect(response.encodeRedirectURL(URIUtil.addPaths(_servletContext.getContextPath(), pathInContext)));
} else {
if (hasClass) { // this is a YaCy servlet, handle the template
handleTemplate(pathInfo, request, response);
} else {
if (included || passConditionalHeaders(request, response, resource)) {
sendData(request, response, included, resource, reqRanges);
}
}
}
} else { // resource is directory
String welcome;
- use default files from yacy.init property "defaultFiles" if no jetty-configuration is given for default files. - fix a problem with default paths if no path is given (i.e. http://localhost:8090 instead of http://localhost:8090/). Without this patch the path was resolved automatically to http://localhost:8090//
2013-12-26 23:59:04 +01:00
if (!endsWithSlash) {
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
StringBuffer buf = request.getRequestURL();
synchronized (buf) {
int param = buf.lastIndexOf(";");
if (param < 0) {
buf.append('/');
} else {
buf.insert(param, '/');
}
String q = request.getQueryString();
if (q != null && q.length() != 0) {
buf.append('?');
buf.append(q);
}
response.setContentLength(0);
response.sendRedirect(response.encodeRedirectURL(buf.toString()));
}
} // else look for a welcome file
else if (null != (welcome = getWelcomeFile(pathInContext))) {
ConcurrentLog.fine("FILEHANDLER","welcome={}" + welcome);
// Forward to the index
RequestDispatcher dispatcher = request.getRequestDispatcher(welcome);
if (dispatcher != null) {
if (included) {
dispatcher.include(request, response);
} else {
dispatcher.forward(request, response);
}
}
} else {
if (included || passConditionalHeaders(request, response, resource)) {
sendDirectory(request, response, resource, pathInContext);
}
}
}
} catch (IllegalArgumentException e) {
ConcurrentLog.logException(e);
if (!response.isCommitted()) {
response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR, e.getMessage());
}
} finally {
if (resource != null) {
- fix typo in Message_p (defaultpath) - use more existing switchboardconstants for getproperties - replace depriciated call defaultservlet
2014-06-01 00:20:25 +02:00
resource.close();
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
}
}
}
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
/* ------------------------------------------------------------ */
@Override
protected void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
doGet(request, response);
}
/* ------------------------------------------------------------ */
/* (non-Javadoc)
* @see javax.servlet.http.HttpServlet#doTrace(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse)
*/
@Override
protected void doTrace(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
resp.sendError(HttpServletResponse.SC_METHOD_NOT_ALLOWED);
}
/* ------------------------------------------------------------ */
@Override
protected void doOptions(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {
resp.setHeader("Allow", "GET,HEAD,POST,OPTIONS");
}
- add welcome file (index.html) support to YaCyDefaultServlet - change SolrServlet default search field (&df) to text_t
2013-09-29 03:34:00 +02:00
/* ------------------------------------------------------------ */
/**
* Finds a matching welcome file for the supplied path.
* The filename to look is set as servlet context init parameter
* default is "index.html"
fix servlet class exist check to use default path only (in Jetty8YaCyDefaultServlet) - del redundant doget code in yacydefaultservlet - small declaration code opts - del obsolete libt/proxyservlet.java
2013-11-03 02:26:00 +01:00
* @param pathInContext path in context
- add welcome file (index.html) support to YaCyDefaultServlet - change SolrServlet default search field (&df) to text_t
2013-09-29 03:34:00 +02:00
* @return The path of the matching welcome file in context or null.
*/
downgrade to Jetty 8 to assure support of JRE 1.6 - introduce a YaCyHttp interface to modulize/separate http server - adjust the Jetty version specific implementation part (in package net.yacy.http) - putting the version specific code in classes starting with Jetty8xxxx - moved existing Jetty9xxx implementation into a test class (to keep the code) - adjust build to the changed jars - make use of the introduced YaCyHttpServer interface in related htroot servlets - adjust other test cases/classes
2013-10-09 00:40:48 +02:00
protected String getWelcomeFile(String pathInContext) {
- add welcome file (index.html) support to YaCyDefaultServlet - change SolrServlet default search field (&df) to text_t
2013-09-29 03:34:00 +02:00
if (_welcomes == null) {
return null;
}
fix servlet class exist check to use default path only (in Jetty8YaCyDefaultServlet) - del redundant doget code in yacydefaultservlet - small declaration code opts - del obsolete libt/proxyservlet.java
2013-11-03 02:26:00 +01:00
for (String _welcome : _welcomes) {
String welcome_in_context = URIUtil.addPaths(pathInContext, _welcome);
- add welcome file (index.html) support to YaCyDefaultServlet - change SolrServlet default search field (&df) to text_t
2013-09-29 03:34:00 +02:00
Resource welcome = getResource(welcome_in_context);
if (welcome != null && welcome.exists()) {
fix servlet class exist check to use default path only (in Jetty8YaCyDefaultServlet) - del redundant doget code in yacydefaultservlet - small declaration code opts - del obsolete libt/proxyservlet.java
2013-11-03 02:26:00 +01:00
return _welcome;
- add welcome file (index.html) support to YaCyDefaultServlet - change SolrServlet default search field (&df) to text_t
2013-09-29 03:34:00 +02:00
}
}
return null;
}
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
/* ------------------------------------------------------------ */
/* Check modification date headers.
move conditionalHeader to DefaultServlet - by removing Jetty specific implementation detail
2013-12-08 00:56:45 +01:00
* send a 304 response instead of content if not modified since
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
*/
move conditionalHeader to DefaultServlet - by removing Jetty specific implementation detail
2013-12-08 00:56:45 +01:00
protected boolean passConditionalHeaders(HttpServletRequest request, HttpServletResponse response, Resource resource)
throws IOException {
try {
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
if (!request.getMethod().equals(HEAD.asString())) {
move conditionalHeader to DefaultServlet - by removing Jetty specific implementation detail
2013-12-08 00:56:45 +01:00
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
String ifms = request.getHeader(IF_MODIFIED_SINCE.asString());
move conditionalHeader to DefaultServlet - by removing Jetty specific implementation detail
2013-12-08 00:56:45 +01:00
if (ifms != null) {
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
long ifmsl = request.getDateHeader(IF_MODIFIED_SINCE.asString());
move conditionalHeader to DefaultServlet - by removing Jetty specific implementation detail
2013-12-08 00:56:45 +01:00
if (ifmsl != -1) {
if (resource.lastModified() / 1000 <= ifmsl / 1000) {
response.reset();
response.setStatus(HttpServletResponse.SC_NOT_MODIFIED);
response.flushBuffer();
return false;
}
}
}
// Parse the if[un]modified dates and compare to resource
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
long date = request.getDateHeader(IF_UNMODIFIED_SINCE.asString());
move conditionalHeader to DefaultServlet - by removing Jetty specific implementation detail
2013-12-08 00:56:45 +01:00
if (date != -1) {
if (resource.lastModified() / 1000 > date / 1000) {
response.sendError(HttpServletResponse.SC_PRECONDITION_FAILED);
return false;
}
}
}
} catch (IllegalArgumentException iae) {
if (!response.isCommitted()) {
response.sendError(HttpServletResponse.SC_BAD_REQUEST, iae.getMessage());
}
throw iae;
}
return true;
}
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
/* ------------------------------------------------------------------- */
protected void sendDirectory(HttpServletRequest request,
HttpServletResponse response,
Resource resource,
String pathInContext)
throws IOException {
if (!_dirAllowed) {
response.sendError(HttpServletResponse.SC_FORBIDDEN);
return;
}
fix servlet class exist check to use default path only (in Jetty8YaCyDefaultServlet) - del redundant doget code in yacydefaultservlet - small declaration code opts - del obsolete libt/proxyservlet.java
2013-11-03 02:26:00 +01:00
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
String base = URIUtil.addPaths(request.getRequestURI(), URIUtil.SLASH);
String dir = resource.getListHTML(base, pathInContext.length() > 1);
if (dir == null) {
use writeOptionHeaders with ServletResponse parameter only
2013-11-17 00:02:08 +01:00
response.sendError(HttpServletResponse.SC_FORBIDDEN, "No directory");
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
return;
}
fix servlet class exist check to use default path only (in Jetty8YaCyDefaultServlet) - del redundant doget code in yacydefaultservlet - small declaration code opts - del obsolete libt/proxyservlet.java
2013-11-03 02:26:00 +01:00
byte[] data = dir.getBytes("UTF-8");
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
response.setContentType(TEXT_HTML_UTF_8.asString());
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
response.setContentLength(data.length);
response.getOutputStream().write(data);
}
/* ------------------------------------------------------------ */
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
/**
* send static content
*
* @param request
* @param response
* @param include is a include file (send without changing/adding headers)
* @param resource the static content
* @param reqRanges
* @throws IOException
*/
protected void sendData(HttpServletRequest request,
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
HttpServletResponse response,
boolean include,
Resource resource,
removed unused anomichttpd code after migration to jetty
2013-12-23 01:23:40 +01:00
Enumeration<String> reqRanges)
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
throws IOException {
final long content_length = resource.length();
// Get the output stream (or writer)
OutputStream out;
try {
out = response.getOutputStream();
} catch (IllegalStateException e) {
out = new WriterOutputStream(response.getWriter());
}
if (reqRanges == null || !reqRanges.hasMoreElements() || content_length < 0) {
// if there were no ranges, send entire entity
if (include) {
resource.writeTo(out, 0, content_length);
} else {
writeHeaders(response, resource, content_length);
resource.writeTo(out, 0, content_length);
}
} else {
// Parse the satisfiable ranges
removed unused anomichttpd code after migration to jetty
2013-12-23 01:23:40 +01:00
List<?> ranges = InclusiveByteRange.satisfiableRanges(reqRanges, content_length);
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
// if there are no satisfiable ranges, send 416 response
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
if (ranges == null || ranges.isEmpty()) {
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
writeHeaders(response, resource, content_length);
response.setStatus(HttpServletResponse.SC_REQUESTED_RANGE_NOT_SATISFIABLE);
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
response.setHeader(CONTENT_RANGE.asString(),
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
InclusiveByteRange.to416HeaderRangeString(content_length));
resource.writeTo(out, 0, content_length);
added close to fix possible resource leak warning
2014-03-28 14:34:49 +01:00
out.close();
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
return;
}
// if there is only a single valid range (must be satisfiable
// since were here now), send that range with a 216 response
if (ranges.size() == 1) {
InclusiveByteRange singleSatisfiableRange =
(InclusiveByteRange) ranges.get(0);
long singleLength = singleSatisfiableRange.getSize(content_length);
writeHeaders(response, resource, singleLength);
response.setStatus(HttpServletResponse.SC_PARTIAL_CONTENT);
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
response.setHeader(CONTENT_RANGE.asString(),
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
singleSatisfiableRange.toHeaderRangeString(content_length));
resource.writeTo(out, singleSatisfiableRange.getFirst(content_length), singleLength);
added close to fix possible resource leak warning
2014-03-28 14:34:49 +01:00
out.close();
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
return;
}
// multiple non-overlapping valid ranges cause a multipart
// 216 response which does not require an overall
// content-length header
//
writeHeaders(response, resource, -1);
String mimetype = response.getContentType();
if (mimetype == null) {
ConcurrentLog.warn("FILEHANDLER","YaCyDefaultServlet: Unknown mimetype for " + request.getRequestURI());
}
MultiPartOutputStream multi = new MultiPartOutputStream(out);
response.setStatus(HttpServletResponse.SC_PARTIAL_CONTENT);
// If the request has a "Request-Range" header then we need to
// send an old style multipart/x-byteranges Content-Type. This
// keeps Netscape and acrobat happy. This is what Apache does.
String ctp;
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
if (request.getHeader(REQUEST_RANGE.asString()) != null) {
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
ctp = "multipart/x-byteranges; boundary=";
} else {
ctp = "multipart/byteranges; boundary=";
}
response.setContentType(ctp + multi.getBoundary());
InputStream in = resource.getInputStream();
long pos = 0;
// calculate the content-length
int length = 0;
String[] header = new String[ranges.size()];
for (int i = 0; i < ranges.size(); i++) {
InclusiveByteRange ibr = (InclusiveByteRange) ranges.get(i);
header[i] = ibr.toHeaderRangeString(content_length);
length +=
((i > 0) ? 2 : 0)
+ 2 + multi.getBoundary().length() + 2
+ (mimetype == null ? 0 : HeaderFramework.CONTENT_TYPE.length() + 2 + mimetype.length()) + 2
+ HeaderFramework.CONTENT_RANGE.length() + 2 + header[i].length() + 2
+ 2
+ (ibr.getLast(content_length) - ibr.getFirst(content_length)) + 1;
}
length += 2 + 2 + multi.getBoundary().length() + 2 + 2;
response.setContentLength(length);
for (int i = 0; i < ranges.size(); i++) {
InclusiveByteRange ibr = (InclusiveByteRange) ranges.get(i);
multi.startPart(mimetype, new String[]{HeaderFramework.CONTENT_RANGE + ": " + header[i]});
long start = ibr.getFirst(content_length);
long size = ibr.getSize(content_length);
if (in != null) {
// Handle non cached resource
if (start < pos) {
in.close();
in = resource.getInputStream();
pos = 0;
}
if (pos < start) {
in.skip(start - pos);
pos = start;
}
FileUtils.copy(in, multi, size);
pos += size;
} else // Handle cached resource
{
(resource).writeTo(multi, start, size);
}
}
added tutorial mode - some menu items will only appear if you 'qualify' for them. Thus, the first-time user will only see four menu items. The other items will unfold as the user interacts.
2014-04-02 02:33:17 +02:00
if (in != null) in.close();
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
multi.close();
}
}
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
/* ------------------------------------------------------------ */
remove unused options and attributes from DefaultServlet cleanup obsolete class files
2013-11-24 20:11:39 +01:00
protected void writeHeaders(HttpServletResponse response, Resource resource, long count) {
if (response.getContentType() == null) {
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
final String extensionmime;
fix NPE on mime detection of unknown file extension
2013-11-29 23:23:47 +01:00
if ((extensionmime = _mimeTypes.getMimeByExtension(resource.getName())) != null) {
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
response.setContentType(extensionmime);
fix NPE on mime detection of unknown file extension
2013-11-29 23:23:47 +01:00
}
move writeHeaders from Jetty8 servlet to YaCyDefaultServlet - after removing Jetty server dependency (of Response using HttpServletResponse only)
2013-11-07 00:32:21 +01:00
}
remove unused options and attributes from DefaultServlet cleanup obsolete class files
2013-11-24 20:11:39 +01:00
long lml = resource.lastModified();
move writeHeaders from Jetty8 servlet to YaCyDefaultServlet - after removing Jetty server dependency (of Response using HttpServletResponse only)
2013-11-07 00:32:21 +01:00
if (lml >= 0) {
response.setDateHeader(HeaderFramework.LAST_MODIFIED, lml);
}
if (count != -1) {
if (count < Integer.MAX_VALUE) {
response.setContentLength((int) count);
} else {
response.setHeader(HeaderFramework.CONTENT_LENGTH, Long.toString(count));
}
}
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
if (_acceptRanges) {
use more YaCy HeaderFramework constants (instead of Jetty's)
2013-09-22 04:23:42 +02:00
response.setHeader(HeaderFramework.ACCEPT_RANGES, "bytes");
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
}
}
remove unused options and attributes from DefaultServlet cleanup obsolete class files
2013-11-24 20:11:39 +01:00
downgrade to Jetty 8 to assure support of JRE 1.6 - introduce a YaCyHttp interface to modulize/separate http server - adjust the Jetty version specific implementation part (in package net.yacy.http) - putting the version specific code in classes starting with Jetty8xxxx - moved existing Jetty9xxx implementation into a test class (to keep the code) - adjust build to the changed jars - make use of the introduced YaCyHttpServer interface in related htroot servlets - adjust other test cases/classes
2013-10-09 00:40:48 +02:00
protected Object invokeServlet(final File targetClass, final RequestHeader request, final serverObjects args) throws IllegalArgumentException, IllegalAccessException, InvocationTargetException {
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
return rewriteMethod(targetClass).invoke(null, new Object[]{request, args, Switchboard.getSwitchboard()}); // add switchboard
}
downgrade to Jetty 8 to assure support of JRE 1.6 - introduce a YaCyHttp interface to modulize/separate http server - adjust the Jetty version specific implementation part (in package net.yacy.http) - putting the version specific code in classes starting with Jetty8xxxx - moved existing Jetty9xxx implementation into a test class (to keep the code) - adjust build to the changed jars - make use of the introduced YaCyHttpServer interface in related htroot servlets - adjust other test cases/classes
2013-10-09 00:40:48 +02:00
protected RequestHeader generateLegacyRequestHeader(HttpServletRequest request, String target, String targetExt) {
add YaCyProxyServlet to handle /proxy.html?url=proxyurl - based on Jetty ProxyServlet - at this time use existing HTTPD ProxyHandler for url rewrite - add jetty-client jar (dependency in Jetty ProxyServlet) reuse ProxyHandler.convertHeaderFromJetty in YaCyDefaultServlet
2013-10-27 05:04:24 +01:00
RequestHeader legacyRequestHeader = ProxyHandler.convertHeaderFromJetty(request);
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
legacyRequestHeader.put(HeaderFramework.CONNECTION_PROP_CLIENTIP, request.getRemoteAddr());
legacyRequestHeader.put(HeaderFramework.CONNECTION_PROP_PATH, target);
legacyRequestHeader.put(HeaderFramework.CONNECTION_PROP_EXT, targetExt);
added tutorial mode - some menu items will only appear if you 'qualify' for them. Thus, the first-time user will only see four menu items. The other items will unfold as the user interacts.
2014-04-02 02:33:17 +02:00
Switchboard sb = Switchboard.getSwitchboard();
implemented DIGEST authentication, which is for remote login more secure as BASIC were pwd is transmitted near clear text (B64enc). This has some implication as RFC 2617 requires and recommends a password hash MD5(user:realm:pwd) for DIGEST. !!! before activating DIGEST you have to reassign all passwords !!! to allow new calculation of the hash - default authentication is still BASIC - configuration at this time only manually in (DATA/settings) or defaults/web.xml (<auth-method> - the realmname is in defaults/yacy.init adminRealm=YaCy-AdminUI - fyi: the realmname is shown on login screen - changing the realm name invalidates all passwords - but for security you are encouraged to do so (as localhostadmin) - implemented to support both, old hashes for BASIC and new hashes for BASIC and DIGEST - to differentiate old / new hash the in Jetty used hash-prefix "MD5:" is used for new pwd-hashes ( "MD5:hash" )
2014-01-17 00:02:23 +01:00
if (legacyRequestHeader.containsKey(RequestHeader.AUTHORIZATION)) {
if (HttpServletRequest.BASIC_AUTH.equalsIgnoreCase(request.getAuthType())) {
} else {
// handle DIGEST auth for legacyHeader (create username:md5pwdhash
if (request.getUserPrincipal() != null) {
String userpassEncoded = request.getHeader(RequestHeader.AUTHORIZATION); // e.g. "Basic AdminMD5hash"
if (userpassEncoded != null) {
added tutorial mode - some menu items will only appear if you 'qualify' for them. Thus, the first-time user will only see four menu items. The other items will unfold as the user interacts.
2014-04-02 02:33:17 +02:00
if (request.isUserInRole(AccessRight.ADMIN_RIGHT.toString()) && !sb.getConfig(SwitchboardConstants.ADMIN_ACCOUNT_B64MD5,"").isEmpty()) {
implemented DIGEST authentication, which is for remote login more secure as BASIC were pwd is transmitted near clear text (B64enc). This has some implication as RFC 2617 requires and recommends a password hash MD5(user:realm:pwd) for DIGEST. !!! before activating DIGEST you have to reassign all passwords !!! to allow new calculation of the hash - default authentication is still BASIC - configuration at this time only manually in (DATA/settings) or defaults/web.xml (<auth-method> - the realmname is in defaults/yacy.init adminRealm=YaCy-AdminUI - fyi: the realmname is shown on login screen - changing the realm name invalidates all passwords - but for security you are encouraged to do so (as localhostadmin) - implemented to support both, old hashes for BASIC and new hashes for BASIC and DIGEST - to differentiate old / new hash the in Jetty used hash-prefix "MD5:" is used for new pwd-hashes ( "MD5:hash" )
2014-01-17 00:02:23 +01:00
// fake admin authentication for legacyRequestHeader (as e.g. DIGEST is not supported by legacyRequestHeader)
legacyRequestHeader.put(RequestHeader.AUTHORIZATION, HttpServletRequest.BASIC_AUTH + " "
added tutorial mode - some menu items will only appear if you 'qualify' for them. Thus, the first-time user will only see four menu items. The other items will unfold as the user interacts.
2014-04-02 02:33:17 +02:00
+ sb.getConfig(SwitchboardConstants.ADMIN_ACCOUNT_B64MD5, ""));
implemented DIGEST authentication, which is for remote login more secure as BASIC were pwd is transmitted near clear text (B64enc). This has some implication as RFC 2617 requires and recommends a password hash MD5(user:realm:pwd) for DIGEST. !!! before activating DIGEST you have to reassign all passwords !!! to allow new calculation of the hash - default authentication is still BASIC - configuration at this time only manually in (DATA/settings) or defaults/web.xml (<auth-method> - the realmname is in defaults/yacy.init adminRealm=YaCy-AdminUI - fyi: the realmname is shown on login screen - changing the realm name invalidates all passwords - but for security you are encouraged to do so (as localhostadmin) - implemented to support both, old hashes for BASIC and new hashes for BASIC and DIGEST - to differentiate old / new hash the in Jetty used hash-prefix "MD5:" is used for new pwd-hashes ( "MD5:hash" )
2014-01-17 00:02:23 +01:00
} else {
// fake Basic auth header for Digest auth (Basic username:md5pwdhash)
String username = request.getRemoteUser();
added tutorial mode - some menu items will only appear if you 'qualify' for them. Thus, the first-time user will only see four menu items. The other items will unfold as the user interacts.
2014-04-02 02:33:17 +02:00
Entry user = sb.userDB.getEntry(username);
implemented DIGEST authentication, which is for remote login more secure as BASIC were pwd is transmitted near clear text (B64enc). This has some implication as RFC 2617 requires and recommends a password hash MD5(user:realm:pwd) for DIGEST. !!! before activating DIGEST you have to reassign all passwords !!! to allow new calculation of the hash - default authentication is still BASIC - configuration at this time only manually in (DATA/settings) or defaults/web.xml (<auth-method> - the realmname is in defaults/yacy.init adminRealm=YaCy-AdminUI - fyi: the realmname is shown on login screen - changing the realm name invalidates all passwords - but for security you are encouraged to do so (as localhostadmin) - implemented to support both, old hashes for BASIC and new hashes for BASIC and DIGEST - to differentiate old / new hash the in Jetty used hash-prefix "MD5:" is used for new pwd-hashes ( "MD5:hash" )
2014-01-17 00:02:23 +01:00
if (user != null) {
legacyRequestHeader.put(RequestHeader.AUTHORIZATION, HttpServletRequest.BASIC_AUTH + " "
+ username + ":" + user.getMD5EncodedUserPwd());
}
}
}
}
simulate Authorization cookie for yacy servlet header
2014-01-10 19:31:36 +01:00
}
}
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
return legacyRequestHeader;
}
/**
* Returns a path to the localized or default file according to the
* parameter localeSelection
*
* @param path relative from htroot
* @param localeSelection language of localized file; locale.language from
* switchboard is used if localeSelection.equals("")
*/
public File getLocalizedFile(final String path, final String localeSelection) throws IOException {
if (!(localeSelection.equals("default"))) {
downgrade to Jetty 8 to assure support of JRE 1.6 - introduce a YaCyHttp interface to modulize/separate http server - adjust the Jetty version specific implementation part (in package net.yacy.http) - putting the version specific code in classes starting with Jetty8xxxx - moved existing Jetty9xxx implementation into a test class (to keep the code) - adjust build to the changed jars - make use of the introduced YaCyHttpServer interface in related htroot servlets - adjust other test cases/classes
2013-10-09 00:40:48 +02:00
final File localePath = new File(_htLocalePath, localeSelection + '/' + path);
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
if (localePath.exists()) {
return localePath; // avoid "NoSuchFile" troubles if the "localeSelection" is misspelled
}
}
downgrade to Jetty 8 to assure support of JRE 1.6 - introduce a YaCyHttp interface to modulize/separate http server - adjust the Jetty version specific implementation part (in package net.yacy.http) - putting the version specific code in classes starting with Jetty8xxxx - moved existing Jetty9xxx implementation into a test class (to keep the code) - adjust build to the changed jars - make use of the introduced YaCyHttpServer interface in related htroot servlets - adjust other test cases/classes
2013-10-09 00:40:48 +02:00
File docsPath = new File(_htDocsPath, path);
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
if (docsPath.exists()) {
return docsPath;
}
return _resourceBase.addPath(path).getFile();
}
downgrade to Jetty 8 to assure support of JRE 1.6 - introduce a YaCyHttp interface to modulize/separate http server - adjust the Jetty version specific implementation part (in package net.yacy.http) - putting the version specific code in classes starting with Jetty8xxxx - moved existing Jetty9xxx implementation into a test class (to keep the code) - adjust build to the changed jars - make use of the introduced YaCyHttpServer interface in related htroot servlets - adjust other test cases/classes
2013-10-09 00:40:48 +02:00
protected File rewriteClassFile(final File template) {
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
try {
String f = template.getCanonicalPath();
final int p = f.lastIndexOf('.');
if (p < 0) {
return null;
}
f = f.substring(0, p) + ".class";
final File cf = new File(f);
if (cf.exists()) {
return cf;
}
return null;
} catch (final IOException e) {
return null;
}
}
downgrade to Jetty 8 to assure support of JRE 1.6 - introduce a YaCyHttp interface to modulize/separate http server - adjust the Jetty version specific implementation part (in package net.yacy.http) - putting the version specific code in classes starting with Jetty8xxxx - moved existing Jetty9xxx implementation into a test class (to keep the code) - adjust build to the changed jars - make use of the introduced YaCyHttpServer interface in related htroot servlets - adjust other test cases/classes
2013-10-09 00:40:48 +02:00
protected Method rewriteMethod(final File classFile) throws InvocationTargetException {
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
Method m = null;
// now make a class out of the stream
try {
final SoftReference<Method> ref = templateMethodCache.get(classFile);
if (ref != null) {
m = ref.get();
if (m == null) {
templateMethodCache.remove(classFile);
} else {
return m;
}
}
fixed generics warnings for generic array instantiation that appeared after migration to Java 7
2014-05-20 21:50:16 +02:00
final Class<?> c = provider.loadClass(classFile);
final Class<?>[] params = (Class<?>[]) Array.newInstance(Class.class, 3);
params[0]= RequestHeader.class;
params[1] = serverObjects.class;
params[2] = serverSwitch.class;
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
m = c.getMethod("respond", params);
- disable Jetty servlet defaultUseCache (prevent double caching) - include short memory status check for class cache in DefaultServlet - remove obsolete Resource interface for Jetty8YaCyDefaultServlet
2013-11-18 03:15:45 +01:00
if (MemoryControl.shortStatus()) {
templateMethodCache.clear();
} else {
// store the method into the cache
templateMethodCache.put(classFile, new SoftReference<Method>(m));
}
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
} catch (final ClassNotFoundException e) {
reduce logging (by assigning logger to existing logger) - small additional cleanups
2013-10-30 00:51:04 +01:00
ConcurrentLog.severe("FILEHANDLER","YaCyDefaultServlet: class " + classFile + " is missing:" + e.getMessage());
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
throw new InvocationTargetException(e, "class " + classFile + " is missing:" + e.getMessage());
} catch (final NoSuchMethodException e) {
reduce logging (by assigning logger to existing logger) - small additional cleanups
2013-10-30 00:51:04 +01:00
ConcurrentLog.severe("FILEHANDLER","YaCyDefaultServlet: method 'respond' not found in class " + classFile + ": " + e.getMessage());
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
throw new InvocationTargetException(e, "method 'respond' not found in class " + classFile + ": " + e.getMessage());
}
return m;
}
re-implementation of file push option in the context of POST http requests. The internal representation of post-arguments is String and therefore not appropriate for byte[] object as submitted by file pushes. Therefore all pushed files are encoded to base64 _after_ uploading with an http form (you do not need to do that encoding yourself) to hand-over the byte[] as string in the post argument. Servlets which read such files must decode the base64 data to get the original byte[] array. This is considered as a temporary solution for file uploads and a proper implementations would need to consider all attributes as handed over as Objects with either String or byte[] Object instances. This would be a major code change and is not done at this time here now. The feature was submitted to realize a feature as pushed with the next commit.
2014-06-12 18:06:22 +02:00
protected void handleTemplate(String target, HttpServletRequest request, HttpServletResponse response) throws IOException, ServletException {
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
Switchboard sb = Switchboard.getSwitchboard();
added tutorial mode - some menu items will only appear if you 'qualify' for them. Thus, the first-time user will only see four menu items. The other items will unfold as the user interacts.
2014-04-02 02:33:17 +02:00
String localeSelection = sb.getConfig("locale.language", "default");
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
File targetFile = getLocalizedFile(target, localeSelection);
File targetClass = rewriteClassFile(_resourceBase.addPath(target).getFile());
join DefaultServlet and Jetty8 implementation - removing Jetty 8 specific dependencies
2013-12-09 23:45:57 +01:00
String targetExt = target.substring(target.lastIndexOf('.') + 1);
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
if ((targetClass != null)) {
serverObjects args = new serverObjects();
Enumeration<String> argNames = request.getParameterNames();
while (argNames.hasMoreElements()) {
String argName = argNames.nextElement();
re-implementation of file push option in the context of POST http requests. The internal representation of post-arguments is String and therefore not appropriate for byte[] object as submitted by file pushes. Therefore all pushed files are encoded to base64 _after_ uploading with an http form (you do not need to do that encoding yourself) to hand-over the byte[] as string in the post argument. Servlets which read such files must decode the base64 data to get the original byte[] array. This is considered as a temporary solution for file uploads and a proper implementations would need to consider all attributes as handed over as Objects with either String or byte[] Object instances. This would be a major code change and is not done at this time here now. The feature was submitted to realize a feature as pushed with the next commit.
2014-06-12 18:06:22 +02:00
// standard attributes are just pushed as string
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
args.put(argName, request.getParameter(argName));
}
//TODO: for SSI request, local parameters are added as attributes, put them back as parameter for the legacy request
// likely this should be implemented via httpservletrequestwrapper to supply complete parameters
Enumeration<String> attNames = request.getAttributeNames();
while (attNames.hasMoreElements()) {
String argName = attNames.nextElement();
args.put(argName, request.getAttribute(argName).toString());
re-implementation of file push option in the context of POST http requests. The internal representation of post-arguments is String and therefore not appropriate for byte[] object as submitted by file pushes. Therefore all pushed files are encoded to base64 _after_ uploading with an http form (you do not need to do that encoding yourself) to hand-over the byte[] as string in the post argument. Servlets which read such files must decode the base64 data to get the original byte[] array. This is considered as a temporary solution for file uploads and a proper implementations would need to consider all attributes as handed over as Objects with either String or byte[] Object instances. This would be a major code change and is not done at this time here now. The feature was submitted to realize a feature as pushed with the next commit.
2014-06-12 18:06:22 +02:00
}
add support for gzip encoded multipart forms (needed for transferRWI.html) - quick and dirty reuse of existing HTTPDemon implementation
2013-10-14 04:18:52 +02:00
RequestHeader legacyRequestHeader = generateLegacyRequestHeader(request, target, targetExt);
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
// add multipart-form fields to parameter
add support for gzip encoded multipart forms (needed for transferRWI.html) - quick and dirty reuse of existing HTTPDemon implementation
2013-10-14 04:18:52 +02:00
if (ServletFileUpload.isMultipartContent(request)) {
final String bodyEncoding = request.getHeader(HeaderFramework.CONTENT_ENCODING);
fix last commit (not needed test of GZipInputStream)
2013-10-14 04:29:34 +02:00
if (HeaderFramework.CONTENT_ENCODING_GZIP.equalsIgnoreCase(bodyEncoding)) {
implement gzip input handling directly in defaultservlet (making reference to legacy httpdemon obsolete)
2014-04-17 03:20:29 +02:00
parseMultipart(new GZIPRequestWrapper(request),args);
add support for gzip encoded multipart forms (needed for transferRWI.html) - quick and dirty reuse of existing HTTPDemon implementation
2013-10-14 04:18:52 +02:00
} else {
parseMultipart(request, args);
}
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
}
// eof modification to read attribute
Object tmp;
try {
modified DefaultServlet parameter on invoke templates call response with post=0 (if post empty) simulating previous behavior. (template servlets typically test for post==null, found one more Crawler.p.java were empty post caused problem, = defaults not correctly set)
2014-01-04 20:49:26 +01:00
if (args.isEmpty()) {
// yacy servlets typically test for args != null (but not for args .isEmpty())
tmp = invokeServlet(targetClass, legacyRequestHeader, null);
} else {
tmp = invokeServlet(targetClass, legacyRequestHeader, args);
}
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
} catch (InvocationTargetException | IllegalArgumentException | IllegalAccessException e) {
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
ConcurrentLog.logException(e);
better logging in template engine: shows filename of servlets where errors in templates occur
2014-04-04 10:55:46 +02:00
throw new ServletException(targetFile.getAbsolutePath());
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
}
if (tmp instanceof RasterPlotter || tmp instanceof EncodedImage || tmp instanceof Image) {
net.yacy.cora.util.ByteBuffer result = null;
if (tmp instanceof RasterPlotter) {
final RasterPlotter yp = (RasterPlotter) tmp;
// send an image to client
result = RasterPlotter.exportImage(yp.getImage(), "png");
added close to fix possible resource leak warning
2014-03-28 14:34:49 +01:00
} else if (tmp instanceof EncodedImage) {
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
final EncodedImage yp = (EncodedImage) tmp;
result = yp.getImage();
added close to fix possible resource leak warning
2014-03-28 14:34:49 +01:00
} else if (tmp instanceof Image) {
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
final Image i = (Image) tmp;
// generate an byte array from the generated image
int width = i.getWidth(null);
if (width < 0) {
width = 96; // bad hack
}
int height = i.getHeight(null);
if (height < 0) {
height = 96; // bad hack
}
final BufferedImage bi = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
bi.createGraphics().drawImage(i, 0, 0, width, height, null);
result = RasterPlotter.exportImage(bi, targetExt);
}
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
final String mimeType = Classification.ext2mime(targetExt, TEXT_HTML.asString());
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
response.setContentType(mimeType);
response.setContentLength(result.length());
response.setStatus(HttpServletResponse.SC_OK);
result.writeTo(response.getOutputStream());
added close to fix possible resource leak warning
2014-03-28 14:34:49 +01:00
result.close();
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
return;
}
add YaCy HttpCommand "location" check to DefaultServlet
2013-10-16 01:48:44 +02:00
servletProperties templatePatterns;
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
if (tmp == null) {
// if no args given, then tp will be an empty Hashtable object (not null)
templatePatterns = new servletProperties();
} else if (tmp instanceof servletProperties) {
templatePatterns = (servletProperties) tmp;
} else {
templatePatterns = new servletProperties((serverObjects) tmp);
}
add YaCy HttpCommand "authenticate" check to DefaultServlet
2013-10-17 00:06:17 +02:00
add YaCy HttpCommand "location" check to DefaultServlet
2013-10-16 01:48:44 +02:00
// handle YaCy http commands
add YaCy HttpCommand "authenticate" check to DefaultServlet
2013-10-17 00:06:17 +02:00
// handle action auth: check if the servlets requests authentication
if (templatePatterns.containsKey(serverObjects.ACTION_AUTHENTICATE)) {
if (!request.authenticate(response)) {
implemented DIGEST authentication, which is for remote login more secure as BASIC were pwd is transmitted near clear text (B64enc). This has some implication as RFC 2617 requires and recommends a password hash MD5(user:realm:pwd) for DIGEST. !!! before activating DIGEST you have to reassign all passwords !!! to allow new calculation of the hash - default authentication is still BASIC - configuration at this time only manually in (DATA/settings) or defaults/web.xml (<auth-method> - the realmname is in defaults/yacy.init adminRealm=YaCy-AdminUI - fyi: the realmname is shown on login screen - changing the realm name invalidates all passwords - but for security you are encouraged to do so (as localhostadmin) - implemented to support both, old hashes for BASIC and new hashes for BASIC and DIGEST - to differentiate old / new hash the in Jetty used hash-prefix "MD5:" is used for new pwd-hashes ( "MD5:hash" )
2014-01-17 00:02:23 +01:00
return;
add YaCy HttpCommand "authenticate" check to DefaultServlet
2013-10-17 00:06:17 +02:00
}
//handle action forward
} else if (templatePatterns.containsKey(serverObjects.ACTION_LOCATION)) {
add YaCy HttpCommand "location" check to DefaultServlet
2013-10-16 01:48:44 +02:00
String location = templatePatterns.get(serverObjects.ACTION_LOCATION, "");
if (location.isEmpty()) {
location = request.getPathInfo();
}
//TODO: handle equivalent of this from httpdfilehandler
// final ResponseHeader headers = getDefaultHeaders(request.getPathInfo());
// headers.setAdditionalHeaderProperties(templatePatterns.getOutgoingHeader().getAdditionalHeaderProperties()); //put the cookies into the new header TODO: can we put all headerlines, without trouble?
add YaCy HttpCommand "authenticate" check to DefaultServlet
2013-10-17 00:06:17 +02:00
add YaCy HttpCommand "location" check to DefaultServlet
2013-10-16 01:48:44 +02:00
response.setHeader(HeaderFramework.LOCATION, location);
response.setStatus(HttpServletResponse.SC_FOUND);
return;
}
added tutorial mode - some menu items will only appear if you 'qualify' for them. Thus, the first-time user will only see four menu items. The other items will unfold as the user interacts.
2014-04-02 02:33:17 +02:00
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
if (targetFile.exists() && targetFile.isFile() && targetFile.canRead()) {
added tutorial mode - some menu items will only appear if you 'qualify' for them. Thus, the first-time user will only see four menu items. The other items will unfold as the user interacts.
2014-04-02 02:33:17 +02:00
sb.setConfig("server.servlets.called", appendPath(sb.getConfig("server.servlets.called", ""), target));
fix for first start options and added german translation for popup texts
2014-04-02 17:10:59 +02:00
if (args != null && !args.isEmpty()) {
sb.setConfig("server.servlets.submitted", appendPath(sb.getConfig("server.servlets.submitted", ""), target));
}
added tutorial mode - some menu items will only appear if you 'qualify' for them. Thus, the first-time user will only see four menu items. The other items will unfold as the user interacts.
2014-04-02 02:33:17 +02:00
// add the application version, the uptime and the client name to every rewrite table
templatePatterns.put(servletProperties.PEER_STAT_VERSION, yacyBuildProperties.getVersion());
move startup time from old serverCore to switchboard (most used here) to make servercore eventually obsolete.
2014-07-10 02:17:56 +02:00
templatePatterns.put(servletProperties.PEER_STAT_UPTIME, ((System.currentTimeMillis() - sb.startupTime) / 1000) / 60); // uptime in minutes
added tutorial mode - some menu items will only appear if you 'qualify' for them. Thus, the first-time user will only see four menu items. The other items will unfold as the user interacts.
2014-04-02 02:33:17 +02:00
templatePatterns.putHTML(servletProperties.PEER_STAT_CLIENTNAME, sb.peers.mySeed().getName());
templatePatterns.putHTML(servletProperties.PEER_STAT_CLIENTID, sb.peers.myID());
templatePatterns.put(servletProperties.PEER_STAT_MYTIME, GenericFormatter.SHORT_SECOND_FORMATTER.format());
Seed myPeer = sb.peers.mySeed();
templatePatterns.put("newpeer", myPeer.getAge() >= 1 ? 0 : 1);
templatePatterns.putHTML("newpeer_peerhash", myPeer.hash);
templatePatterns.put("navigation-p2p", sb.getConfigBool(SwitchboardConstants.DHT_ENABLED, true) || !sb.isRobinsonMode() ? 1 : 0);
fix for first start options and added german translation for popup texts
2014-04-02 17:10:59 +02:00
String submitted = sb.getConfig("server.servlets.submitted", "");
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
boolean crawler_enabled = submitted.contains("Crawler_p");
boolean advanced_enabled = crawler_enabled || submitted.contains("ConfigBasic") || submitted.contains("CrawlStart");
better logging in template engine: shows filename of servlets where errors in templates occur
2014-04-04 10:55:46 +02:00
templatePatterns.put("navigation-crawlmonitor", crawler_enabled);
templatePatterns.put("navigation-advanced", advanced_enabled);
added tutorial mode - some menu items will only appear if you 'qualify' for them. Thus, the first-time user will only see four menu items. The other items will unfold as the user interacts.
2014-04-02 02:33:17 +02:00
templatePatterns.put(SwitchboardConstants.GREETING_HOMEPAGE, sb.getConfig(SwitchboardConstants.GREETING_HOMEPAGE, ""));
templatePatterns.put(SwitchboardConstants.GREETING_SMALL_IMAGE, sb.getConfig(SwitchboardConstants.GREETING_SMALL_IMAGE, ""));
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
String mimeType = Classification.ext2mime(targetExt, TEXT_HTML.asString());
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
fix servlet class exist check to use default path only (in Jetty8YaCyDefaultServlet) - del redundant doget code in yacydefaultservlet - small declaration code opts - del obsolete libt/proxyservlet.java
2013-11-03 02:26:00 +01:00
InputStream fis;
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
long fileSize = targetFile.length();
if (fileSize <= Math.min(4 * 1024 * 1204, MemoryControl.available() / 100)) {
// read file completely into ram, avoid that too many files are open at the same time
fis = new ByteArrayInputStream(FileUtils.read(targetFile));
} else {
fis = new BufferedInputStream(new FileInputStream(targetFile));
}
// set response header
response.setContentType(mimeType);
response.setStatus(HttpServletResponse.SC_OK);
ByteArrayOutputStream bas = new ByteArrayOutputStream(4096);
// apply templates
better logging in template engine: shows filename of servlets where errors in templates occur
2014-04-04 10:55:46 +02:00
TemplateEngine.writeTemplate(targetFile.getName(), fis, bas, templatePatterns);
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
fis.close();
// handle SSI
optimize parse ServerSideInclude
2014-01-22 21:23:32 +01:00
parseSSI (bas.toByteArray(),request,response);
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
}
}
}
added tutorial mode - some menu items will only appear if you 'qualify' for them. Thus, the first-time user will only see four menu items. The other items will unfold as the user interacts.
2014-04-02 02:33:17 +02:00
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
private static String appendPath(String proplist, String path) {
added tutorial mode - some menu items will only appear if you 'qualify' for them. Thus, the first-time user will only see four menu items. The other items will unfold as the user interacts.
2014-04-02 02:33:17 +02:00
if (proplist.length() == 0) return path;
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
if (proplist.contains(path)) return proplist;
added tutorial mode - some menu items will only appear if you 'qualify' for them. Thus, the first-time user will only see four menu items. The other items will unfold as the user interacts.
2014-04-02 02:33:17 +02:00
return proplist + "," + path;
}
optimize parse ServerSideInclude
2014-01-22 21:23:32 +01:00
/**
* parse SSI line and include resource (<!--#include virtual="file.html" -->)
*/
protected void parseSSI(final byte[] in, HttpServletRequest request, HttpServletResponse response) throws IOException, ServletException {
ByteBuffer buffer = new ByteBuffer(in);
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
OutputStream out = response.getOutputStream();
optimize parse ServerSideInclude
2014-01-22 21:23:32 +01:00
final byte[] inctxt ="<!--#include virtual=\"".getBytes();
int offset = 0;
int p = buffer.indexOf(inctxt, offset);
int end;
while (p >= 0 && (end = buffer.indexOf("-->".getBytes(), p + 24)) > 0 ) { // min length 24; <!--#include virtual="a"
out.write(in, offset, p - offset);
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
out.flush();
optimize parse ServerSideInclude
2014-01-22 21:23:32 +01:00
// find right end quote
final int rightquote = buffer.indexOf("\"".getBytes(), p + 23);
if (rightquote > 0 && rightquote < end) {
final String path = buffer.toString(p + 22, rightquote - p - 22);
RequestDispatcher dispatcher = request.getRequestDispatcher(path);
fix IOexception logging issue in DefaultServlet reason not sure but .logException triggers another exception
2014-01-20 08:12:35 +01:00
try {
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
dispatcher.include(request, response);
fix IOexception logging issue in DefaultServlet reason not sure but .logException triggers another exception
2014-01-20 08:12:35 +01:00
} catch (IOException ex) {
fixed a time-out problem in the default servlet which is also a logging problem because the error log showed the wrong reason (file not found) instead the actual reason (time-out).
2014-04-04 15:27:29 +02:00
if (path.indexOf("yacysearch") < 0) ConcurrentLog.warn("FILEHANDLER", "YaCyDefaultServlet: parseSSI dispatcher problem - " + ex.getMessage() + ": " + path);
// this is probably a time-out; it may occur during search requests; for search requests we consider that normal
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
}
optimize parse ServerSideInclude
2014-01-22 21:23:32 +01:00
} else {
fixed a time-out problem in the default servlet which is also a logging problem because the error log showed the wrong reason (file not found) instead the actual reason (time-out).
2014-04-04 15:27:29 +02:00
ConcurrentLog.warn("FILEHANDLER", "YaCyDefaultServlet: parseSSI closing quote missing " + buffer.toString(p, end - p) + " in " + request.getPathInfo());
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
}
optimize parse ServerSideInclude
2014-01-22 21:23:32 +01:00
offset = end + 3; // after "-->"
p = buffer.indexOf(inctxt, offset);
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
}
optimize parse ServerSideInclude
2014-01-22 21:23:32 +01:00
out.write(in, offset, in.length - offset);
fixed visibility of custom icon
2014-03-28 17:25:39 +01:00
out.close();
fixed 'resource leak' warning
2014-02-03 13:38:26 +01:00
buffer.close();
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
}
optimize parse ServerSideInclude
2014-01-22 21:23:32 +01:00
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
/**
* TODO: add same functionality & checks as in HTTPDemon.parseMultipart
*
* parse multi-part form data for formfields (only), see also original
* implementation in HTTPDemon.parseMultipart
*
* @param request
* @param args found fields/values are added to the map
*/
using concurrency to do base64 encoding in file POST commands
2014-06-20 13:55:15 +02:00
protected void parseMultipart(final HttpServletRequest request, final serverObjects args) throws IOException {
- add size check to multipart form data handling of YaCyDefaultServlet (same as in HTTPDemon.parseMultipart) - reduce Jetty logging - give build.run a bit more memory (set to YaCy.default 600m from 512m)
2013-10-13 20:56:03 +02:00
// reject too large uploads
if (request.getContentLength() > SIZE_FILE_THRESHOLD) throw new IOException("FileUploadException: uploaded file too large = " + request.getContentLength());
// check if we have enough memory
if (!MemoryControl.request(request.getContentLength() * 3, false)) {
throw new IOException("not enough memory available for request. request.getContentLength() = " + request.getContentLength() + ", MemoryControl.available() = " + MemoryControl.available());
add support for gzip encoded multipart forms (needed for transferRWI.html) - quick and dirty reuse of existing HTTPDemon implementation
2013-10-14 04:18:52 +02:00
}
- add size check to multipart form data handling of YaCyDefaultServlet (same as in HTTPDemon.parseMultipart) - reduce Jetty logging - give build.run a bit more memory (set to YaCy.default 600m from 512m)
2013-10-13 20:56:03 +02:00
ServletFileUpload upload = new ServletFileUpload(DISK_FILE_ITEM_FACTORY);
implement gzip input handling directly in defaultservlet (making reference to legacy httpdemon obsolete)
2014-04-17 03:20:29 +02:00
upload.setFileSizeMax(SIZE_FILE_THRESHOLD);
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
try {
// Parse the request to get form field items
add support for gzip encoded multipart forms (needed for transferRWI.html) - quick and dirty reuse of existing HTTPDemon implementation
2013-10-14 04:18:52 +02:00
@SuppressWarnings("unchecked")
List<FileItem> fileItems = upload.parseRequest(request);
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
// Process the uploaded file items
Iterator<FileItem> i = fileItems.iterator();
using concurrency to do base64 encoding in file POST commands
2014-06-20 13:55:15 +02:00
final BlockingQueue<Map.Entry<String, byte[]>> files = new LinkedBlockingQueue<>();
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
while (i.hasNext()) {
- add size check to multipart form data handling of YaCyDefaultServlet (same as in HTTPDemon.parseMultipart) - reduce Jetty logging - give build.run a bit more memory (set to YaCy.default 600m from 512m)
2013-10-13 20:56:03 +02:00
FileItem item = i.next();
if (item.isFormField()) {
// simple text
if (item.getContentType() == null || !item.getContentType().contains("charset")) {
// old yacy clients use their local default charset, on most systems UTF-8 (I hope ;)
args.add(item.getFieldName(), item.getString("UTF-8"));
} else {
// use default encoding (given as header or ISO-8859-1)
args.add(item.getFieldName(), item.getString());
}
re-implementation of file push option in the context of POST http requests. The internal representation of post-arguments is String and therefore not appropriate for byte[] object as submitted by file pushes. Therefore all pushed files are encoded to base64 _after_ uploading with an http form (you do not need to do that encoding yourself) to hand-over the byte[] as string in the post argument. Servlets which read such files must decode the base64 data to get the original byte[] array. This is considered as a temporary solution for file uploads and a proper implementations would need to consider all attributes as handed over as Objects with either String or byte[] Object instances. This would be a major code change and is not done at this time here now. The feature was submitted to realize a feature as pushed with the next commit.
2014-06-12 18:06:22 +02:00
} else {
using concurrency to do base64 encoding in file POST commands
2014-06-20 13:55:15 +02:00
// read file upload
InputStream filecontent = null;
re-implementation of file push option in the context of POST http requests. The internal representation of post-arguments is String and therefore not appropriate for byte[] object as submitted by file pushes. Therefore all pushed files are encoded to base64 _after_ uploading with an http form (you do not need to do that encoding yourself) to hand-over the byte[] as string in the post argument. Servlets which read such files must decode the base64 data to get the original byte[] array. This is considered as a temporary solution for file uploads and a proper implementations would need to consider all attributes as handed over as Objects with either String or byte[] Object instances. This would be a major code change and is not done at this time here now. The feature was submitted to realize a feature as pushed with the next commit.
2014-06-12 18:06:22 +02:00
try {
using concurrency to do base64 encoding in file POST commands
2014-06-20 13:55:15 +02:00
filecontent = item.getInputStream();
files.put(new AbstractMap.SimpleEntry<String, byte[]>(item.getFieldName(), FileUtils.read(filecontent)));
re-implementation of file push option in the context of POST http requests. The internal representation of post-arguments is String and therefore not appropriate for byte[] object as submitted by file pushes. Therefore all pushed files are encoded to base64 _after_ uploading with an http form (you do not need to do that encoding yourself) to hand-over the byte[] as string in the post argument. Servlets which read such files must decode the base64 data to get the original byte[] array. This is considered as a temporary solution for file uploads and a proper implementations would need to consider all attributes as handed over as Objects with either String or byte[] Object instances. This would be a major code change and is not done at this time here now. The feature was submitted to realize a feature as pushed with the next commit.
2014-06-12 18:06:22 +02:00
} catch (IOException e) {
ConcurrentLog.logException(e);
using concurrency to do base64 encoding in file POST commands
2014-06-20 13:55:15 +02:00
} finally {
if (filecontent != null) try {filecontent.close();} catch (IOException e) {ConcurrentLog.logException(e);}
re-implementation of file push option in the context of POST http requests. The internal representation of post-arguments is String and therefore not appropriate for byte[] object as submitted by file pushes. Therefore all pushed files are encoded to base64 _after_ uploading with an http form (you do not need to do that encoding yourself) to hand-over the byte[] as string in the post argument. Servlets which read such files must decode the base64 data to get the original byte[] array. This is considered as a temporary solution for file uploads and a proper implementations would need to consider all attributes as handed over as Objects with either String or byte[] Object instances. This would be a major code change and is not done at this time here now. The feature was submitted to realize a feature as pushed with the next commit.
2014-06-12 18:06:22 +02:00
}
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
}
}
using concurrency to do base64 encoding in file POST commands
2014-06-20 13:55:15 +02:00
if (files.size() <= 1) {
for (Map.Entry<String, byte[]> fe: files) {
// the file is written in base64 encoded form to a string
args.put(fe.getKey(), Base64Order.standardCoder.encode(fe.getValue()));
}
} else {
// do this concurrently (this would all be superfluous if serverObjects could store byte[] instead only String)
int t = Math.min(files.size(), Runtime.getRuntime().availableProcessors());
final Map.Entry<String, byte[]> POISON = new AbstractMap.SimpleEntry<>(null, null);
Thread[] p = new Thread[t];
for (int j = 0; j < t; j++) {
files.put(POISON);
p[j] = new Thread() {
@Override
public void run() {
Map.Entry<String, byte[]> job;
try {while ((job = files.take()) != POISON) {
String b64 = Base64Order.standardCoder.encode(job.getValue());
synchronized (args) {args.put(job.getKey(), b64);}
}} catch (InterruptedException e) {}
}
};
p[j].start();
}
for (int j = 0; j < t; j++) p[j].join();
}
implement a YaCyDefaultServlet to handle YaCy-servlets within Jetty server - the implementation is inspired by Jetty's DefaultServlet - handles static html content and YaCy servlets - translates between standard servlet request/response and YaCy request/response specification With the implementation of YaCy-servlets as servlet instead via a jetty handler it's closer to servlet standard and carries less jetty specific dependencies.
2013-09-22 01:57:32 +02:00
} catch (Exception ex) {
ConcurrentLog.logException(ex);
}
}
implement gzip input handling directly in defaultservlet (making reference to legacy httpdemon obsolete)
2014-04-17 03:20:29 +02:00
/**
* wraps request to uncompress gzip'ed input stream
*/
private class GZIPRequestWrapper extends HttpServletRequestWrapper {
private final ServletInputStream is;
public GZIPRequestWrapper(HttpServletRequest request) throws IOException {
super(request);
this.is = new GZIPRequestStream(request);
}
@Override
public ServletInputStream getInputStream() throws IOException {
return is;
}
}
private class GZIPRequestStream extends ServletInputStream {
private final GZIPInputStream in;
public GZIPRequestStream(HttpServletRequest request) throws IOException {
this.in = new GZIPInputStream(request.getInputStream());
}
@Override
public int read() throws IOException {
return in.read();
}
@Override
public int read(byte[] b) throws IOException {
return in.read(b);
}
@Override
public int read(byte[] b, int off, int len) throws IOException {
return in.read(b, off, len);
}
@Override
public void close() throws IOException {
in.close();
}
update to Jetty 9 besides adjustments in code it makes the servlet settings in web.xml significant. This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
2014-05-11 01:53:11 +02:00
@Override
public boolean isFinished() {
try {
return (in.available() < 1);
} catch (IOException ex) {
return true;
}
}
@Override
public boolean isReady() {
try {
return in.available() > 0;
} catch (IOException ex) {
return false;
}
}
@Override
public void setReadListener(ReadListener rl) {
// TODO: implement
}
implement gzip input handling directly in defaultservlet (making reference to legacy httpdemon obsolete)
2014-04-17 03:20:29 +02:00
}
add support for gzip encoded multipart forms (needed for transferRWI.html) - quick and dirty reuse of existing HTTPDemon implementation
2013-10-14 04:18:52 +02:00
}
Reference in New Issue Copy Permalink