This project provides a webserver service for use with the trapperkeeper service framework To use this service in your trapperkeeper application, simply add this project as a dependency in your leiningen project file:
Then add the webserver service to your bootstrap.cfg
file, via:
puppetlabs.trapperkeeper.services.webserver.jetty10-service/jetty10-service
Note that this implementation of the
:WebserverService interface is based on Jetty 10. This service requires JDK 11 or greater;
however, the interface is intended to be agnostic to the underlying web server
implementation. We also provide a
Jetty 9 version of the service,
which can be used interchangeably with this one and will support older JDKs.
You should only need to change your lein dependencies and your bootstrap.cfg
file--no code changes.
The web server is configured via the
trapperkeeper configuration service;
so, you can control various properties of the server (ports, SSL, etc.) by adding a webserver
section to one of your Trapperkeeper configuration files, and setting various properties
therein. For more info, see Configuring the Webserver. It is possible to configure
both a single webserver or multiple webservers.
The webserver-service currently supports web applications built using
Clojure's Ring library and Java's Servlet
API. There is also an experimental webserver service that supports loading
ruby Rack applications via JRuby; for more info, see the
trapperkeeper-ruby project.
Four examples are included with this project:
- A Ring example (source code)
- A Java servlet example (source code)
- A WAR example (source code)
- A multiserver configuration example (source code)
This is the protocol for the current implementation of the :WebserverService:
(defprotocol WebserverService
(add-context-handler [this base-path context-path] [this base-path context-path options])
(add-ring-handler [this handler path] [this handler path options])
(add-websocket-handler [this handlers path] [this handler path options])
(add-servlet-handler [this servlet path] [this servlet path options])
(add-war-handler [this war path] [this war path options])
(add-proxy-route [this target path] [this target path options])
(override-webserver-settings! [this overrides] [this server-id overrides])
(get-registered-endpoints [this] [this server-id])
(log-registered-endpoints [this] [this server-id])
(join [this] [this server-id])Here is a bit more info about each of these functions:
add-ring-handler takes two arguments: [handler path]. The handler argument
is just a normal Ring application (the same as what you would pass to run-jetty
if you were using the ring-jetty-adapter). The path is a URL prefix / context
string that will be prepended to all your handler's URLs; this is key to allowing
the registration of multiple handlers in the same web server without the possibility
of URL collisions. So, for example, if your ring handler has routes /foo and
/bar, and you call:
(add-ring-handler my-app "/my-app")Then your routes will be served at /my-app/foo and my-app/bar.
You may specify "" as the value for path if you are only registering a single
handler and do not need to prefix the URL.
In the handler, in the request passed to it, under the :response key is the
Response that is used
to deliver the response from the server.
There is also a three argument version of this function which takes these arguments:
[handler path options]. options is a map containing three optional keys.
The first is
:server-id, which specifies which server you want to add the ring-handler to. If
:server-id is specified, the ring handler will be added to the server with id
:server-id. If no :server-id is specified, or the two argument version is called,
the ring handler will be added to the default server. Calling the two-argument version or
leaving out :server-id will not work in a multiserver set-up if no default server is specified.
The second optional argument is :redirect-if-no-trailing-slash. When set to true,
all requests made to the endpoint at which the ring-handler was registered will, if
no trailing slash is present, return a 302 redirect response to the same URL but with a trailing slash
added. If the option is set to false, no redirect will occur, and the request will be
routed through to the registered handler. This option defaults to false.
The third optional argument is :normalize-request-uri. When set to true, the
URI made available to the ring handler request map via the :uri key will have
been "normalized". See the [Request URI Normalization]
(#request-uri-normalization) section for more information on the
normalization process. When set to false (the default value), the raw path
component from the HTTP request URI will be the value for the :uri key.
Here's an example of how to use the :WebserverService:
(defservice MyWebService
[[:WebserverService add-ring-handler]]
;; initialization
(init [this context]
(add-ring-handler my-app "/my-app")
context))This would add your ring handler to the default server at endpoint "/my-app". Alternatively, if you did this:
(defservice MyWebService
[[:WebserverService add-ring-handler]]
;; initialization
(init [this context]
(add-ring-handler my-app "/my-app" {:server-id :foo})
context))it would add your ring handler to the server with id :foo at endpoint "/my-app",
rather than the default server.
NOTE FOR COMPOJURE APPS: If you are using compojure, it's important to note
that compojure requires use of the context macro
in order to support nested routes. So, if you're not already using context,
you will need to do something like this:
(ns foo
(:require [compojure.core :as c]
;;...
))
(defservice MyWebService
[[:WebserverService add-ring-handler]]
;; initialization
(init [this svc-context]
(let [context-path "/my-app"
context-app (c/context context-path [] my-compojure-app)]
(add-ring-handler context-app context-path))
svc-context))The :normalize-request-uri setting, which can be provided in the options
argument for an add handler call, controls whether or not the
path component from the HTTP
request URI is normalized. The value for the setting is expected to be a
boolean.
When set to false (the default value), the "raw" path component will be
used by the webserver when evaluating a request to the handler.
When set to true, the path component that the webserver evaluates for a
request to the handler will have been "normalized". For a Ring request handler,
the "normalized" value (instead of the "raw" value) will be associated with the
:uri key in the Ring request map. For a Servlet request handler, the
"normalized" value (instead of the "raw" value) will be returned from a call
made to the getRequestURI method on the HttpServletRequest object.
The following steps, in order, are performed against the raw path component
when the :normalize-request-uri setting is true:
- URL (percent) decode the path, assuming any percent-encodings represent UTF-8 characters.
For example:
/foo//bar/%2E%2E/ba%7A => /foo//bar/../baz
If a non-percent encoded semicolon character, U+003B, is found in the path during the percent decoding step, that character and all following characters will be removed from the resulting path, unless there is another forward slash, in which case the characters from the semicolon to the next forward slash (including the semicolon) will be removed.
For example:
/foo//bar/%2E%2E/ba%7A;bim => /foo//bar/../baz
/foo/bar;bar=chocolate/baz;baz=bim => /foo/bar/baz
Requests intending to include a semicolon in the path should percent-encode the semicolon. In this case, the server will preserve the semicolon after the decoding step.
For example:
/foo//bar/%2E%2E/ba%7A%3Bbim => /foo//bar/../baz;bim
If the request has malformed content, e.g., partially-formed percent-encoded characters like '%A%B', an HTTP 400 (Bad Request) error will be returned.
If the request has invalid % encoded UTF-8 characters, the path will be decoded as an ISO-8859-1 encoded string.
- Check the percent-decoded path for any relative path segments ('..' or '.').
If one or more relative path segments are found, an HTTP 400 (Bad Request) error will be returned.
For example, an error would be returned for any of the following paths:
.
..
/foo//bar/../baz
/foo//./bar/baz
The following paths would not be considered to contain relative paths:
/foo//bar/baz
/foo//bar/.../baz
/foo//bar/a.b/baz
/foo//bar/a..b/baz
- Compact any repeated forward slash characters in a path.
For example:
/foo//bar/baz => /foo/bar/baz
/foo/bar////baz => /foo/bar/baz
The following example shows the result after normalization of a URI request path which includes repeated forward slash characters which have been percent-encoded:
/foo%2F%2Fbar/ba%7A => /foo/bar/baz
add-context-handler takes two arguments: [base-path context-path]. The base-path
argument is a URL string pointing to a location containing static resources which are
made accessible at the context-path URL prefix.
For example, to make your CSS files stored in the resources/css directory available
at /css:
(defservice MyWebService
[[:WebserverService add-context-handler]]
;; initialization
(init [this context]
(add-context-handler "resources/css" "/css")
context))There is also a three argument version of the function which takes these arguments:
[base-path context-path options], where the first two arguments are the
same as in the two argument version and options is a map containing five optional keys,
:server-id, :redirect-if-no-trailing-slash, :normalize-request-uri,
:follow-links, and :context-listeners.
The value stored in :server-id specifies which server
to add the context handler to, similar to how it is done in add-ring-handler. Again, like
add-ring-handler, if this key is absent or the two argument version is called, the context handler
will be added to the default server. Calling the two-argument version or leaving out :server-id
will not work in a multiserver set-up if no default server is specified.
The value stored in :redirect-if-no-trailing-slash is a boolean indicating whether or not
to redirect when a request is made to this handler without a trailing slash, just like with
add-ring-handler. Again, this defaults to false.
The value stored in :normalize-request-uri is a boolean indicating whether
or not the request URI should be normalized before it is made available to the
handler. See the Request URI Normalization
section for more information on the normalization process.
The value stored in :follow-links is a
boolean indicating whether or not symbolic links
should be served. The service does NOT serve symbolic links by default.
The value stored in :context-listeners is a list of objects implementing the
[ServletContextListener] (http://docs.oracle.com/javaee/7/api/javax/servlet/ServletContextListener.html)
interface. These listeners are registered with the context created for serving the
static content and receive notifications about the lifecycle events in the context
as defined in the ServletContextListener interface. Of particular interest is the
contextInitialized event notification as it provides access to the configuration
of the context through the methods defined in the [ServletContext]
(http://docs.oracle.com/javaee/7/api/javax/servlet/ServletContext.html)
interface. This opens up wide possibilities for customizing the context - in an
extreme case the context originally capable of serving just the static content can
be changed through this mechanism to a fully dynamic web application (in fact this
very mechanism is used in the [trapperkeeper-ruby]
(https://github.com/puppetlabs/trapperkeeper-ruby) project to turn the context into
a container for hosting an arbitrary ruby rack application - see [here]
(https://github.com/puppetlabs/trapperkeeper-ruby/blob/master/src/clojure/puppetlabs/trapperkeeper/services/rack_jetty/rack_jetty_service.clj)).
add-servlet-handler takes two arguments: [servlet path]. The servlet argument
is a normal Java Servlet.
The path is the URL prefix at which the servlet will be registered.
There is also a three argument version of the function which takes these arguments:
[servlet path options], where the first two arguments are the same as
in the two argument version and options is a map containing four optional keys,
:server-id, :redirect-if-no-trailing-slash, normalize-request-uri, and
:servlet-init-params.
As in add-ring-handler, :server-id specifies which server to add
the handler to. If :server-id is absent or the two-argument function is called, the servlet
handler will be added to the default server. Calling the two-argument version or leaving out
:server-id will not work in a multiserver set-up if no default server is specified.
The value stored in :redirect-if-no-trailing-slash is a boolean indicating whether or not
to redirect when a request is made to this handler without a trailing slash, just like with
add-ring-handler. Again, this defaults to false.
The value stored in :normalize-request-uri is a boolean indicating whether
or not the request URI should be normalized before it is made available to the
handler. See the Request URI Normalization
section for more information on the normalization process.
The value stored at the :servlet-init-params key is a map of servlet init
parameters.
For example, to host a servlet at /my-app:
(ns foo
;; ...
(:import [bar.baz SomeServlet]))
(defservice MyWebService
[[:WebserverService add-servlet-handler]]
;; initialization
(init [this context]
(add-servlet-handler (SomeServlet. "some config") "/my-app")
context))For more information see the example servlet app.
add-websocket-handler takes two arguments: [handlers path].
The handlers is a map of callbacks to invoke when handling a websocket session.
The path is the URL prefix where this websocket servlet will be registered.
The possible callbacks for the handlers map are:
{:on-connect (fn [ws])
:on-error (fn [ws error])
:on-close (fn [ws status-code reason])
:on-text (fn [ws text])
:on-bytes (fn [ws bytes offset len])}Querying data or sending messages over the websocket is supported by
the functions of WebSocketProtocol protocol from the
puppetlabs.trapperkeeper.services.websocket-session namespace:
(connected? [this]
"Returns a boolean indicating if the session is currently connected")
(send! [this msg]
"Send a message to the websocket client")
(close! [this] [this code reason]
"Close the websocket session")
(remote-addr [this]
"Find the remote address of a websocket client")
(ssl? [this]
"Returns a boolean indicating if the session was established by wss://")
(peer-certs [this]
"Returns an array of X509Certs presented by the ssl peer, if any")
(request-path [this]
"Returns the URI path used in the websocket upgrade request to the server"))For example, to provide a simple websockets echo service as /wsecho:
(ns foo
(:require [puppetlabs.trapperkeeper.services.websocket-session :as ws-session]))
(def echo-handlers
{:on-text (fn [ws text] (ws-session/send! ws text))})
(defservice wsecho-webservice
[[:WebserverService add-websocket-handler]]
(init [this context]
(add-websocket-handler echo-handlers "/wsecho")
context))add-war-handler takes two arguments: [war path].
The war is the file path or the URL to a WAR file.
The path is the URL prefix at which the WAR will be registered.
For example, to host resources/cas.war WAR at /cas:
(defservice cas-webservice
[[:WebserverService add-war-handler]]
(init [this context]
(add-war-handler "resources/cas.war" "/cas")
context))There is also a three-argument version that takes these parameters:
[war path options]. options is a map containing three optional
keys, :server-id, :redirect-if-no-trailing-slash, and
:normalize-request-uri.
As with add-ring-handler,
this determines which server the handler is added to. If this key is absent or the two argument
version is called, the handler will be added to the default server. Calling
the two-argument version or leaving out :server-id will not work in a
multiserver set-up if no default server is specified.
The value stored in :redirect-if-no-trailing-slash is a boolean indicating whether or not
to redirect when a request is made to this handler without a trailing slash, just like with
add-ring-handler. Again, this defaults to false.
The value stored in :normalize-request-uri is a boolean indicating whether
or not the request URI should be normalized before it is made available to the
handler. See the Request URI Normalization
section for more information on the normalization process.
add-proxy-route is used to configure certain the server as a reverse proxy for
certain routes. This function will accept two or three arguments: [target path], or
[target path options].
path is the URL prefix for requests that you wish to proxy.
target is a map that controls how matching requests will be proxied; here are
the keys required in the target map:
:host: required; a string representing the host or IP to proxy requests to.:port: required; an integer representing the port on the remote host that requests should be proxied to.:path: required; the URL prefix that should be prepended to all proxied requests.
options, if provided, is a map containing optional configuration for the proxy
route:
:scheme: optional; legal values are:orig,:http, and:https. If you specify:httpor:https, then all proxied requests will use the specified scheme. The default value is:orig, which means that proxied requests will use the same scheme as the original request.:ssl-config: optional; may be set to either:use-server-config(default) or to a map containing the keys:ssl-cert,:ssl-key, and:ssl-ca-cert, as well as the optional keys:cipher-suitesand:protocols. If:use-server-config, then any proxied requests that use HTTPS will use the same SSL context/configuration that the web server is configured with. If you specify a map, then the entries must point to the PEM files that should be used for the SSL context. These keys have the same meaning as they do for the SSL configuration of the main web server.:rewrite-uri-callback-fn: optional; a function to manipulate the rewritten target URI (e.g. change the port, or even change the entire URI) before Jetty continues on with the proxy. The function must accept two arguments,[target-uri req]. For more information, see below.:callback-fn: optional; a function to manipulate the request object (e.g. to add additional headers) before Jetty continues on with the proxy. The function must accept two arguments,[proxy-req req]. For more information, see below.:request-buffer-size: optional; an integer value to which to set the size of the request buffer used by the HTTP Client. This allows HTTP requests with very large cookies to go through, as a large cookie can cause the request buffer to overflow unless the size is increased. The default is 4096 bytes.:follow-redirects: optional; a boolean that indicates whether or not the HttpClient created by a ProxyServlet should follow redirects. Defaults tofalse.:server-id: optional; the id of the server to which to add the proxy handler. If absent, the handler will be added to the default server. If the two argument version of this function is called, the handler will also be added to the default server. Leaving out:server-idor calling the two argument version of this function will not work in a multiserver set-up if no default server is specified.:redirect-if-no-trailing-slash: optional; a boolean indicating whether or not to redirect when a request is made to this proxy route without a trailing slash, as withadd-ring-handler. Defaults to false.:idle-timeout: optional; sets the maximum amount of time, measured in seconds, for the proxy to wait for a response from the upstream server. If no response is received within the time specified, then an HTTP 504 error is returned. If this option is not specified then a value of 30 seconds is used.
Simple example:
(defservice foo-service
[[:WebserverService add-proxy-route]]
(init [this context]
(add-proxy-route
{:host "localhost"
:port 10000
:path "/bar"}
"/foo")
context))In this example, all incoming requests with a prefix of /foo will be proxied
to localhost:10000, with a prefix of /bar, using the same scheme (HTTP/HTTPS)
that the original request used, and using the SSL context of the main webserver.
So, e.g., an HTTPS request to the main webserver at /foo/hello-world would be
proxied to https://localhost:10000/bar/hello-world.
A slightly more complex example:
(defservice foo-service
[[:WebserverService add-proxy-route]]
(init [this context]
(add-proxy-route
{:host "localhost"
:port 10000
:path "/bar"}
"/foo"
{:scheme :https
:ssl-config {:ssl-cert "/tmp/cert.pem"
:ssl-key "/tmp/key.pem"
:ssl-ca-cert "/tmp/ca.pem"}})
context))In this example, all incoming requests with a prefix of foo will be proxied
to https://localhost:10000/bar. We'll proxy using HTTPS even if the original
request was HTTP, and we'll use the three pem files in /tmp to configure the
HTTPS client, regardless of the SSL configuration of the main web server.
#####:rewrite-uri-callback-fn
This option lets you provide a function to manipulate the rewritten target URI. The
function is called in the overridden implementation of
rewriteURI
method after the target URI is computed. It must take two arguments, [target-uri req], where target-uri is a
URI
and req is an
HttpServletRequest.
target-uri will be modified and returned by the function.
An example with a rewrite URI callback function:
(defservice foo-service
[[:WebserverService add-proxy-route]]
(init [this context]
(add-proxy-route
{:host "localhost"
:port 10000
:path "/bar"}
"/foo"
{:rewrite-uri-callback-fn (fn [target-uri req]
(if-not (= "GET" (.getMethod req))
(URI. "http://localhost:11111/errors/unsupported-method")
target-uri))})
context))In this example, all incoming requests with a method other than GET will be proxied
to http://localhost:11111/errors/unsupported-method.
#####:callback-fn
This option lets you provide a function to manipulate the request object. The
function will be passed to the
customizeProxyRequest
method. It must take two arguments, [proxy-req req], where proxy-req is a
Request
and req is an
HttpServletRequest.
proxy-req will be modified and returned by the function.
An example with a callback function:
(defservice foo-service
[[:WebserverService add-proxy-route]]
(init [this context]
(add-proxy-route
{:host "localhost"
:port 10000
:path "/bar"}
"/foo"
{:callback-fn (fn [proxy-req req]
(.header proxy-req "x-example" "baz"))})
context))In this example, all incoming requests with a prefix of foo will be proxied
to https://localhost:10000/bar, using the same scheme (HTTP/HTTPS) that the
original request used, and using the SSL context of the main webserver. In
addition, a header "x-example" with the value "baz" will be added to the
request before it is proxied, using the
header
method.
override-webserver-settings! is used to override settings in the webserver
section of the webserver service's config file. This function will accept one
argument, [overrides]. overrides is a map which should contain a
key/value pair for each setting to be overridden. The name of the setting to
override should be expressed as a Clojure keyword. For any setting expressed in
the service config which is not overridden, the setting value from the config
will be used.
For example, the webserver config may contain:
webserver {
ssl-host: 0.0.0.0
ssl-port: 9001
ssl-cert: mycert.pem
ssl-key: mykey.pem
ssl-ca-cert: myca.pem
}
Overrides may be supplied from the service using code like the following:
(defservice foo-service
[[:WebserverService override-webserver-settings!]]
(init [this context]
(override-webserver-settings!
{:ssl-port 9002
:ssl-cert "myoverriddencert.pem"
:ssl-key "myoverriddenkey.pem"})
context))For this example, the effective settings used during webserver startup would be:
{:ssl-host "0.0.0.0"
:ssl-port 9002
:ssl-cert "myoverriddencert.pem"
:ssl-key "myoverriddenkey.pem"
:ssl-ca-cert "myca.pem"}The overridden webserver settings will be considered only at the point the webserver is being started -- during the start lifecycle phase of the webserver service. For this reason, a call to this function must be made during a service's init lifecycle phase in order for the overridden settings to be considered.
Only one call from a service may be made to this function during application startup.
If a call is made to this function after webserver startup or after another call has already been made to this function (e.g., from other service), a java.lang.IllegalStateException will be thrown.
A three argument version is available which takes these parameters: [server-id overrides].
server-id is the id of the server for which you wish to override the settings. If the
two argument version is called, they will be overridden for the default server. The one-argument
version of this function will not work in a multiserver set-up if no default server is specified.
This function returns a map containing information on each URL endpoint registered by the Jetty10 service on the default server. Each key in the map is a URL endpoint, with each value being an array of maps containing information on each handler registered at that URL endpoint. The possible keys appearing in these maps are:
:type: The type of the registered endpoint. The possible types are:context,:ring,:servlet,:war, and:proxy. Returned for every endpoint.:base-path: The base-path of a context handler. Returned only for endpoints of type:context.:context-listeners: The context listeners for a context handler. Returned only for endpoints of type:contextthat have context listeners.:servlet: The servlet for a servlet handler. Only returned for endpoints of type:servlet.:war-path: The local path of the war registered by a war handler. Only returned for endpoints of type:war.:target-host: The targeted host of a proxy request. Only returned for endpoints of type:proxy.:target-port: The targeted port of a proxy request. Only returned for endpoints of type:proxy.:target-path: The targeted prefix of a proxy request. Only returned for endpoints of type:proxy.
The schema for the various types of handler maps can be viewed here.
There is also a version that takes one argument, [server-id], which specifies which server
for which you want to pull the endpoints. If this parameter is absent, the endpoints will be
pulled for the default server. The no-argument version of this function will not work in a
multiserver set-up if no default server is specified.
This function logs the data returned by get-registered-endpoints at the info level.
There is a version of this function that takes a single argument, [server-id]. This
specifies which server for which you want to log the endpoints. If this is absent,
the endpoints registered on the default server will be logged. The no-argument version
of this function will not work in a multiserver set-up if no default server is specified.
This function is not recommended for normal use, but is provided for compatibility
with the ring-jetty-adapter. ring-jetty-adapter/run-jetty, by default,
calls join on the underlying Jetty server instance. This allows your thread
to block until Jetty shuts down. This should not be necessary for normal
trapperkeeper usage, because trapperkeeper already blocks the main thread and
waits for a termination condition before allowing the process to exit. However,
if you do need this functionality for some reason, you can simply call (join)
to cause your thread to wait for the Jetty server to shut down.
There is another version of this function that takes a single argument, [server-id].
This is the id of the server you want to join. If this is not specified, then
the default server will be joined. The no-argument version of this function will not
work in a multi-server set-up if no default server is specified.
The Trapperkeeper service manipulates the Java Jetty code in the following ways during these lifecycle phases.
A ContextHandlerCollection is created during the init lifecycle which allows for
consumers to use the add-*-handler and add-proxy-route functions,
but the Jetty server itself has not started yet. This allows the service
consumer to setup SSL keys and perform other operations needed before the server is started.
In the start lifecycle phase the Jetty server object is created, the ContextHandlerCollection is added to it, and the server is then started. Adding handlers
after this phase should still work fine, but it is recommended that handlers be added
during the consuming service's init phase.
This project provides a secondary Webrouting Service, which in many cases is preferable for use over the Webserver Service. Documentation is available for it here.
This project provides some utility code for testing. Documentation on these test utils is available here.
For issues on the Trapperkeeper Webserver Service, submissions are welcome on Trapperkeeper webserver project on GitHub.