Styx Forwarding Path Improvements

[OwenLindsell]

Styx was originally a customisable proxy server. Over the time it has evolved towards being a HTTP processing framework that can equally well serve static or dynamically generated content. Improvements to the data forwarding path will ensure Styx serves both purposes equally well.

A routing object is a reusable Styx software component that implements a HttpHandler interface:
@FunctionalInterface public interface HttpHandler { Eventual<LiveHttpResponse> handle(LiveHttpRequest request, HttpInterceptor.Context context); }

LiveHttpRequest and LiveHttpResponse are HTTP message representations that are delivered in asynchronous, reactive fashion. They have full HTTP headers but a body is a stream of byte chunks that will be delivered asynchronously upon subscription. Styx core can pass these live objects to HttpHandlers immediately as the data starts arriving from the network. It is not necessary to buffer the whole message in memory before handling.

The full HTTP headers are always available in all LiveHttpRequest / LiveHttpResponse objects. The body is a ByteStream Publisher that delivers the body content as it arrives from the network. As Reactive Streams Publisher, consumers must subscribe to it in order to receive this data. Furthermore, Styx HTTP pipeline must have at least one routing object that fully consumes the body data. Otherwise the message body data doesn't get read from the network socket, choking the underlying TCP connection.

Proxy functionality will be a ordinary routing object that relays received messages on to a remote peer, and conversely relays the responses:

Styx forwarding path consist of all Styx classes, components, and code that:

• Receives HTTP traffic from network socket
• Converts the received network traffic to Styx LiveHttpRequest/Response
• Any component or API that manipulate or pass on the live HTTP messages
• A reactive Publisher implementation that produces ByteStream data for live message body
• A reactive Subscriber implementation that consumes HTTP message content from ByteStream.

In particular, our forwarding path components must be fully compliant to the Reactive Streams
specification and equally suitable for proxying and non-proxying routing objects, while paying attention to HTTP protocol requirements.

Objectives

1. Replace any legacy Rx.Java code with Reactor Core. Everywhere. And remove Rx.Java dependency.
   • Rx.Java 1.0 is end-of-life, and cannot be upgraded to 2.0 or newer without major changes. As a team we have decided to go for Reactor Core.
   • Most data processing components, apart from the HTTP pipeline, still use Rx.Java. Last time around it was considered too risky change due to limited development time.
   • Currently Reactor Core and Rx.Java are bridged with a 3rd party library which is a potential source of bugs.
2. For proxying routing objects the upstream and downstream forwarding paths should be mirror images of each others.
3. Support fast responses
   • An HTTP server may respond immediately without having to read the full request. For example it makes sense to send FORBIDDEN immediately after the first HTTP header line without having to waste cycles to read remaining headers or body data.
   • Styx data plane was never designed fast responses in mind. Support for fast responses was implemented retrospectively, and has been a never-ending source of bugs.
4. Optimal use of pooled connections
   • Styx is over eager to close the connections that otherwise could be pooled, especially when fast responses are ### involved.
5. Optimal content streaming
   • Rather limiting back pressure mechanism allows only one in-flight request chunk (Currently fixed to request(1)). The writer components (HttpResponseWriter, HttpRequestOperation) only ever request an additional chunk after Netty has successfully acknowledged the previous chunk.
   • Improved data path will lift this restriction, allowing us to optimise the in-flight queue depth.

Terminology

Netty Channel - for our purposes a Netty channel is an abstraction for a TCP socket. It is associated with a TCP connection, and a Netty pipeline. See the Netty ChannelPipeline documentation.
Inbound - Also known as Upstream in older versions of Netty. This means data coming into Styx, from either the browser side or the origin side.
Outbound - Also known as Downstream in older versions of Netty. This means data going out of Styx, from either the browser side or the origin side.
Data chunk - A chunk of HTTP body content. A typical chunk size is about 8 or 16 kilobytes
Client Side/Server Side - These terms have deliberately been avoided due to confusion, since Styx is a server to the client (browser) and a client to the server (origin).
Upstream/Downstream - These terms have deliberately been avoided due to the confusion between the Netty meaning (Upstream=Inbound, Downstream=Outbound) and the software architecture meaning (Upstream=Client Side, Downstream=Server Side)

Current Request Processing Architecture

NettyToStyxRequestDecoder is a Netty pipeline handler that converts a Netty HTTP message to a Styx LiveHttpRequest. It has a simple Rx.Java Producer that streams HTTP request content over the observable and reactive steams.

• The internal content producer doesn't support full Reactive Stream back-pressure protocol: It ignores the provided N value. It merely provides one content chunk event for each subscriber request.
• The static internal FlowControllingHttpContentProducer class should not be confused with the public class of the same name. The internal class is a simplistic content producer with limited back pressure support, where as the public class is a full-fledged reactive-streams compliant content producer.
• Inbound response data path doesn't have a similar class, this functionality is split between NettyToStyxResponsePropagator and the public FlowControllingHttpContentProducer classes (see the Current Response Processing Architecture below).

HttpPipelineHandler is a Netty pipeline handler that feeds a received request to a Styx HTTP pipeline and waits for the response. It maintains a Finite State Machine (FSM) that represents a state of the associated Netty channel. It helps us to track a progression of received requests and responses sent over the the channel.

NettyConnection has a write method that sends a request over the (origin-bound) channel. On its own it doesn't do much: it merely executes the HttpRequestOperation (see below).

HttpRequestOperation prepares an origin bound channel and writes the request to it.

• Adds (and removes) an idle state handler and a NettyToStyxResponsePropagator channel handlers to the Netty channel pipeline.
• Converts a Styx HTTP Request back to Netty representation.
• It then writes the request in this channel.
• To sum up, it has similar functionality to HttpResponseWriter (see below) but for handling data when communicating to downstream services (i.e. origins).
• Netty channel preparation could be moved to NettyConnection and writing the request could be implemented with a HttpRequestWriter or similar class.
  

Current Response Processing Architecture

NettyToStyxResponsePropagator is a Netty pipeline handler that absorbs an HTTP response from a remote origin. It adapts Netty events to a finite state machine that publishes HTTP response content in a reactive steams compliant manner. A public FlowControllingHttpContentProducer class implements the FSM.

• To ease unit testing, we separated the FSM from the NettyToStyxResponsePropagator to a different class.

FlowControllingHttpContentProducer is a reactive producer for HTTP response content.

• It fully supports RX back pressure protocol.
• It could be re-used for HTTP requests.
• It mediates between content subscriber (HttpResponseWriter, in styx server thread) and event source (HttpResponsePropagator, in styx client thread).

HttpResponseWriter consumes the response and its body stream and writes them to inbound Netty channel.

• Subscribes to and consumes the response body stream and writes it to the inbound Netty channel.
• After each successful write, requests one additional data chunk (using the Rx back pressure protocol) from response content producer.
  

Improved Architecture for Proxy Objects

We will rework the upstream request processing path to make it a mirror image of the present downstream response path.

This means:

• NettyToStyxRequestDecoder will be removed. We will re-use the (public) FlowControllingHttpContentProducer alongside HttpPipelineHandler to provide the same functionality.
• HttpRequestOperation will be removed. The outbound channel will be prepared in NettyConnection and we will introduce a HttpRequestWriter that will write the request and its content stream to the outbound Netty channel. The HttpRequestWriter will be very similar (bar types signatures) from HttpResponseWriter and we will share as much code as possible.
• The downstream response flow remains largely as it is.
• With these changes the upstream response flow will be a mirror image of the downstream flow.

There will be a pair of content producer (FlowControllingHttpContentProducer) and a subscriber (HttpRequest/ResponseWriter) on each side. The pair of producer and subscriber will be co-located enabling them to pass messages to each other. This lets us to deal better with quickly responding origins.

The improved data path will be 100% reactor core, both upstream and downstream. The Rx.Java will be dropped.

Improved Architecture for Non-Proxy Objects

Styx Core doesn't know if the routing object is proxying or not. They both look identical. A typical non-proxying routing object is much simpler. They would use LiveHttpRequest.aggregate() to read in the request content, and respond by converting an immutable HttpResponse object to a live one with HttpRequest.stream() method.

[OwenLindsell]

Related PRs:
#541
#547
#552
#560
#564
#566
#569
#570
#571
#592

[kvosper]

@OwenLindsell informs me that this work is complete.