Fixes #8979 - Jetty 12 - HttpClientTransport network "modes".

* Introduced oej.io.TransportProtocol as the abstraction for the low-level transport of high-level protocols.
Now protocols such as HTTP/1.1 or HTTP/2 can be transported over QUIC, Unix-Domain, memory, and possibly over other low-level custom protocols too.
* Introduced oej.client.Request.transportProtocol(TransportProtocol) to specify TransportProtocol for each request.
* Introduced TransportProtocol to [HTTP2Client|HTTP3Client].connect(...) methods.
* Introduced [Client|Server]QuicConfiguration so that it can be used in other Connectors such as MemoryConnector.
* Introduced oej.server.MemoryConnector and EndPoint.Pipe for memory communication between peers, along with a MemoryTransportProtocol.
* Introduced QuicTransportProtocol as a wrapper for other TransportProtocols, so that QUIC can now also be transported over memory.

Signed-off-by: Simone Bordet <simone.bordet@gmail.com>

documentation/jetty-documentation/src/main/asciidoc/programming-guide/client/http/client-http-transport.adoc

@@ -187,9 +187,11 @@ All the transports can be configured with a `ClientConnector`, the component tha
 
 By default, `ClientConnector` uses TCP networking to send bytes to the server and receive bytes from the server.
 
-When you are using Java 16 or later, `ClientConnector` also support xref:pg-client-io-arch-unix-domain[Unix-Domain sockets], and every transport can be configured to use Unix-Domain sockets instead of TCP networking.
+When you are using Java 16 or later, `ClientConnector` also support xref:pg-client-io-arch-unix-domain[Unix-Domain sockets].
 
-To configure Unix-Domain sockets, you can create a `ClientConnector` instance in the following way:
+Unix-Domain sockets can be used for HTTP/1.1 and HTTP/2 but not for HTTP/3, because HTTP/3 is based on UDP and there is currently no support in Java for UDP over Unix-Domain sockets.
+
+You can send requests to a Unix-Domain sockets server in the following way:
 
 [source,java,indent=0]
 ----

documentation/jetty-documentation/src/main/asciidoc/programming-guide/server/http/server-http-connector.adoc

@@ -20,12 +20,14 @@ The available implementations are:
 
 * `org.eclipse.jetty.server.ServerConnector`, for TCP/IP sockets.
 * `org.eclipse.jetty.unixdomain.server.UnixDomainServerConnector` for Unix-Domain sockets (requires Java 16 or later).
-* `org.eclipse.jetty.quic.server.QuicServerConnector`, for the low-level QUIC protocol.
-* `org.eclipse.jetty.http3.server.HTTP3ServerConnector` for the HTTP/3 protocol.
+* `org.eclipse.jetty.quic.server.QuicServerConnector`, for the low-level QUIC protocol and HTTP/3.
+* `org.eclipse.jetty.server.MemoryConnector`, for memory communication between client and server.
 
-The first two use a `java.nio.channels.ServerSocketChannel` to listen to a socket address and to accept socket connections, while last two use a `java.nio.channels.DatagramChannel`.
+`ServerConnector` and `UnixDomainServerConnector` use a `java.nio.channels.ServerSocketChannel` to listen to a socket address and to accept socket connections.
+`QuicServerConnector` uses a `java.nio.channels.DatagramChannel` to listen to incoming UDP packets.
+`MemoryConnector` uses memory for the communication between client and server, avoiding the use of sockets
 
-Since `ServerConnector` wraps a `ServerSocketChannel`, it can be configured in a similar way, for example the IP port to listen to, the IP address to bind to, etc.:
+Since `ServerConnector` wraps a `ServerSocketChannel`, it can be configured in a similar way, for example the TCP port to listen to, the IP address to bind to, etc.:
 
 [source,java,indent=0]
 ----
@@ -44,13 +46,15 @@ include::../../{doc_code}/org/eclipse/jetty/docs/programming/server/http/HTTPSer
 You can use Unix-Domain sockets support only when you run your server with Java 16 or later.
 ====
 
-`QuicServerConnector` and its extension `HTTP3ServerConnector` wrap a `DatagramChannel` and can be configured in a similar way:
+`QuicServerConnector` wraps a `DatagramChannel` and can be configured in a similar way:
 
 [source,java,indent=0]
 ----
 include::../../{doc_code}/org/eclipse/jetty/docs/programming/server/http/HTTPServerDocs.java[tags=configureConnectorQuic]
 ----
 
+// TODO: add a section on MemoryConnector.
+
 [[pg-server-http-connector-acceptors]]
 ===== Acceptors
 

documentation/jetty-documentation/src/main/java/org/eclipse/jetty/docs/programming/client/ClientConnectorDocs.java

@@ -22,6 +22,7 @@
 import java.util.HashMap;
 import java.util.Map;
 import java.util.concurrent.CompletableFuture;
+import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.Executor;
 import java.util.function.Consumer;
 
@@ -153,7 +154,7 @@ public void onFillable()
         CompletableFuture<CustomConnection> connectionPromise = new Promise.Completable<>();
 
         // Populate the context with the mandatory keys to create and obtain connections.
-        Map<String, Object> context = new HashMap<>();
+        Map<String, Object> context = new ConcurrentHashMap<>();
         context.put(ClientConnector.CLIENT_CONNECTION_FACTORY_CONTEXT_KEY, connectionFactory);
         context.put(ClientConnector.CONNECTION_PROMISE_CONTEXT_KEY, connectionPromise);
         clientConnector.connect(address, context);

documentation/jetty-documentation/src/main/java/org/eclipse/jetty/docs/programming/client/http/HTTPClientDocs.java

@@ -69,6 +69,8 @@
 import org.eclipse.jetty.io.ClientConnectionFactory;
 import org.eclipse.jetty.io.ClientConnector;
 import org.eclipse.jetty.io.Content;
+import org.eclipse.jetty.io.TransportProtocol;
+import org.eclipse.jetty.quic.client.ClientQuicConfiguration;
 import org.eclipse.jetty.util.Callback;
 import org.eclipse.jetty.util.component.LifeCycle;
 import org.eclipse.jetty.util.ssl.SslContextFactory;
@@ -871,8 +873,11 @@ public void http2Transport() throws Exception
     public void http3Transport() throws Exception
     {
         // tag::http3Transport[]
+        // HTTP/3 requires secure communication.
+        SslContextFactory.Client sslContextFactory = new SslContextFactory.Client();
         // The HTTP3Client powers the HTTP/3 transport.
-        HTTP3Client h3Client = new HTTP3Client();
+        ClientQuicConfiguration clientQuicConfig = new ClientQuicConfiguration(sslContextFactory, null);
+        HTTP3Client h3Client = new HTTP3Client(clientQuicConfig);
         h3Client.getQuicConfiguration().setSessionRecvWindow(64 * 1024 * 1024);
 
         // Create and configure the HTTP/3 transport.
@@ -1018,25 +1023,29 @@ public void unixDomain() throws Exception
         // This is the path where the server "listens" on.
         Path unixDomainPath = Path.of("/path/to/server.sock");
 
-        // Creates a ClientConnector that uses Unix-Domain
-        // sockets, not the network, to connect to the server.
-        ClientConnector unixDomainClientConnector = ClientConnector.forUnixDomain(unixDomainPath);
+        // Creates a ClientConnector.
+        ClientConnector clientConnector = new ClientConnector();
 
-        // Use Unix-Domain for HTTP/1.1.
-        HttpClientTransportOverHTTP http1Transport = new HttpClientTransportOverHTTP(unixDomainClientConnector);
+        // You can use Unix-Domain for HTTP/1.1.
+        HttpClientTransportOverHTTP http1Transport = new HttpClientTransportOverHTTP(clientConnector);
 
         // You can use Unix-Domain also for HTTP/2.
-        HTTP2Client http2Client = new HTTP2Client(unixDomainClientConnector);
+        HTTP2Client http2Client = new HTTP2Client(clientConnector);
         HttpClientTransportOverHTTP2 http2Transport = new HttpClientTransportOverHTTP2(http2Client);
 
-        // You can also use UnixDomain for the dynamic transport.
+        // You can use Unix-Domain also for the dynamic transport.
         ClientConnectionFactory.Info http1 = HttpClientConnectionFactory.HTTP11;
         ClientConnectionFactoryOverHTTP2.HTTP2 http2 = new ClientConnectionFactoryOverHTTP2.HTTP2(http2Client);
-        HttpClientTransportDynamic dynamicTransport = new HttpClientTransportDynamic(unixDomainClientConnector, http1, http2);
+        HttpClientTransportDynamic dynamicTransport = new HttpClientTransportDynamic(clientConnector, http1, http2);
 
         // Choose the transport you prefer for HttpClient, for example the dynamic transport.
         HttpClient httpClient = new HttpClient(dynamicTransport);
         httpClient.start();
+
+        // Make a request and specify that you want to send it over Unix-Domain.
+        ContentResponse response = httpClient.newRequest("jetty.org", 80)
+            .transportProtocol(new TransportProtocol.TCPUnix(unixDomainPath))
+            .send();
         // end::unixDomain[]
     }
 }

documentation/jetty-documentation/src/main/java/org/eclipse/jetty/docs/programming/client/http3/HTTP3ClientDocs.java

@@ -33,6 +33,8 @@
 import org.eclipse.jetty.http3.client.HTTP3Client;
 import org.eclipse.jetty.http3.frames.DataFrame;
 import org.eclipse.jetty.http3.frames.HeadersFrame;
+import org.eclipse.jetty.quic.client.ClientQuicConfiguration;
+import org.eclipse.jetty.util.ssl.SslContextFactory;
 
 import static java.lang.System.Logger.Level.INFO;
 
@@ -43,7 +45,8 @@ public void start() throws Exception
     {
         // tag::start[]
         // Instantiate HTTP3Client.
-        HTTP3Client http3Client = new HTTP3Client();
+        SslContextFactory.Client sslContextFactory = new SslContextFactory.Client();
+        HTTP3Client http3Client = new HTTP3Client(new ClientQuicConfiguration(sslContextFactory, null));
 
         // Configure HTTP3Client, for example:
         http3Client.getHTTP3Configuration().setStreamIdleTimeout(15000);
@@ -55,7 +58,8 @@ public void start() throws Exception
 
     public void stop() throws Exception
     {
-        HTTP3Client http3Client = new HTTP3Client();
+        SslContextFactory.Client sslContextFactory = new SslContextFactory.Client();
+        HTTP3Client http3Client = new HTTP3Client(new ClientQuicConfiguration(sslContextFactory, null));
         http3Client.start();
         // tag::stop[]
         // Stop HTTP3Client.
@@ -65,7 +69,8 @@ public void stop() throws Exception
 
     public void connect() throws Exception
     {
-        HTTP3Client http3Client = new HTTP3Client();
+        SslContextFactory.Client sslContextFactory = new SslContextFactory.Client();
+        HTTP3Client http3Client = new HTTP3Client(new ClientQuicConfiguration(sslContextFactory, null));
         http3Client.start();
         // tag::connect[]
         // Address of the server's port.
@@ -83,7 +88,8 @@ public void connect() throws Exception
 
     public void configure() throws Exception
     {
-        HTTP3Client http3Client = new HTTP3Client();
+        SslContextFactory.Client sslContextFactory = new SslContextFactory.Client();
+        HTTP3Client http3Client = new HTTP3Client(new ClientQuicConfiguration(sslContextFactory, null));
         http3Client.start();
 
         // tag::configure[]
@@ -105,7 +111,8 @@ public Map<Long, Long> onPreface(Session session)
 
     public void newStream() throws Exception
     {
-        HTTP3Client http3Client = new HTTP3Client();
+        SslContextFactory.Client sslContextFactory = new SslContextFactory.Client();
+        HTTP3Client http3Client = new HTTP3Client(new ClientQuicConfiguration(sslContextFactory, null));
         http3Client.start();
         // tag::newStream[]
         SocketAddress serverAddress = new InetSocketAddress("localhost", 8444);
@@ -130,7 +137,8 @@ public void newStream() throws Exception
 
     public void newStreamWithData() throws Exception
     {
-        HTTP3Client http3Client = new HTTP3Client();
+        SslContextFactory.Client sslContextFactory = new SslContextFactory.Client();
+        HTTP3Client http3Client = new HTTP3Client(new ClientQuicConfiguration(sslContextFactory, null));
         http3Client.start();
         // tag::newStreamWithData[]
         SocketAddress serverAddress = new InetSocketAddress("localhost", 8444);
@@ -173,7 +181,8 @@ public void newStreamWithData() throws Exception
 
     public void responseListener() throws Exception
     {
-        HTTP3Client http3Client = new HTTP3Client();
+        SslContextFactory.Client sslContextFactory = new SslContextFactory.Client();
+        HTTP3Client http3Client = new HTTP3Client(new ClientQuicConfiguration(sslContextFactory, null));
         http3Client.start();
         SocketAddress serverAddress = new InetSocketAddress("localhost", 8444);
         CompletableFuture<Session.Client> sessionCF = http3Client.connect(serverAddress, new Session.Client.Listener() {});
@@ -231,7 +240,8 @@ private void process(ByteBuffer byteBuffer)
 
     public void reset() throws Exception
     {
-        HTTP3Client http3Client = new HTTP3Client();
+        SslContextFactory.Client sslContextFactory = new SslContextFactory.Client();
+        HTTP3Client http3Client = new HTTP3Client(new ClientQuicConfiguration(sslContextFactory, null));
         http3Client.start();
         SocketAddress serverAddress = new InetSocketAddress("localhost", 8444);
         CompletableFuture<Session.Client> sessionCF = http3Client.connect(serverAddress, new Session.Client.Listener() {});

documentation/jetty-documentation/src/main/java/org/eclipse/jetty/docs/programming/client/websocket/WebSocketClientDocs.java

@@ -25,6 +25,7 @@
 import org.eclipse.jetty.http2.client.HTTP2Client;
 import org.eclipse.jetty.http2.client.transport.ClientConnectionFactoryOverHTTP2;
 import org.eclipse.jetty.http2.client.transport.HttpClientTransportOverHTTP2;
+import org.eclipse.jetty.io.ClientConnector;
 import org.eclipse.jetty.util.component.LifeCycle;
 import org.eclipse.jetty.websocket.api.Session;
 import org.eclipse.jetty.websocket.client.ClientUpgradeRequest;
@@ -124,8 +125,9 @@ public void connectHTTP2Dynamic() throws Exception
     {
         // tag::connectHTTP2Dynamic[]
         // Use the dynamic HTTP/2 transport for HttpClient.
-        HTTP2Client http2Client = new HTTP2Client();
-        HttpClient httpClient = new HttpClient(new HttpClientTransportDynamic(new ClientConnectionFactoryOverHTTP2.HTTP2(http2Client)));
+        ClientConnector clientConnector = new ClientConnector();
+        HTTP2Client http2Client = new HTTP2Client(clientConnector);
+        HttpClient httpClient = new HttpClient(new HttpClientTransportDynamic(clientConnector, new ClientConnectionFactoryOverHTTP2.HTTP2(http2Client)));
 
         // Create and start WebSocketClient.
         WebSocketClient webSocketClient = new WebSocketClient(httpClient);

documentation/jetty-documentation/src/main/java/org/eclipse/jetty/docs/programming/server/http/HTTPServerDocs.java

@@ -46,8 +46,9 @@
 import org.eclipse.jetty.http2.server.HTTP2CServerConnectionFactory;
 import org.eclipse.jetty.http2.server.HTTP2ServerConnectionFactory;
 import org.eclipse.jetty.http3.server.HTTP3ServerConnectionFactory;
-import org.eclipse.jetty.http3.server.HTTP3ServerConnector;
 import org.eclipse.jetty.io.Content;
+import org.eclipse.jetty.quic.server.QuicServerConnector;
+import org.eclipse.jetty.quic.server.ServerQuicConfiguration;
 import org.eclipse.jetty.rewrite.handler.CompactPathRule;
 import org.eclipse.jetty.rewrite.handler.RedirectRegexRule;
 import org.eclipse.jetty.rewrite.handler.RewriteHandler;
@@ -223,8 +224,10 @@ public void configureConnectorQuic() throws Exception
         sslContextFactory.setKeyStorePath("/path/to/keystore");
         sslContextFactory.setKeyStorePassword("secret");
 
-        // Create an HTTP3ServerConnector instance.
-        HTTP3ServerConnector connector = new HTTP3ServerConnector(server, sslContextFactory, new HTTP3ServerConnectionFactory());
+        // Create a QuicServerConnector instance.
+        Path pemWorkDir = Path.of("/path/to/pem/dir");
+        ServerQuicConfiguration serverQuicConfig = new ServerQuicConfiguration(sslContextFactory, pemWorkDir);
+        QuicServerConnector connector = new QuicServerConnector(server, serverQuicConfig, new HTTP3ServerConnectionFactory(serverQuicConfig));
 
         // The port to listen to.
         connector.setPort(8080);
@@ -289,8 +292,9 @@ public void sameRandomPort() throws Exception
         server.addConnector(plainConnector);
 
         // Third, create the connector for HTTP/3.
-        HTTP3ServerConnectionFactory http3 = new HTTP3ServerConnectionFactory(secureConfig);
-        HTTP3ServerConnector http3Connector = new HTTP3ServerConnector(server, sslContextFactory, http3);
+        Path pemWorkDir = Path.of("/path/to/pem/dir");
+        ServerQuicConfiguration serverQuicConfig = new ServerQuicConfiguration(sslContextFactory, pemWorkDir);
+        QuicServerConnector http3Connector = new QuicServerConnector(server, serverQuicConfig, new HTTP3ServerConnectionFactory(serverQuicConfig));
         server.addConnector(http3Connector);
 
         // Set up a listener so that when the secure connector starts,
@@ -493,12 +497,12 @@ public void h3() throws Exception
         httpConfig.addCustomizer(new SecureRequestCustomizer());
 
         // Create and configure the HTTP/3 connector.
-        HTTP3ServerConnector connector = new HTTP3ServerConnector(server, sslContextFactory, new HTTP3ServerConnectionFactory(httpConfig));
+        // It is mandatory to configure the PEM directory.
+        Path pemWorkDir = Path.of("/path/to/pem/dir");
+        ServerQuicConfiguration serverQuicConfig = new ServerQuicConfiguration(sslContextFactory, pemWorkDir);
+        QuicServerConnector connector = new QuicServerConnector(server, serverQuicConfig, new HTTP3ServerConnectionFactory(serverQuicConfig));
         connector.setPort(843);
 
-        // It is mandatory to set the PEM directory.
-        connector.getQuicConfiguration().setPemWorkDirectory(Path.of("/path/to/pem/dir"));
-
         server.addConnector(connector);
 
         server.start();

documentation/jetty-documentation/src/main/java/org/eclipse/jetty/docs/programming/server/http3/HTTP3ServerDocs.java

@@ -15,6 +15,7 @@
 
 import java.net.SocketAddress;
 import java.nio.ByteBuffer;
+import java.nio.file.Path;
 import java.util.HashMap;
 import java.util.Map;
 import java.util.concurrent.RejectedExecutionException;
@@ -29,8 +30,9 @@
 import org.eclipse.jetty.http3.api.Stream;
 import org.eclipse.jetty.http3.frames.DataFrame;
 import org.eclipse.jetty.http3.frames.HeadersFrame;
-import org.eclipse.jetty.http3.server.HTTP3ServerConnector;
 import org.eclipse.jetty.http3.server.RawHTTP3ServerConnectionFactory;
+import org.eclipse.jetty.quic.server.QuicServerConnector;
+import org.eclipse.jetty.quic.server.ServerQuicConfiguration;
 import org.eclipse.jetty.server.Server;
 import org.eclipse.jetty.util.ssl.SslContextFactory;
 
@@ -53,14 +55,16 @@ public void setup() throws Exception
         // The listener for session events.
         Session.Server.Listener sessionListener = new Session.Server.Listener() {};
 
+        ServerQuicConfiguration quicConfiguration = new ServerQuicConfiguration(sslContextFactory, Path.of("/path/to/pem/dir"));
+        // Configure the max number of requests per QUIC connection.
+        quicConfiguration.setMaxBidirectionalRemoteStreams(1024);
+
         // Create and configure the RawHTTP3ServerConnectionFactory.
-        RawHTTP3ServerConnectionFactory http3 = new RawHTTP3ServerConnectionFactory(sessionListener);
+        RawHTTP3ServerConnectionFactory http3 = new RawHTTP3ServerConnectionFactory(quicConfiguration, sessionListener);
         http3.getHTTP3Configuration().setStreamIdleTimeout(15000);
 
-        // Create and configure the HTTP3ServerConnector.
-        HTTP3ServerConnector connector = new HTTP3ServerConnector(server, sslContextFactory, http3);
-        // Configure the max number of requests per QUIC connection.
-        connector.getQuicConfiguration().setMaxBidirectionalRemoteStreams(1024);
+        // Create and configure the QuicServerConnector.
+        QuicServerConnector connector = new QuicServerConnector(server, quicConfiguration, http3);
 
         // Add the Connector to the Server.
         server.addConnector(connector);

jetty-core/jetty-client/src/main/java/org/eclipse/jetty/client/AbstractConnectorHttpClientTransport.java

@@ -70,7 +70,8 @@ public void connect(SocketAddress address, Map<String, Object> context)
         @SuppressWarnings("unchecked")
         Promise<Connection> promise = (Promise<Connection>)context.get(HTTP_CONNECTION_PROMISE_CONTEXT_KEY);
         context.put(ClientConnector.CONNECTION_PROMISE_CONTEXT_KEY, Promise.from(ioConnection -> {}, promise::failed));
-        connector.connect(address, context);
+        context.put(ClientConnector.CLIENT_CONNECTOR_CONTEXT_KEY, connector);
+        destination.getOrigin().getTransportProtocol().connect(address, context);
     }
 
     @Override