Fix Jetty multipart client support with Java 25

[sdeleuze]

While working on #35530, I figured out that with Jetty and Java 25, MultipartWebClientIntegrationTests#transferTo is always broken.

MultipartWebClientIntegrationTests is known to be flaky, so there may be something to fix in the client multipart support, but that's now straightforward, so this issue is a reminder for this test that will be skipped for now.

[Lublanski]

Hi @sdeleuze , I’d like to take a look at this issue.

My plan is:

1. Switch to the java25 branch and run the MultipartWebClientIntegrationTests with JDK 25.
2. Reproduce the failure of #transferTo and investigate the root cause (whether it’s in WebClient’s multipart support or related to Jetty/Java 25).
3. Propose a fix or at least narrow down the problem with a reproducible case.

Is it ok if I work on this?

[sdeleuze]

Hi, yes feel free to have a deeper look. Please share your findings and potential link to your branch here initially, not in a PR.

What java25 branch are you referring to?

[Lublanski]

@sdeleuze
Thanks for the clarification!

I’ll start from the main branch in my fork, create a dedicated branch for investigation, and run the MultipartWebClientIntegrationTests with JDK 25 + Jetty to reproduce the failure.

I’ll share my findings and a link to my branch here before opening any PR.

[Lublanski]

Hi @sdeleuze
I was able to reproduce the failure locally.

With java 24, for example, the test passes, but with java 25 it consistently fails.

As you already described, the failure is on MultipartWebClientIntegrationTests#transferTo when using Jetty, with the following error:

WebClientResponseException: 200 OK ... but response failed with cause: 
reactor.netty.http.client.PrematureCloseException: Connection prematurely closed DURING response

This suggests that the response starts correctly (200 OK), but the connection is closed before the body is fully consumed.

I’ll continue investigating whether this comes from Jetty on JDK 25, or from Reactor Netty’s handling of the response, and will share further findings.

[Lublanski]

Hi @sdeleuze , a quick update on my investigation:

With JDK 25, I noticed the following debug log when running MultipartWebClientIntegrationTests#transferTo with Jetty:

13:46:02.707 [InnocuousThread-2] DEBUG o.s.h.s.r.ServletHttpHandlerAdapter - [4048aa2] AsyncEvent onError: java.lang.SecurityException: setContextClassLoader

This seems to happen on InnocuousThread-1, and may explain why the connection is being closed prematurely on the client side (PrematureCloseException). With JDK 24, this does not occur.

I’ll keep digging to confirm whether this is caused by Jetty’s use of setContextClassLoader under JDK 25 restrictions.

[Lublanski]

Hi again @sdeleuze, quick update on this.

I was able to capture the full stacktrace of the failure when running MultipartWebClientIntegrationTests#transferTo with JDK 25 and Jetty:

java.lang.SecurityException: setContextClassLoader
	at java.base/jdk.internal.misc.InnocuousThread.setContextClassLoader(InnocuousThread.java:130)
	at org.eclipse.jetty.server.handler.ContextHandler.exitScope(ContextHandler.java:754)
	at org.eclipse.jetty.server.handler.ContextHandler$ScopedContext.run(ContextHandler.java:1647)
	at org.eclipse.jetty.server.handler.ContextResponse$1.succeeded(ContextResponse.java:41)
	at org.eclipse.jetty.util.thread.SerializedInvoker$Link.run(SerializedInvoker.java:274)
	at org.eclipse.jetty.util.thread.SerializedInvoker.run(SerializedInvoker.java:174)
	at org.eclipse.jetty.server.internal.HttpChannelState$ChannelResponse.succeeded(HttpChannelState.java:1359)
	at org.eclipse.jetty.server.internal.HttpConnection$SendCallback.onCompleteSuccess(HttpConnection.java:1022)
	at org.eclipse.jetty.util.IteratingCallback.onCompleted(IteratingCallback.java:264)
	at org.eclipse.jetty.server.internal.HttpConnection$SendCallback.onCompleted(HttpConnection.java:797)
	at org.eclipse.jetty.util.IteratingCallback.doCompleteSuccess(IteratingCallback.java:276)
	at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:501)
	at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:354)
	at org.eclipse.jetty.server.internal.HttpConnection$HttpStreamOverHTTP1.send(HttpConnection.java:1578)
	at org.eclipse.jetty.server.HttpStream$Wrapper.send(HttpStream.java:197)
	at org.eclipse.jetty.server.internal.HttpChannelState$ChannelResponse.write(HttpChannelState.java:1333)
	at org.eclipse.jetty.server.Response$Wrapper.write(Response.java:836)
	at org.eclipse.jetty.server.handler.ContextResponse.write(ContextResponse.java:56)
	at org.eclipse.jetty.ee11.servlet.ServletContextResponse.write(ServletContextResponse.java:225)
	at org.eclipse.jetty.ee11.servlet.HttpOutput.channelWrite(HttpOutput.java:248)
	at org.eclipse.jetty.ee11.servlet.HttpOutput$AsyncWrite.process(HttpOutput.java:1643)
	at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:377)
	at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:354)
	at org.eclipse.jetty.ee11.servlet.HttpOutput.write(HttpOutput.java:997)
	at org.springframework.http.server.reactive.ServletServerHttpResponse.writeToOutputStream(ServletServerHttpResponse.java:217)
	at org.springframework.http.server.reactive.ServletServerHttpResponse$ResponseBodyProcessor.write(ServletServerHttpResponse.java:383)
	at org.springframework.http.server.reactive.ServletServerHttpResponse$ResponseBodyProcessor.write(ServletServerHttpResponse.java:353)
	at org.springframework.http.server.reactive.AbstractListenerWriteProcessor$State$3.onWritePossible(AbstractListenerWriteProcessor.java:392)
	at org.springframework.http.server.reactive.AbstractListenerWriteProcessor.onWritePossible(AbstractListenerWriteProcessor.java:157)
	at org.springframework.http.server.reactive.AbstractListenerWriteProcessor.writeIfPossible(AbstractListenerWriteProcessor.java:311)
	at org.springframework.http.server.reactive.AbstractListenerWriteProcessor.changeStateToReceived(AbstractListenerWriteProcessor.java:290)
	at org.springframework.http.server.reactive.AbstractListenerWriteProcessor$State$2.onNext(AbstractListenerWriteProcessor.java:371)
	at org.springframework.http.server.reactive.AbstractListenerWriteProcessor.onNext(AbstractListenerWriteProcessor.java:118)
	at org.springframework.http.server.reactive.ChannelSendOperator$WriteBarrier.emitCachedSignals(ChannelSendOperator.java:312)
	at org.springframework.http.server.reactive.ChannelSendOperator$WriteBarrier.request(ChannelSendOperator.java:282)
	at org.springframework.http.server.reactive.AbstractListenerWriteProcessor$State$1.onSubscribe(AbstractListenerWriteProcessor.java:348)
	at org.springframework.http.server.reactive.AbstractListenerWriteProcessor.onSubscribe(AbstractListenerWriteProcessor.java:110)
	at org.springframework.http.server.reactive.ChannelSendOperator$WriteBarrier.subscribe(ChannelSendOperator.java:358)
	at org.springframework.http.server.reactive.AbstractListenerWriteFlushProcessor$State$2.onNext(AbstractListenerWriteFlushProcessor.java:292)
	at org.springframework.http.server.reactive.AbstractListenerWriteFlushProcessor.onNext(AbstractListenerWriteFlushProcessor.java:119)
	at org.springframework.http.server.reactive.AbstractListenerWriteFlushProcessor.onNext(AbstractListenerWriteFlushProcessor.java:43)
	at reactor.core.publisher.StrictSubscriber.onNext(StrictSubscriber.java:89)
	at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2570)
	at reactor.core.publisher.StrictSubscriber.onSubscribe(StrictSubscriber.java:77)
	at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55)
	at reactor.core.publisher.Mono.subscribe(Mono.java:4571)
	at org.springframework.http.server.reactive.AbstractListenerServerHttpResponse.lambda$writeAndFlushWithInternal$0(AbstractListenerServerHttpResponse.java:64)
	at reactor.core.publisher.MonoFromPublisher.subscribe(MonoFromPublisher.java:64)
	at reactor.core.publisher.Mono.subscribe(Mono.java:4571)
	at reactor.core.publisher.FluxConcatArray$ConcatArraySubscriber.onComplete(FluxConcatArray.java:238)
	at reactor.core.publisher.FluxConcatArray.subscribe(FluxConcatArray.java:79)
	at reactor.core.publisher.Mono.subscribe(Mono.java:4571)
	at org.springframework.http.server.reactive.ChannelSendOperator$WriteBarrier.onNext(ChannelSendOperator.java:187)
	at reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:200)
	at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122)
	at reactor.core.publisher.FluxConcatMapNoPrefetch$FluxConcatMapNoPrefetchSubscriber.innerNext(FluxConcatMapNoPrefetch.java:259)
	at reactor.core.publisher.FluxConcatMap$ConcatMapInner.onNext(FluxConcatMap.java:865)
	at reactor.core.publisher.MonoFlatMap$FlatMapMain.secondComplete(MonoFlatMap.java:245)
	at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:305)
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.complete(MonoIgnoreThen.java:294)
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onNext(MonoIgnoreThen.java:188)
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.subscribeNext(MonoIgnoreThen.java:237)
	at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onComplete(MonoIgnoreThen.java:204)
	at reactor.core.publisher.MonoCreate$DefaultMonoSink.success(MonoCreate.java:144)
	at reactor.core.publisher.LambdaSubscriber.onComplete(LambdaSubscriber.java:132)
	at reactor.core.publisher.FluxCreate$BaseSink.complete(FluxCreate.java:465)
	at reactor.core.publisher.FluxCreate$BufferAsyncSink.drain(FluxCreate.java:871)
	at reactor.core.publisher.FluxCreate$BufferAsyncSink.complete(FluxCreate.java:819)
	at reactor.core.publisher.FluxCreate$SerializedFluxSink.drainLoop(FluxCreate.java:249)
	at reactor.core.publisher.FluxCreate$SerializedFluxSink.drain(FluxCreate.java:215)
	at reactor.core.publisher.FluxCreate$SerializedFluxSink.complete(FluxCreate.java:206)
	at org.springframework.core.io.buffer.DataBufferUtils$WriteCompletionHandler.completed(DataBufferUtils.java:1259)
	at org.springframework.core.io.buffer.DataBufferUtils$WriteCompletionHandler.completed(DataBufferUtils.java:1177)
	at java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:121)
	at java.base/sun.nio.ch.SimpleAsynchronousFileChannelImpl$3.run(SimpleAsynchronousFileChannelImpl.java:410)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1090)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:614)
	at java.base/java.lang.Thread.run(Thread.java:1474)
	at java.base/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:148)
14:07:26.035 [reactor-http-epoll-2] DEBUG o.s.w.r.f.c.ExchangeFunctions - [48d31d25] [4d59a85f-1, L:/127.0.0.1:41416 - R:localhost/127.0.0.1:39673] Response 200 OK
14:07:26.083 [reactor-http-epoll-2] DEBUG o.s.c.c.StringDecoder - [48d31d25] [4d59a85f-1, L:/127.0.0.1:41416 - R:localhost/127.0.0.1:39673] Decoded "/tmp/MultipartIntegrationTests12650419040079562514foo.txt"
14:07:26.281 [Test worker] DEBUG o.s.w.t.h.s.r.b.JettyHttpServer - Stopping JettyHttpServer...
14:07:26.287 [Test worker] DEBUG o.s.w.t.h.s.r.b.JettyHttpServer - Server stopped (0 millis).

So the root cause is not in Spring, but in Jetty.

* Jetty calls Thread.setContextClassLoader inside an InnocuousThread.

* With JDK 25 this is no longer permitted and results in a SecurityException.

* This aborts the async response, which is why the client side sees a PrematureCloseException.

This seems to be a Jetty + JDK 25 incompatibility. The Spring test just exposes it.
Next step would likely be to raise this upstream with Jetty so they can adjust their use of setContextClassLoader for JDK 25.

[sdeleuze]

@Lublanski Thanks for digging more, could you please create a related issue on Jetty side and share the link here?

[Lublanski]

@sdeleuze I've created a bug and I'm looking at the jetty code to try to fix it. I don't know if I'll be able to, but I'll try.

Here's the issue link: jetty/jetty.project#13666

[Lublanski]

@sdeleuze Good morning, I apparently managed to fix the issue locally.

I'll open the PR in the Jetty repository and wait for the review. But locally, the test started passing every time.

Here's the PR link for the fix: jetty/jetty.project#13669