Fix HttpClient resource leak in HTTP transports

[TsengX]

This pull request addresses the HttpClient resource leak issue described in #620.

Fixes #620

• Add support for external HttpClient instances in HttpClientStreamableHttpTransport and HttpClientSseClientTransport builders
• Implement proper HttpClient resource cleanup using reflection to close SelectorManager threads
• Add shouldCloseHttpClient flag to control resource management lifecycle
• Prevent thread leaks caused by unclosed HttpClient instances created via HttpClient.Builder.build()
• Add comprehensive tests for external HttpClient usage and resource cleanup

Fixes thread accumulation issue where HttpClient-xxxx-SelectorManager threads would continuously grow, leading to memory exhaustion. This addresses the underlying JDK issue documented in JDK-8308364.

Related: https://bugs.openjdk.org/browse/JDK-8308364

Motivation and Context

The HttpClientStreamableHttpTransport and HttpClientSseClientTransport classes create new HttpClient instances via HttpClient.Builder.build() without properly managing their lifecycle. This leads to accumulation of HttpClient-xxxx-SelectorManager threads that are never cleaned up, eventually causing memory exhaustion.

Root cause analysis traced this to the OpenJDK 17 HttpClientImpl implementation (JDK-8308364), where each HttpClient spawns dedicated SelectorManager threads for network I/O but lacks public APIs for proper resource cleanup.

This change introduces two solutions:

1. External HttpClient support - Allow users to provide pre-created HttpClient instances for resource sharing across multiple transports
2. Automatic resource cleanup - Properly clean up internally-created HttpClient resources using reflection to access internal components

References

• JDK-8308364: HttpClient SelectorManager thread leak
• Related discussion: HttpClient Selector Manager Threads

How Has This Been Tested?

• Added tests for external HttpClient usage in HttpClientStreamableHttpSyncClientTests and HttpSseMcpSyncClientTests
• Updated existing transport test to verify proper resource cleanup during graceful shutdown
• Verified backward compatibility with existing test suite
• All existing tests continue to pass without modification

Breaking Changes

No breaking changes. This is an additive change that maintains full API compatibility. Existing code will continue to work as before, but will now benefit from proper resource management and the option to use external HttpClient instances.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update

Checklist

• I have read the MCP Documentation
• My code follows the repository's style guidelines
• New and existing tests pass locally
• I have added appropriate error handling
• I have added or updated documentation as needed

Additional context

[Kehrlann]

Hi @TsengX 👋

Thanks for the report, and for the proposed fix. A few notes on this PR:

1. On the HttpClientImpl, from JDK 21 onwards, there is a #close() method, and we should try to use this is available:

   for (var method : httpClient.getClass().getMethods()) {
       if (method.getName().equals("close")) {
           method.invoke(httpClient);
           return;
       }
   }

   We'd only fall back to reflection if there's no close.
2. Do you have a strong case for passing an HTTP client from the outside? It'd like to keep the API surface smaller, and not have conflicting HttpClient.Builder vs HttpClient.
3. If there's a strong case for the point above, I would prefer having a Consumer<HttpClient> in the builder for "on close" operations rather than a boolean. It would default to the "close client resources" implementation you proposed (or calling close if available).

[Kehrlann]

[issue] This test passes even if the client is closed.

[Kehrlann]

[suggestion] We should try to make an HTTP request here and ensure the client is closed.

[TsengX]

Hi @TsengX 👋

Thanks for the report, and for the proposed fix. A few notes on this PR:

1. On the HttpClientImpl, from JDK 21 onwards, there is a #close() method, and we should try to use this is available:

   for (var method : httpClient.getClass().getMethods()) {
       if (method.getName().equals("close")) {
           method.invoke(httpClient);
           return;
       }
   }

   We'd only fall back to reflection if there's no close.

2. Do you have a strong case for passing an HTTP client from the outside? It'd like to keep the API surface smaller, and not have conflicting HttpClient.Builder vs HttpClient.

3. If there's a strong case for the point above, I would prefer having a Consumer<HttpClient> in the builder for "on close" operations rather than a boolean. It would default to the "close client resources" implementation you proposed (or calling close if available).

Hi @Kehrlann,

Thank you for the detailed feedback! After reconsidering the design, I agree with your concerns about API complexity.

Addressing your points:

1. JDK 21+ close() method: Absolutely! I'll implement the fallback strategy you suggested.

2. Rationale for external HttpClient support: In enterprise AI Agent platform development scenarios, we often create 50+ (sometimes even 100+) MCP connections. Creating a shared HttpClient instance at application startup avoids duplicate resource initialization and connection pool setup, with proper cleanup only when the application lifecycle ends.

3. Regarding Consumer approach: I agree this is an elegant design. However, I have concerns about your suggestion that "it would default to the 'close client resources' implementation." If this means the external HttpClient gets closed when transport.closeGracefully() is called, it could be problematic for the enterprise use case mentioned above. When one transport closes, it would shut down a globally shared HttpClient that may still be in use by other parts of the application. This is why I believe a boolean flag might be more appropriate to explicitly control whether the SDK should manage the HttpClient lifecycle.

About Test improvements: I'll add proper HTTP requests to verify client state and resource cleanup.

I'd appreciate any suggestions or feedback on this approach!

[Kehrlann]

Thanks for the insights @TsengX . I'm OK with injecting a fully built HttpClient.

I propose we do the follwing:

1. Add an .httpClient method to both builders.
2. Remove .clientBuilder, .customizeClient and .connectTimeout methods entirely.
3. Add a .onCloseClient(Consumer<HttpClient>) which defaults to your reflection closing.

---

However, there's an unresolved issue here. The current closeGracefully() implementation actually creates a new session. So with the current implementation, it's possible to do the following:

HttpClientStreamableHttpTransport transport = ...;
transport.sendMessage(...)
    .then(transport.closeGracefully())
    .then(transport.sendMessage(...));

And it works. With you proposed change, this would not work anymore ... But I think the proposed implementation respects the contract from McpTransport better. Let me discuss with the team.