Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ApacheHttpChannels exposes a Closeable instance #430

Merged
merged 11 commits into from
Feb 25, 2020
6 changes: 6 additions & 0 deletions changelog/@unreleased/pr-430.v2.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
type: improvement
improvement:
description: ApacheHttpChannels exposes a Closeable instance, allowing us to release
resources when the client is no longer needed.
links:
- https://github.com/palantir/dialogue/pull/430
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
*/
package com.palantir.dialogue.hc4;

import com.google.common.collect.ImmutableList;
import com.google.common.collect.ImmutableSet;
import com.google.common.primitives.Ints;
import com.palantir.conjure.java.api.config.service.BasicCredentials;
Expand All @@ -40,6 +39,7 @@
import java.util.Queue;
import java.util.Set;
import java.util.concurrent.TimeUnit;
import java.util.stream.Collectors;
import javax.annotation.Nullable;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSocketFactory;
Expand Down Expand Up @@ -75,15 +75,26 @@ public final class ApacheHttpClientChannels {

private ApacheHttpClientChannels() {}

public static Channel create(ClientConfiguration conf, UserAgent baseAgent) {
public static Channel create(ClientConfiguration conf, UserAgent agent) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should all channel implementations be closeable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually started with this, but I think it will get hella confusing in the scenario where our NodeSelectionStrategy channels contain a bunch of internal channels and possibly want to live reload a new uri... I think it's clearer to only make things closeable that actually directly hold resources, and if they don't hold resources then they can just be thrown away.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think queuedchannel is the only other piece that holds resources

CloseableHttpClient client = createCloseableHttpClient(conf);
List<Channel> channels =
conf.uris().stream().map(uri -> createSingleUri(uri, client)).collect(Collectors.toList());

return Channels.create(channels, agent, conf);
}

public static Channel createSingleUri(String uri, CloseableHttpClient client) {
return BlockingChannelAdapter.of(new ApacheHttpClientBlockingChannel(client, url(uri)));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another possibility is that we create a new client for each URI for additional isolation and avoiding contention in the connection pool.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This API gives us the freedom to do that from our WC facade, because we can choose to pass in the same closeablehttpclient or not.

}

public static CloseableHttpClient createCloseableHttpClient(ClientConfiguration conf) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A little bit wonky in public api since it's not clear what functionality the returned client provides compared to our channel wrappers. Will we cause bugs if we move functionality from the client into a channel?
At the same time, this should be an implementation dependency of components which simply provide channels, so it's probably fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so I wanted to just convey the idea of some closeable shared resource - would you prefer if I wrap it in some opaque type so that people can't use it directly?

Preconditions.checkArgument(
!conf.fallbackToCommonNameVerification(), "fallback-to-common-name-verification is not supported");
Preconditions.checkArgument(!conf.meshProxy().isPresent(), "Mesh proxy is not supported");

long socketTimeoutMillis =
Math.max(conf.readTimeout().toMillis(), conf.writeTimeout().toMillis());
int connectTimeout = Ints.checkedCast(conf.connectTimeout().toMillis());
// TODO(ckozak): close resources?
HttpClientBuilder builder = HttpClients.custom()
.setDefaultRequestConfig(RequestConfig.custom()
.setSocketTimeout(Ints.checkedCast(socketTimeoutMillis))
Expand Down Expand Up @@ -128,12 +139,8 @@ public static Channel create(ClientConfiguration conf, UserAgent baseAgent) {
.register(AuthSchemes.BASIC, new BasicSchemeFactory())
.build());
});
CloseableHttpClient client = builder.build();
ImmutableList<Channel> channels = conf.uris().stream()
.map(uri -> BlockingChannelAdapter.of(new ApacheHttpClientBlockingChannel(client, url(uri))))
.collect(ImmutableList.toImmutableList());

return Channels.create(channels, baseAgent, conf);
return builder.build();
}

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,72 @@
*/
package com.palantir.dialogue.hc4;

import static org.assertj.core.api.Assertions.assertThatThrownBy;

import com.google.common.util.concurrent.Futures;
import com.google.common.util.concurrent.ListenableFuture;
import com.palantir.conjure.java.api.config.service.UserAgent;
import com.palantir.conjure.java.client.config.ClientConfiguration;
import com.palantir.dialogue.AbstractChannelTest;
import com.palantir.dialogue.Channel;
import com.palantir.dialogue.Endpoint;
import com.palantir.dialogue.HttpMethod;
import com.palantir.dialogue.Request;
import com.palantir.dialogue.Response;
import com.palantir.dialogue.TestConfigurations;
import com.palantir.dialogue.UrlBuilder;
import java.net.UnknownHostException;
import java.util.Map;
import org.apache.http.impl.client.CloseableHttpClient;
import org.junit.Test;

public final class ApacheHttpClientChannelsTest extends AbstractChannelTest {

@Override
protected Channel createChannel(ClientConfiguration config, UserAgent agent) {
return ApacheHttpClientChannels.create(config, agent);
}

@Test
public void close_works() throws Exception {
ClientConfiguration conf = TestConfigurations.create("http://foo");

Channel channel;
try (CloseableHttpClient client = ApacheHttpClientChannels.createCloseableHttpClient(conf)) {

channel = ApacheHttpClientChannels.createSingleUri("http://foo", client);
ListenableFuture<Response> response =
channel.execute(new TestEndpoint(), Request.builder().build());
assertThatThrownBy(() -> Futures.getUnchecked(response)).hasCauseInstanceOf(UnknownHostException.class);
}

ListenableFuture<Response> again =
channel.execute(new TestEndpoint(), Request.builder().build());
assertThatThrownBy(() -> again.get()).hasMessageContaining("Connection pool shut down");
}

private static final class TestEndpoint implements Endpoint {
@Override
public void renderPath(Map<String, String> _params, UrlBuilder _url) {}

@Override
public HttpMethod httpMethod() {
return HttpMethod.GET;
}

@Override
public String serviceName() {
return "service";
}

@Override
public String endpointName() {
return "endpoint";
}

@Override
public String version() {
return "1.0.0";
}
}
}