Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ApacheHttpChannels exposes a Closeable instance #430

Merged
merged 11 commits into from
Feb 25, 2020
Merged

Conversation

iamdanfox
Copy link
Contributor

@iamdanfox iamdanfox commented Feb 25, 2020

Before this PR

Previously, our ApacheHttpChannels#create method would create a client pool internally, but gave us no way to shut this down.

After this PR

==COMMIT_MSG==
ApacheHttpChannels exposes a Closeable instance, allowing callers to release resources when they determine the client is no longer needed.
==COMMIT_MSG==

Possible downsides?

  • This approach means we have more API surface, as we have to expose quite primitive building blocks and trust that the WC facade will abstract them appropriately. This is in contrast to my earlier clientPool approach which hid all these details.

@changelog-app
Copy link

changelog-app bot commented Feb 25, 2020

Generate changelog in changelog/@unreleased

Type

  • Feature
  • Improvement
  • Fix
  • Break
  • Deprecation
  • Manual task
  • Migration

Description

ApacheHttpChannels exposes a Closeable instance, allowing us to release resources when the client is no longer needed.

Check the box to generate changelog(s)

  • Generate changelog entry

return BlockingChannelAdapter.of(new ApacheHttpClientBlockingChannel(client, url(uri)));
}

public static CloseableHttpClient createCloseableHttpClient(ClientConfiguration conf) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A little bit wonky in public api since it's not clear what functionality the returned client provides compared to our channel wrappers. Will we cause bugs if we move functionality from the client into a channel?
At the same time, this should be an implementation dependency of components which simply provide channels, so it's probably fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so I wanted to just convey the idea of some closeable shared resource - would you prefer if I wrap it in some opaque type so that people can't use it directly?

}

public static Channel createSingleUri(String uri, CloseableHttpClient client) {
return BlockingChannelAdapter.of(new ApacheHttpClientBlockingChannel(client, url(uri)));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another possibility is that we create a new client for each URI for additional isolation and avoiding contention in the connection pool.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This API gives us the freedom to do that from our WC facade, because we can choose to pass in the same closeablehttpclient or not.

@@ -75,15 +75,26 @@

private ApacheHttpClientChannels() {}

public static Channel create(ClientConfiguration conf, UserAgent baseAgent) {
public static Channel create(ClientConfiguration conf, UserAgent agent) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should all channel implementations be closeable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually started with this, but I think it will get hella confusing in the scenario where our NodeSelectionStrategy channels contain a bunch of internal channels and possibly want to live reload a new uri... I think it's clearer to only make things closeable that actually directly hold resources, and if they don't hold resources then they can just be thrown away.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think queuedchannel is the only other piece that holds resources

@carterkozak
Copy link
Contributor

👍

@bulldozer-bot bulldozer-bot bot merged commit 3abd36d into develop Feb 25, 2020
@bulldozer-bot bulldozer-bot bot deleted the dfox/closeable branch February 25, 2020 14:33
@svc-autorelease
Copy link
Collaborator

Released 0.10.1

return BlockingChannelAdapter.of(new ApacheHttpClientBlockingChannel(client.client, url(uri)));
}

public static CloseableClient createCloseableHttpClient(ClientConfiguration conf) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ClientCloseable? :) palantir/conjure-java#790

return BlockingChannelAdapter.of(new ApacheHttpClientBlockingChannel(client.client, url(uri)));
}

public static CloseableClient createCloseableHttpClient(ClientConfiguration conf) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ClientCloseable? :) palantir/conjure-java#790


@Override
public String toString() {
return "SharedResource{client=" + client + '}';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SharedResource?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤦‍♂ #449

}

/** Intentionally opaque wrapper type - we don't want people using the inner Apache client directly. */
public static final class CloseableClient implements Closeable {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this public if it's never used outside this class?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It needs to be public because we create some of these in Witchcraft and then re-use them to avoid re-creating clients on every url live-reload.

See https://internal-github/foundry/witchcraft/pull/2233/files#diff-2819734323a961448d10003f575f1b4bR217

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants