Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BinaryRequestBody and ContentBody use InputStream.transferTo(OutputStream) #1983

Merged
merged 2 commits into from
Jul 28, 2023

Conversation

schlosna
Copy link
Contributor

@schlosna schlosna commented Jul 28, 2023

Before this PR

Dialogue used Guava ByteStreams.copy for 8KiB byte buffer chunk copies.

After this PR

==COMMIT_MSG==
BinaryRequestBody and ContentBody use InputStream.transferTo(OutputStream)

Allow for optimization when underlying input stream (such as ByteArrayInputStream, ChannelInputStream) overrides transferTo(OutputStream) to avoid extra array allocations and copy larger chunks at a time (e.g. allowing 16KiB chunks via ApacheHttpClientBlockingChannel.ModulatingOutputStream from #1790).

When running on JDK 21+, this also enables 16KiB byte chunk copies via InputStream.transferTo(OutputStream) per JDK-8299336, where as on JDK < 21 and when using Guava ByteStreams.copy 8KiB byte chunk copies are used.

References:

==COMMIT_MSG==

Possible downsides?

Allow for optimization when underlying input stream (such as
ByteArrayInputStream, ChannelInputStream) overrides
transferTo(OutputStream) to avoid extra array allocations and copy
larger chunks at a time (e.g. allowing 16KiB chunks via
ApacheHttpClientBlockingChannel.ModulatingOutputStream from #1790).

When moving to JDK 21+, this will also enable 16KiB byte chunk copies
via InputStream.transferTo(OutputStream) per JDK-8299336, where as on
JDK < 21 and when using Guava ByteStreams.copy 8KiB byte chunk copies
are used.

References:
* palantir/hadoop-crypto#586
* https://bugs.openjdk.org/browse/JDK-8299336
* https://bugs.openjdk.org/browse/JDK-8067661
* https://bugs.openjdk.org/browse/JDK-8265891
* https://bugs.openjdk.org/browse/JDK-8273038
* https://bugs.openjdk.org/browse/JDK-8279283
* https://bugs.openjdk.org/browse/JDK-8296431
@changelog-app
Copy link

changelog-app bot commented Jul 28, 2023

Generate changelog in changelog/@unreleased

What do the change types mean?
  • feature: A new feature of the service.
  • improvement: An incremental improvement in the functionality or operation of the service.
  • fix: Remedies the incorrect behaviour of a component of the service in a backwards-compatible way.
  • break: Has the potential to break consumers of this service's API, inclusive of both Palantir services
    and external consumers of the service's API (e.g. customer-written software or integrations).
  • deprecation: Advertises the intention to remove service functionality without any change to the
    operation of the service itself.
  • manualTask: Requires the possibility of manual intervention (running a script, eyeballing configuration,
    performing database surgery, ...) at the time of upgrade for it to succeed.
  • migration: A fully automatic upgrade migration task with no engineer input required.

Note: only one type should be chosen.

How are new versions calculated?
  • ❗The break and manual task changelog types will result in a major release!
  • 🐛 The fix changelog type will result in a minor release in most cases, and a patch release version for patch branches. This behaviour is configurable in autorelease.
  • ✨ All others will result in a minor version release.

Type

  • Feature
  • Improvement
  • Fix
  • Break
  • Deprecation
  • Manual task
  • Migration

Description

BinaryRequestBody and ContentBody use InputStream.transferToOutputStream

Allow for optimization when underlying input stream (such as ByteArrayInputStream, ChannelInputStream) overrides transferTo(OutputStream) to avoid extra array allocations and copy larger chunks at a time (e.g. allowing 16KiB chunks via ApacheHttpClientBlockingChannel.ModulatingOutputStream from #1790).

When running on JDK 21+, this also enables 16KiB byte chunk copies via InputStream.transferTo(OutputStream) per JDK-8299336, where as on JDK < 21 and when using Guava ByteStreams.copy 8KiB byte chunk copies are used.

References:

Check the box to generate changelog(s)

  • Generate changelog entry

@bulldozer-bot bulldozer-bot bot merged commit c2a4177 into develop Jul 28, 2023
@bulldozer-bot bulldozer-bot bot deleted the davids/transferTo branch July 28, 2023 17:21
@svc-autorelease
Copy link
Collaborator

Released 3.88.0

@schlosna schlosna changed the title BinaryRequestBody and ContentBody use InputStream.transferToOutputStream BinaryRequestBody and ContentBody use InputStream.transferTo(OutputStream) Jul 28, 2023
@@ -39,7 +38,7 @@ final class InputStreamContentBody implements ContentBody {

@Override
public void writeTo(OutputStream output) throws IOException {
ByteStreams.copy(inputStream, output);
inputStream.transferTo(output);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we get this into error-prone if this is now the preferred approach? I know I'm guilty of many uses of this method

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good call, I'd like to write up an error-prone rule to migrate folks soon.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants