Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: make it possible to disable the buffer of ReadChannels returned from Storage.reader #1974

Merged
merged 1 commit into from
Apr 12, 2023

Conversation

BenWhitehead
Copy link
Collaborator

For some scenarios, an external client needs the ability to manage buffer blocking itself. To support this, providing 0 to ReadChannel#setChunkSize will disable buffering allowing for client to avoid the need to alight multiple levels of buffer alignments.

Because the buffering is disabled, that means reads can be much more variable in size and individually more impacted by small network latencies (with buffering these can be amortized by the buffer making followup read faster). This is considered advanced usage and will require more work from the client integrator.

… from Storage.reader

For some scenarios, an external client needs the ability to manage buffer blocking itself. To support this, providing 0 to ReadChannel#setChunkSize will disable buffering allowing for client to avoid the need to alight multiple levels of buffer alignments.

Because the buffering is disabled, that means reads can be much more variable in size and individually more impacted by small network latencies (with buffering these can be amortized by the buffer making followup read faster). This is considered advanced usage and will require more work from the client integrator.
@BenWhitehead BenWhitehead requested a review from a team as a code owner April 10, 2023 22:17
@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: storage Issues related to the googleapis/java-storage API. labels Apr 10, 2023
Copy link
Collaborator

@sydney-munro sydney-munro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor comment feel free to merge once acknowledged.

@@ -107,6 +108,35 @@ public void storageReadChannel_getObject_returns() throws Exception {
}
}

@Test
// @CrossRun.Exclude(transports = Transport.GRPC)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can we remove this line?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'll add it's removal in my next PR (don't want to waste all the cpu time to do a full rebuild and test to remove this comment in a testclass)

@BenWhitehead BenWhitehead merged commit 702ab2b into main Apr 12, 2023
@BenWhitehead BenWhitehead deleted the nobuffer-reader branch April 12, 2023 17:11
gcf-merge-on-green bot pushed a commit that referenced this pull request Apr 12, 2023
🤖 I have created a release *beep* *boop*
---


## [2.22.0](https://togithub.com/googleapis/java-storage/compare/v2.21.0...v2.22.0) (2023-04-12)


### Features

* Add matchGlob parameter to BlobListOption ([#1965](https://togithub.com/googleapis/java-storage/issues/1965)) ([93be97a](https://togithub.com/googleapis/java-storage/commit/93be97a9ae061241f510cbeb566f9f839063faa0))
* Handle generation numbers in BlobId#{to,from}GsUtilUri ([#1929](https://togithub.com/googleapis/java-storage/issues/1929)) ([0a033b3](https://togithub.com/googleapis/java-storage/commit/0a033b36a2be7e1d92c8b2f7323f2851005034cf))
* Implement GrpcStorageImpl#{get,list,create,delete}Notification ([#1958](https://togithub.com/googleapis/java-storage/issues/1958)) ([830052b](https://togithub.com/googleapis/java-storage/commit/830052b3f26337f50770436784f0ea9f3366dbca))
* Make it possible to disable the buffer of ReadChannels returned from Storage.reader ([#1974](https://togithub.com/googleapis/java-storage/issues/1974)) ([702ab2b](https://togithub.com/googleapis/java-storage/commit/702ab2bb1ceb9f428296591adc6e09023b4a8484))


### Bug Fixes

* Update GrpcStorageOptions to set allowNonDefaultServiceAccount = true ([#1977](https://togithub.com/googleapis/java-storage/issues/1977)) ([f1b9493](https://togithub.com/googleapis/java-storage/commit/f1b94930509c87ecbdbe6627fec6118305331780))


### Documentation

* Update spec for WriteObjectRequest.object_size to clarify that it applies to both one-shot and resumable writes ([#1976](https://togithub.com/googleapis/java-storage/issues/1976)) ([7354e19](https://togithub.com/googleapis/java-storage/commit/7354e198521384cd2b8583f073217a5a6cb74fe0))


### Dependencies

* Update dependency com.google.cloud:google-cloud-shared-dependencies to v3.7.0 ([#1978](https://togithub.com/googleapis/java-storage/issues/1978)) ([1f7e9c9](https://togithub.com/googleapis/java-storage/commit/1f7e9c9130a9516acfcdfda4b23e642fbe799146))
* Update dependency org.graalvm.buildtools:native-maven-plugin to v0.9.21 ([#1979](https://togithub.com/googleapis/java-storage/issues/1979)) ([d6accdf](https://togithub.com/googleapis/java-storage/commit/d6accdf9c81419728c39a61e348a61683dc117ad))

---
This PR was generated with [Release Please](https://togithub.com/googleapis/release-please). See [documentation](https://togithub.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the googleapis/java-storage API. size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants