Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade GCS SDK to 1.117.1 #74938

Conversation

original-brownbear
Copy link
Member

We're behind the upgrade schedule by quite a bit here, upgrading to the latest version
and adjusting our test fixture accordingly.

We're behind the ugprade schedule by quite a bit here, upgrading to the latest version
and adjusting our test fixture accordingly.
@elasticmachine elasticmachine added the Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. label Jul 5, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

if ("bytes */*".equals(range) == false) {
final int start = getContentRangeStart(range);
final int end = getContentRangeEnd(range);
exchange.getResponseHeaders().add("Range", String.format(Locale.ROOT, "bytes=%d-%d", start, end));
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may actually fix the JDK-8 issue we've been seeing (I'm not 100% on this yet). The header format was broken here and the retries tests with the new SDK caught this now. I wonder if this was the spot that made the JDK-8 URL client spontaneously close connections on header parse failures (as it turns out it doesn't log/thrown on those in any way). I'll revisit this in a follow-up on 7.x.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exception handling in HttpServer is pretty debugging-unfriendly, see #68967.


blob = CompositeBytesReference.of(blob, requestBody);
blobs.put(blobName, blob);

if (limit == null) {
exchange.getResponseHeaders().add("Range", String.format(Locale.ROOT, "bytes=%d/%d", start, end));
if ("bytes */*".equals(range) == false) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new SDK sends the unknown range header to determine where to continue uploading now so we have to deal with that :) For now I just made us not sent a range back but a follow-up would be helpful here to properly account for the bytes we've already seen so that the existing entry from blobs is reflected in the response correctly.

Copy link
Contributor

@fcofdez fcofdez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks Armin!

if ("bytes */*".equals(range) == false) {
final int start = getContentRangeStart(range);
final int end = getContentRangeEnd(range);
exchange.getResponseHeaders().add("Range", String.format(Locale.ROOT, "bytes=%d-%d", start, end));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

@original-brownbear
Copy link
Member Author

Thanks Francisco!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. >upgrade v7.15.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants