Adjust encoding of Azure block IDs #68957
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Today we represent block IDs sent to Azure using the URL-safe base-64 encoding. This makes sense: these IDs appear in URLs. It turns out that Azure rejects this encoding for block IDs and instead demands that they are represented using the regular, URL-unsafe, base-64 encoding instead, then further wrapped in %-encoding to deal with the URL-unsafe characters that inevitably result. Relates elastic#66489
DaveCTurner
added
>bug
:Distributed Coordination/Snapshot/Restore
elasticmachine
added
the
Team:Distributed (Obsolete)
|
Pinging @elastic/es-distributed (Team:Distributed) |
|
Unfortunately I don't have a good way to test this against a real Azure account right now, and |
I worked this out, it's because I was running it on a machine that doesn't have Docker. |
|
@elasticmachine please run elasticsearch-ci/2 |
fcofdez
approved these changes
Feb 15, 2021
There was a problem hiding this comment.
LGTM. Thanks for fixing this @DaveCTurner! I've run the test suites a few times agains azure (using a really small multi-block size threshold) and it passed.
DaveCTurner
added a commit
to DaveCTurner/elasticsearch
that referenced
this pull request
Feb 15, 2021
Today we represent block IDs sent to Azure using the URL-safe base-64 encoding. This makes sense: these IDs appear in URLs. It turns out that Azure rejects this encoding for block IDs and instead demands that they are represented using the regular, URL-unsafe, base-64 encoding instead, then further wrapped in %-encoding to deal with the URL-unsafe characters that inevitably result. Relates elastic#66489 Backport of elastic#68957
DaveCTurner
added a commit
to DaveCTurner/elasticsearch
that referenced
this pull request
Feb 15, 2021
Today we represent block IDs sent to Azure using the URL-safe base-64 encoding. This makes sense: these IDs appear in URLs. It turns out that Azure rejects this encoding for block IDs and instead demands that they are represented using the regular, URL-unsafe, base-64 encoding instead, then further wrapped in %-encoding to deal with the URL-unsafe characters that inevitably result. Relates elastic#66489 Backport of elastic#68957
|
Merged despite unrelated CI failures, with approval from @jasontedor. |
DaveCTurner
added a commit
that referenced
this pull request
Feb 15, 2021
Today we represent block IDs sent to Azure using the URL-safe base-64 encoding. This makes sense: these IDs appear in URLs. It turns out that Azure rejects this encoding for block IDs and instead demands that they are represented using the regular, URL-unsafe, base-64 encoding instead, then further wrapped in %-encoding to deal with the URL-unsafe characters that inevitably result. Relates #66489 Backport of #68957
DaveCTurner
added a commit
that referenced
this pull request
Feb 15, 2021
Today we represent block IDs sent to Azure using the URL-safe base-64 encoding. This makes sense: these IDs appear in URLs. It turns out that Azure rejects this encoding for block IDs and instead demands that they are represented using the regular, URL-unsafe, base-64 encoding instead, then further wrapped in %-encoding to deal with the URL-unsafe characters that inevitably result. Relates #66489 Backport of #68957
|
Hello @DaveCTurner , UPD, just found this comment: https://github.com/elastic/dev/issues/1617#issuecomment-779160469 |
fcofdez
added a commit
that referenced
this pull request
Mar 1, 2021
fcofdez
added a commit
to fcofdez/elasticsearch
that referenced
this pull request
Mar 1, 2021
Relates elastic#68957 Backport of elastic#69267
fcofdez
added a commit
that referenced
this pull request
Mar 1, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Today we represent block IDs sent to Azure using the URL-safe base-64
encoding. This makes sense: these IDs appear in URLs. It turns out that
Azure rejects this encoding for block IDs and instead demands that they
are represented using the regular, URL-unsafe, base-64 encoding instead,
then further wrapped in %-encoding to deal with the URL-unsafe
characters that inevitably result.
Relates #66489