Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[C++][FS][Azure] Produce better error message when Move is attempted on flat-namespace accounts #40405

Closed
felipecrv opened this issue Mar 7, 2024 · 1 comment
Closed

[C++][FS][Azure] Produce better error message when Move is attempted on flat-namespace accounts #40405

felipecrv opened this issue Mar 7, 2024 · 1 comment
Assignees
@felipecrv
Labels
Component: C++ Type: enhancement
Milestone
16.0.0

Comments

@felipecrv
Copy link
Contributor

Describe the enhancement requested

Azure Blobs Storage doesn't support renaming of blobs (without downloading and re-uploading the Blob) and offers even less support for atomic rename of multiple blobs.

Users are advised to enable Hierarchical Namespace support [1] if they intend to perform Move operations. AzureFileSystem will automatically detect HNS is enabled and use the Azure Data Lake File System Gen 2 APIs [1] and perform Move operations through that. Then even directories can be moved (something that even the Arrow S3FileSystem doesn't support at the moment).

Before giving up, I started the work in [2]. If anyone wants to implement just blob Move or directory Move, I recommend cloning the branch and continuing the work from there.

[1] https://learn.microsoft.com/en-gb/azure/storage/blobs/data-lake-storage-namespace
[2] #40025

Component(s)

C++
@github-actions github-actions bot mentioned this issue Mar 7, 2024
Merged
@felipecrv felipecrv mentioned this issue Mar 7, 2024
Closed
kou pushed a commit that referenced this issue Mar 8, 2024
@felipecrv
GH-40405: [C++] Produce better error message when Move is attempted o…
d2970e1 
…n flat-namespace accounts (#40406)

### Rationale for this change

To guide users that might try to `Move` on accounts without HNS.

### What changes are included in this PR?

The rewrite of the error message.

### Are these changes tested?

Covered by existing tests.
* GitHub Issue: #40405

Authored-by: Felipe Oliveira Carvalho <felipekde@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@kou
Copy link
Member

kou commented Mar 8, 2024

Issue resolved by pull request 40406
#40406

@kou kou added this to the 16.0.0 milestone Mar 8, 2024
@kou kou closed this as completed Mar 8, 2024
thisisnic pushed a commit to thisisnic/arrow that referenced this issue Mar 8, 2024
@felipecrv @thisisnic
apacheGH-40405: [C++] Produce better error message when Move is attem…
943fe99 
…pted on flat-namespace accounts (apache#40406)

### Rationale for this change

To guide users that might try to `Move` on accounts without HNS.

### What changes are included in this PR?

The rewrite of the error message.

### Are these changes tested?

Covered by existing tests.
* GitHub Issue: apache#40405

Authored-by: Felipe Oliveira Carvalho <felipekde@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@felipecrv felipecrv mentioned this issue Mar 14, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@felipecrv felipecrv

Labels
Component: C++ Type: enhancement
Projects
None yet
Milestone
16.0.0
Development

No branches or pull requests
2 participants
@kou @felipecrv