-
-
Notifications
You must be signed in to change notification settings - Fork 286
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[v3] remote store support (s3, gcs, azure, http) #1757
Comments
Hi, I'm watching. I can help, but I don't have a huge number of hours, of course. I notice the "blocked by" - and there has been no progress on that issue yet, |
Personally I don't think we need to wait for a final store API before implementing remote storage. Any friction generated in that process can go towards shaping the final API :) |
Is the use of universal_pathlib desired/required? |
@martindurant |
I think |
OK, I can have it optional. Honestly I doubt we'll see it much, since typical invocation would be like |
UPath becomes interesting if you store some path = UPath("s3://bucket", key='some', secret='credentials')
z1 = zarr.open(path / "data.zarr")
z2 = zarr.open(path / "other.zarr") |
The v3 branch currently has an initial (but broken) implementation of a remote store. This issue tracks getting that operational so we car start using v3 against s3, gcs, azure, http, etc.
TODOs:
FsspecStore
@martindurant - any chance you can lend a hand on this one? I optimistically put you in as the assignee but let us know if this is not something you can engage on.
Blocked by #1755
The text was updated successfully, but these errors were encountered: