Is there a way to call zarr.save
and get a single file like a .zip or .tar
#1292
Unanswered
NickleDave
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi, thanks for the
zarr
project.I have a question I don't see answered in the docs; maybe it would be good to add an example/tutorial/vignette for it?
What I'm trying to do is (1) use zarr to save some numpy arrays from a relatively expensive step of data preparation and then (2) upload the saved arrays to an open science platform that allows sharing single files, specifically the Open Science Framework.
My question then is: can I call
zarr.save
to get a single file instead of aDirectoryStore
? If so is there more than one way to do this and what are the pros and cons of each?The default
zarr.save
gives me a directory with, as I understand it now, array chunks in blosc format. What I want is a single file like a .tar.gz that I can upload. The section in the docs on "Storage Alternatives" provides an example of usingZipStore
directly but immediately caveats this as having drawbacks. I'm hoping there's a way I can usezarr.save
to get a single file while remaining agnostic to what that file type is, like when my library callszarr.load
on that file again.I am guessing a lot of researchers would want to do something similar, since like me they are probably using some platform that makes single files the easiest thing to use.
I do understand that I could alternatively use some object storage-formats on cloud platforms like S3 but I don't have a good mental model for that and I expect the same will be true for many other researchers.
Thanks in advance for any help and advice, and for your patience while I better understand the API and use cases that zarr is designed for.
Beta Was this translation helpful? Give feedback.
All reactions