Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support suffix for metadata files #63

Closed
alimanfoo opened this issue May 1, 2020 · 3 comments
Closed

Support suffix for metadata files #63

alimanfoo opened this issue May 1, 2020 · 3 comments
Assignees
Labels
core-protocol-v3.0 Issue relates to the core protocol version 3.0 spec todo pre-rfc

Comments

@alimanfoo
Copy link
Member

alimanfoo commented May 1, 2020

In the current v3 core protocol draft, keys for metadata documents have no suffix to represent the document format, they are just keys like "meta/root/foo.group" and "meta/root/foo/bar.array". In some situations (e.g., where file systems or web servers are being used as storage) it would be useful to have a suffix that indicates the format, i.e., ".json" for the default. Consider adding support for this.

@alimanfoo alimanfoo added the core-protocol-v3.0 Issue relates to the core protocol version 3.0 spec label May 1, 2020
@Carreau
Copy link
Contributor

Carreau commented Sep 25, 2020

Add something like :

"metadata_key_suffix": ".json",

@Carreau Carreau self-assigned this Sep 25, 2020
Carreau added a commit to Carreau/zarr-specs that referenced this issue Oct 12, 2020
Closes zarr-developers#63

The rational is that backends may be explored by non zarr-aware tools,
like a filesytem and a GUI browser like Finder/Explorer/Nautilus, and
the `.json` extension will help to view/edit json in those.

While it may seem like this shoudl be store specific, once again the
fact that a zarr-hierarchy can be copied from one storage to another by
non-zarr-aware tool indicate that this shoudl be handled above the sotre
level.
Carreau added a commit to Carreau/zarr-specs that referenced this issue Oct 12, 2020
Closes zarr-developers#63

The rational is that backends may be explored by non zarr-aware tools,
like a filesytem and a GUI browser like Finder/Explorer/Nautilus, and
the `.json` extension will help to view/edit json in those.

While it may seem like this shoudl be store specific, once again the
fact that a zarr-hierarchy can be copied from one storage to another by
non-zarr-aware tool indicate that this shoudl be handled above the sotre
level.
@jbms
Copy link
Contributor

jbms commented Feb 9, 2022

An alternative is that the extension is determined by metadata_encoding. E.g. for the default metadata_encoding the extension is .json, and for a hypothetical cbor metadata encoding the extension would be .cbor or something.

As the spec is right now, if you specify a non-default metadata_encoding then the default metadata_key_suffix of .json is probably not appropriate.

@joshmoore joshmoore reopened this May 20, 2022
@jstriebel jstriebel added this to ZEP1 Nov 16, 2022
@jstriebel jstriebel moved this to Needs PR in ZEP1 Nov 16, 2022
@jstriebel jstriebel moved this from Needs PR to In Discussion in ZEP1 Dec 9, 2022
@jstriebel
Copy link
Member

The default zarr.json metadata now has a .json suffix, other metadata formats with different suffixes can be added via group extensions, see #37.

@github-project-automation github-project-automation bot moved this from In Discussion to Done in ZEP1 Feb 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core-protocol-v3.0 Issue relates to the core protocol version 3.0 spec todo pre-rfc
Projects
Status: Done
Development

No branches or pull requests

5 participants