Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Approach to Null value handling #32

Open
TomNicholas opened this issue Mar 14, 2024 · 2 comments
Open

Approach to Null value handling #32

TomNicholas opened this issue Mar 14, 2024 · 2 comments

Comments

@TomNicholas
Copy link
Member

I'm not sure how to think about handling null values in the context of ManifestArray. Partly because I'm not very familiar with how zarr / zarr-python handles them.

Relevant for #16, #22 and #29.

@rabernat
Copy link
Collaborator

Zarr has the concept of a fill_value. This value is used in the case where a chunk is not found in the store to represent missing data.

Zarr has no concept of an arbitrary null value mask (like Xarray / CF conventions). When you use Zarr with Xarray, Xarray uses its own decoding / encoding mechanism to translate fill_value to NaN.

@TomNicholas
Copy link
Member Author

Thanks.

This value is used in the case where a chunk is not found in the store to represent missing data.

This answers #16, and means that the code I wrote to check that every chunk in the grid exists in the manifest might have been too conservative...

https://github.com/TomNicholas/VirtualiZarr/blob/28e02ce704b01f93edb8d8fb854f74dfbf9aaacd/virtualizarr/manifests/manifest.py#L169

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants