Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ArrayV3Metadata does not declare storage_transformers #2009

Closed
Tracked by #175
ghidalgo3 opened this issue Jul 3, 2024 · 3 comments
Closed
Tracked by #175

ArrayV3Metadata does not declare storage_transformers #2009

ghidalgo3 opened this issue Jul 3, 2024 · 3 comments
Assignees
Labels
bug Potential issues with the zarr-python library
Milestone

Comments

@ghidalgo3
Copy link

ghidalgo3 commented Jul 3, 2024

Zarr version

3.0.0a0

Numcodecs version

0.12.1

Python Version

3.10.12

Operating System

Linux

Installation

pip install zarr==3.0.0a0

Description

According to the ZarrV3 specification on storage_transformers, array v3 metadata must support a storage_transformers property. If I create a Zarr store with VirtualiZarr which uses the storage_transformers key in the zarr.json, then attempting to read store will produces this exception TypeError: ArrayV3Metadata.__init__() got an unexpected keyword argument 'storage_transformers'.

Steps to reproduce

import zarr
import xarray as xr
import numpy as np
from virtualizarr import ManifestArray, ChunkManifest

arr = ManifestArray(
        chunkmanifest=ChunkManifest(
            entries={"0.0": dict(path="test.nc", offset=6144, length=48)}
        ),
        zarray=dict(
            shape=(2, 3),
            dtype=np.dtype("<i8"),
            chunks=(2, 3),
            compressor=None,
            filters=None,
            fill_value=np.nan,
            order="C",
            zarr_format=3,
        ),
    )
original = xr.Dataset({"a": (["x", "y"], arr)}, attrs={"something": 0})
original.virtualize.to_zarr("store.zarr")
group = zarr.open_group(store="store.zarr")
group.arrays() # This will throw TypeError: ArrayV3Metadata.__init__() got an unexpected keyword argument 'storage_transformers'

Additional output

This is the zarr.json for the a variable in the repro script:

{
    "attributes": {},
    "chunk_grid": {
        "configuration": {
            "chunk_shape": [
                2,
                3
            ]
        },
        "name": "regular"
    },
    "chunk_key_encoding": {
        "configuration": {
            "separator": "/"
        },
        "name": "default"
    },
    "codecs": null,
    "data_type": "int64",
    "dimension_names": [
        "x",
        "y"
    ],
    "fill_value": null,
    "node_type": "array",
    "shape": [
        2,
        3
    ],
    "storage_transformers": [
        {
            "configuration": {
                "manifest": "./manifest.json"
            },
            "name": "chunk-manifest-json"
        }
    ],
    "zarr_format": 3
}
@ghidalgo3 ghidalgo3 added the bug Potential issues with the zarr-python library label Jul 3, 2024
@d-v-b
Copy link
Contributor

d-v-b commented Jul 3, 2024

thanks for the bug report! I will fix this shortly, and hopefully that fix makes it into the next alpha release

@d-v-b d-v-b self-assigned this Jul 3, 2024
@dstansby
Copy link
Contributor

@d-v-b do you remember if you fixed this?

@dstansby dstansby added this to the 3.0.0 milestone Dec 30, 2024
@jhamman
Copy link
Member

jhamman commented Jan 1, 2025

Closed by #2180.

@jhamman jhamman closed this as completed Jan 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Potential issues with the zarr-python library
Projects
None yet
Development

No branches or pull requests

4 participants