zarr outdated: 'zarr.storage' has no attribute 'Store'

[VeckoTheGecko]

Parcels version: master
zarr version: 2.8.1

Just encountered the error attached when running an example notebook. Looks like there is an incompatability introduced in #1303 with older versions of zarr. The Store functionality was only introduced in zarr 2.11.0 in PR zarr-developers/zarr-python#789.

Possibly we need to specify zarr>=2.11.0 in the recipe and dev environments?

Not sure if this is a widespread issue, or just something to do with my unique environment.

CC: @willirath

AttributeError                            Traceback (most recent call last)
Cell In[5], line 11
     7 time = np.arange(0, npart) * timedelta(hours=2).total_seconds()  # release every particle two hours later
     9 pset = ParticleSet(fieldset=fieldset, pclass=JITParticle, lon=lon, lat=lat, time=time)
---> 11 output_file = pset.ParticleFile(name="output.nc", outputdt=timedelta(hours=2))
    13 pset.execute(
    14     AdvectionRK4,
    15     runtime=delta(hours=24),
    16     dt=delta(minutes=5),
    17     output_file=output_file
    18 )

File ~/coding/repos/my-parcels/parcels/particleset/particlesetsoa.py:639, in ParticleSetSOA.ParticleFile(self, *args, **kwargs)
   636 def ParticleFile(self, *args, **kwargs):
   637     """Wrapper method to initialise a :class:`parcels.particlefile.ParticleFile`
   638     object from the ParticleSet"""
--> 639     return ParticleFile(*args, particleset=self, **kwargs)

File ~/coding/repos/my-parcels/parcels/particlefile/particlefilesoa.py:22, in ParticleFileSOA.__init__(self, name, particleset, outputdt, chunks, write_ondelete)
    21 def __init__(self, name, particleset, outputdt=np.infty, chunks=None, write_ondelete=False):
---> 22     super(ParticleFileSOA, self).__init__(name=name, particleset=particleset, outputdt=outputdt,
    23                                           chunks=chunks, write_ondelete=write_ondelete)

File ~/coding/repos/my-parcels/parcels/particlefile/baseparticlefile.py:96, in BaseParticleFile.__init__(self, name, particleset, outputdt, chunks, write_ondelete, create_new_zarrfile)
    86 self.fmt_map = {np.float16: 'f2', np.float32: 'f4', np.float64: 'f8',
    87                 np.bool_: 'i1', np.int8: 'i1', np.int16: 'i2',
    88                 np.int32: 'i4', np.int64: 'i8', np.uint8: 'u1',
    89                 np.uint16: 'u2', np.uint32: 'u4', np.uint64: 'u8'}
    90 self.fill_value_map = {np.float16: np.nan, np.float32: np.nan, np.float64: np.nan,
    91                        np.bool_: np.iinfo(np.int8).max, np.int8: np.iinfo(np.int8).max,
    92                        np.int16: np.iinfo(np.int16).max, np.int32: np.iinfo(np.int32).max,
    93                        np.int64: np.iinfo(np.int64).max, np.uint8: np.iinfo(np.uint8).max,
    94                        np.uint16: np.iinfo(np.uint16).max, np.uint32: np.iinfo(np.uint32).max,
    95                        np.uint64: np.iinfo(np.uint64).max}
---> 96 if issubclass(type(name), zarr.storage.Store):
    97     # If we already got a Zarr store, we won't need any of the naming logic below.
    98     # But we need to handle incompatibility with MPI mode for now:
    99     if MPI and MPI.COMM_WORLD.Get_size() > 1:
   100         raise ValueError("Currently, MPI mode is not compatible with directly passing a Zarr store.")

AttributeError: module 'zarr.storage' has no attribute 'Store'

[willirath]

Good catch!

There's two options go solve this in the Parcels code:

1. To avoid having to pin Zarr versions, we could also check for the name being a collections.MutableMapping. I've opened a Draft PR in [Do not merge] Hotfix for old zarr store #1330

2. Pin Zarr version to be geq 2.11.0. This would make life a lot easier on the dev side.

[erikvansebille]

I think that pinning to 2.11.0 is the cleaner and more long-term solution?

[willirath]

I agree. I also don't want to imagine maintaining tests for the thing I've proposed in #1330

[VeckoTheGecko]

I think that pinning to 2.11.0 is the cleaner and more long-term solution?

Would we be able to patch this, or is there another solution in mind? At least one other person has run into this (posted a comment in this thread, before deleting it shortly after; according to my emails 😋) so I think others may also run into it.

(I would PR myself, but not sure how conda packaging works with minor releases etc).