KeyError with InferenceData.to_netcdf

[michaelosthege]

Describe the bug
My collaboration partners ran into this when trying to do idata.to_netcdf.

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~\anaconda3\envs\CARenv\lib\site-packages\xarray\backends\file_manager.py in _acquire_with_cache_info(self, needs_lock)
    198             try:
--> 199                 file = self._cache[self._key]
    200             except KeyError:

~\anaconda3\envs\CARenv\lib\site-packages\xarray\backends\lru_cache.py in __getitem__(self, key)
     52         with self._lock:
---> 53             value = self._cache[key]
     54             self._cache.move_to_end(key)

KeyError: [<class 'netCDF4._netCDF4.Dataset'>, ('C:\\Git Repos\\CarbtModel\\notebooks\\NikolasNC.nc',), 'a', (('clobber', True), ('diskless', False), ('format', 'NETCDF4'), ('persist', False))]

During handling of the above exception, another exception occurred:

RuntimeError                              Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_1096/4099136273.py in <module>
----> 1

The file was created and contains a posterior group, but other groups like sampler stats are missing.

Any ideas?

ArviZ version: 0.11.2

[ahartikainen]

What kind of data the object contains? Maybe there is some reference to an object that fails? Or maybe the number format is not the basic python/numpy but something more exotic like tensorflow etc?

[lhelleckes]

I have the same issue and we already found out that it might be a matter of package installation. I saved a trace that I cannot save to NetCDF to zarray instead and send it to @michaelosthege, who in turn loaded it and saved it as NetCDF. I cannot load this new NetCDF he sent me.

We found the issue both with arviz 0.11.2 and 0.11.4 and xarray 0.19.0 and xarray 0.18.2.

[lhelleckes]

Update: We found our environment difference: I have NetCDF 1.5.7. Downgrading to 1.5.6 solved the issue.

[OriolAbril]

Is there something we can do to fix this?

[michaelosthege]

While I never had this problem again, I do recall several instances where recent NetCDF4 and h5py versions caused incompatibilities.
Generally I would recommend to make more releases on smaller increments, because every time there's a new release the conda-gorge magic happens and could reveal/fix incompatibilities.

[tomicapretto]

@OriolAbril should this still be open? Maybe we can close it and reopen if someone comes with a reproducible example?

[OriolAbril]

we also depend on h5necdf instead of netcdf4 thanks to @varchasgopalaswamy great contribution on #2122 which seems to be more reliable. So I do think it is safe to close and will do it. To anyone reading this, please don't hesitate to comment here or open new issues if you face anything similar.