Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't rely on FileNotFoundError from xr.open_dataset #459

Open
pont-us opened this issue May 19, 2021 · 0 comments
Open

Don't rely on FileNotFoundError from xr.open_dataset #459

pont-us opened this issue May 19, 2021 · 0 comments

Comments

@pont-us
Copy link
Member

pont-us commented May 19, 2021

In core/timeslice.py, FileNotFoundError is caught and used to trigger a fallback behaviour for a missing file. This code broke with the xarray 0.18.0 release. I reported this as a bug at pydata/xarray#5329 and implemented a temporary workaround (catching ValueError as well as FileNotFoundError), but ongoing discussions there indicate that we probably won't be able to rely on FileNotFoundError always being thrown. We should:

  1. Explicitly specify the zarr store for open_dataset here -- this doesn't directly fix the problem (since the zarr engine throws a GroupNotFoundError rather than the FileNotFoundError previously thrown by the automatically selected netcdf4 engine), but makes the behaviour more robust and explicit and any errors easier to diagnose.
  2. Catch the GroupNotFoundError to trigger the fallback behaviour. Don't catch ValueError or FileNotFoundError any more, since they're no longer expected to be thrown when the zarr engine is explicitly specified.
  3. Do an explicit check for file presence before the call, which should mean that the GroupNotFoundError is rarely thrown in the first place.

This should also be done anywhere else where we're relying on FileNotFoundError in the same way elsewhere in the xcube codebase, but from a quick grep it appears that there aren't any other usages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant