-
Notifications
You must be signed in to change notification settings - Fork 7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Run on zarr-python v3 with zarr-format v2 #288
Run on zarr-python v3 with zarr-format v2 #288
Conversation
Nice 👍 |
c0539ee
to
e38859f
Compare
I rebased this branch on main after #292 went in. The failures are due to a problem with non-contiguous arrays that is fixed in zarr-developers/zarr-python#2515, but that hasn't been released yet. The other thing I noticed is that the validation tests are a lot slower running on v3 than v2 With this PR and zarr-python v2:
With this PR and zarr-python v3:
I'll see if I can find out what is causing this slowdown. |
It's because in v2 Here's one of the places where this is happening: bio2zarr/bio2zarr/vcf2zarr/verification.py Line 173 in b1d7ef2
|
Thanks for tracking that one down @tomwhite - I added the original |
e38859f
to
b866285
Compare
I updated this to reflect all the changes upstream in Zarr (there have been a lot in the last month!). The main changes are:
There is one failing test |
|
After some hunting I tracked this down, see zarr-developers/zarr-python#2696. In the meantime, I have excluded |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks!
This is like #284, but the Zarr format is always v2, even when zarr-python v3 is being used. This is the approach I took for the Zarr v3 support in sgkit. This limits the scope of the changes, however there are still test failures under both zarr-python v2 and v3.
This still needs quite a lot of work, so I've opened it as a draft.