-
-
Notifications
You must be signed in to change notification settings - Fork 282
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prune stores #1791
Prune stores #1791
Conversation
* refactor(v3): Using appropriate types * fix(v3): Typing fixes + minor code fixes * fix(v3): _sync_iter works with coroutines * docs(v3/store/core.py): clearer comment * fix(metadata.py): Use Any outside TYPE_CHECKING for Pydantic * fix(zarr/v3): correct zarr format + remove unused method * fix(v3/store/core.py): Potential suggestion on handling str store_like * refactor(zarr/v3): Add more typing * ci(.pre-commit-config.yaml): zarr v3 mypy checks turned on in pre-commit
…elopers#1728) * Specify v3 hatch envs using GitHub actions matrix * Update .github/workflows/test-v3.yml Co-authored-by: Joe Hamman <jhamman1@gmail.com> * Update .github/workflows/test-v3.yml Co-authored-by: Joe Hamman <jhamman1@gmail.com> * test on 3.12 too * no 3.12 --------- Co-authored-by: Joe Hamman <jhamman1@gmail.com> Co-authored-by: Joe Hamman <joe@earthmover.io>
* black -> ruff + cleanup * format * Preserve git blame * pre-commit fix
…ntributing.rst (zarr-developers#1643) Co-authored-by: Joe Hamman <joe@earthmover.io>
This PR specifically removes following the following v2 stores, and their v3 counterparts: One question that @jhamman raised was whether we want to remove Once we decide on whether or not we keep |
My feeling is that Zip stores are really an underappreciated feature of Zarr. Whatever we choose in terms of design, we should document Zip stores prominently. I'm curious--are there performance or feature differences between the native Zip store and the fsspec version? |
fsspec's ZipFS ought to be able to do everything, I would have thought, with the additional plus of being able to work with remote zips too (usually "r" and "w" modes only). It has certainly not been tested for speed, convenience being the priority over performance. A more specialised, and therefore simple, implementation can be reasonable if there is someone to maintain it. @mxmlnkn , you might be interested by another way of thinking of compressed bunch-of-files containers discussed here. |
Updates:
I think the |
Is this still something we want to get into zarr v3? And if so, would it be better to deprecate everything for a few releases and then remove it first? If not, and it's just going to be removed without warning, there should be a clear guide to how users can adapt their code if they were using these stores. |
closing as we have done this elsewhere |
Removes a lot of stores from the v2 codebase. I basically removed every store that required an extra dependency (other than fsspec), and also removed the N5 stores and the sqlite store. The logic for doing this is to thin down the v2 codebase.
Depends on #1742.
TODO: