Update notebook to sync with icechunk api #327
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
abarciauskas-bgse
merged commit ad17b83
into
zarr-developers:icechunk-append
1 of 2 checks passed
abarciauskas-bgse
added a commit
that referenced
this pull request
Dec 5, 2024
* Initial attempt at appending * Working on tests for generate chunk key function * Linting * Refactor gen virtual dataset method * Fix spelling * Linting * Linting * Linting * Passing compression test * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * linting * Fix test failing due to incorrect dtype * linting * Linting * Remove obsolete test file for appending * Create netcdf4 files factor in conftest * Linting * Refactor to use combineable zarr arrays * linting * Implement no append dim test * Add test for when append dim is not in dims * Fix mypy errors * type ignore import untyped zarr * Use Union type for check_combineable_zarr_arrays arg * Fix import * Fix imports for get_codecs * use new factory in test * Remove need for dask in fixture * Fix for when zarr is not installed * Address test failures * Add get_codecs file * Add dask to upstream * Remove dependency on dask and h5netcdf engine * Remove obsolete comment * Remove duplicate zarr array type check * Move codecs module and type output * Actually add codecs file * Fix merge mistake * Ignore import untyped * Add tests for codecs * Resolve mypy errors * Fix test * Import zarr in function * Use existing importorskip function * Modify comments * Comment updates and spelling of combinable * Revert change to check compatible encoding * Ignore zarr untyped import errors * Implement a manifest.utils module * pass the array into resize_array Co-authored-by: Tom Nicholas <tom@cworthy.org> * Refactor resize_array * Remove unnecessary zarr imports * Add pinned version of icechunk as an optional dependency * Add append_dim in docstring * Kludgy solution to v2 v3 codecs difference * Add normalize to v3 parameter * Add more info to docstring * Fix typing issues * Add decorator for zarr python v3 test * Fix mypy and ruff errors * Only append if append_dim in dims * Add example notebook * Add a runtime * Add failing test * Fix multiple appends * Fix test error message * Add new cell to notebook to display original time chunk * Upgrade icechunk to 1.0.0a5 * Upgrade icechunk in upstream.yml * Updated notebook with kechunk comment an upgraded icechunk version * Modify test so it fails without updated icechunk * Update icechunk dependency * Fix mypy errors * update icechunk version in pyproject * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove obsolete comment * Use icechunk 0.1.0a7 * Updated notebook * Updated notebook * print store * Update notebook (#327) Co-authored-by: Aimee Barciauskas <aimee@developmentseed.org> * Add append to examples * Add to releases.rst * Revert change to .gitignore * Update ci/upstream.yml Co-authored-by: Tom Nicholas <tom@cworthy.org> * Update pyproject.toml Co-authored-by: Tom Nicholas <tom@cworthy.org> * Update virtualizarr/tests/test_writers/test_icechunk.py Co-authored-by: Tom Nicholas <tom@cworthy.org> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update virtualizarr/accessor.py Co-authored-by: Tom Nicholas <tom@cworthy.org> * Separate out multiple arrays test --------- Co-authored-by: Tom Nicholas <tom@cworthy.org> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Matthew Iannucci <matthew@earthmover.io>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
docs/releases.rstapi.rst