opendap / dap4 support for pydap backend #10182
Conversation
github-actions
bot
added
topic-backends
topic-indexing
topic-documentation
topic-plotting
topic-performance
topic-zarr
|
Apologies for all the added labels / contributors. Didn't have much time to work on this pr this week, and so I had to rebase and merge the main branch to this one. Changed files are correct though. |
|
Ready for review :) |
| @@ -49,36 +51,37 @@ def __getitem__(self, key): | |||
| ) | |||
|
|
|||
| def _getitem(self, key): | |||
| # pull the data from the array attribute if possible, to avoid | |||
| # downloading coordinate data twice | |||
| array = getattr(self.array, "array", self.array) | |||
There was a problem hiding this comment.
don't we need this any more?
There was a problem hiding this comment.
I set the output_grid=False by default on pydap, so that should not be necessary. But I'll double check (All the tests passed on the xarray-tests environment)
pytest -v xarray/tests/test_backends.py --run-network-tests
xarray/tests/test_backends.py
Outdated
| assert "chunks" in actual | ||
| assert actual["chunks"] == ("auto" if has_zarr_v3 else None) | ||
|
|
||
| var = xr.Variable("x", [1, 2], encoding={"chunks": (1,)}) | ||
| actual = backends.zarr.extract_zarr_variable_encoding(var, zarr_format=3) | ||
| actual = backends.zarr.extract_zarr_variable_encoding(var) |
There was a problem hiding this comment.
this is needed, let's undo this change
There was a problem hiding this comment.
I don't really know where this is from... I merged main branch into this PR and did a git pull --rebase origin pydap4 . Totally happy to change that.
There was a problem hiding this comment.
feel free to just merge, no rebase is necessary if that makes it easier
xarray/tests/test_backends.py
Outdated
| @@ -2628,10 +2627,12 @@ def test_hidden_zarr_keys(self) -> None: | |||
| for var in expected.variables.keys(): | |||
| assert self.DIMENSION_KEY not in expected[var].attrs | |||
|
|
|||
| if has_zarr_v3: | |||
xarray/tests/test_backends.py
Outdated
| @@ -3343,67 +3344,6 @@ def test_cache_members(self) -> None: | |||
| observed_keys_2 = sorted(zstore_mut.array_keys()) | |||
| assert observed_keys_2 == sorted(array_keys + [new_key]) | |||
|
|
|||
| @requires_dask | |||
for more information, see https://pre-commit.ci
|
It looks like the failing for ubuntu-latest py3.10 min-all-deps has pydap pinned to |
github-actions
bot
added
the
dependencies
There have been some (revitalizing) upgrades over the last year to pydap (with some more to come), including authentication, pydap data model (dap2 vs dap4). This
DraftPR aims toDataTree.output_gridoption. It is now set toFalseas default inpydap.output_gridas an input parameter. It will always beFalseas default (and there is no need for it to beTrue, as it only adds unnecessary logic).This PR enables, for example (check test opendap dap4 url)