Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ffill does not spread chunks when used with dask.array #6997

Closed
eserie opened this issue Sep 6, 2022 · 5 comments
Closed

ffill does not spread chunks when used with dask.array #6997

eserie opened this issue Sep 6, 2022 · 5 comments
Labels
plan to close May be closeable, needs more eyeballs

Comments

@eserie
Copy link

eserie commented Sep 6, 2022

What is your issue?

When using xarray with dask.array as in the following example:

import numpy as np
import xarray as xr
import dask
ds_ = xr.Dataset()
ds_["test"] = ("dim_0",), dask.array.from_array(
    [0, 1, np.nan, np.nan, np.nan], chunks=2
)

then the ffill method does not seem to propagate through the underlying chunks used by dask.array.
In the example above, this means that

 ds_.ffill("dim_0").compute().test.values

gives

>>> array([ 0., 1., 1., 1., nan])

where

ds_.compute().ffill("dim_0").test.values

gives

>>> array([0., 1., 1., 1., 1., 1.])

Is it an expected behavior ?

@eserie eserie added the needs triage Issue that has not been reviewed by xarray team member label Sep 6, 2022
@mathause
Copy link
Collaborator

mathause commented Sep 7, 2022

What version of xarray are you using? I think this should have been fixed in #5187 (i.e. in v0.18.0).

Edit: quickly checked and the newest version gets this right.

@mathause mathause added plan to close May be closeable, needs more eyeballs and removed needs triage Issue that has not been reviewed by xarray team member labels Sep 7, 2022
@eserie
Copy link
Author

eserie commented Sep 8, 2022

Hi @mathause, I'm in version 0.18.2.

@mathause
Copy link
Collaborator

mathause commented Sep 8, 2022

Ok, I see there is another bugfix in v0.21.0 (#6118) - thus you will need at least this version for your example to work.

@eserie
Copy link
Author

eserie commented Sep 8, 2022

Ok, thanks, that solves the problem!

@mathause
Copy link
Collaborator

mathause commented Sep 8, 2022

Ok, thanks, that solves the problem!

Great to hear!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
plan to close May be closeable, needs more eyeballs
Projects
None yet
Development

No branches or pull requests

2 participants