Skip to content

Wrap bottleneck for fast moving window aggregations #130

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
shoyer opened this issue May 15, 2014 · 4 comments
Closed

Wrap bottleneck for fast moving window aggregations #130

shoyer opened this issue May 15, 2014 · 4 comments

Comments

@shoyer
Copy link
Member

shoyer commented May 15, 2014

Like pandas, we should wrap bottleneck to create fast moving window operations and missing value operation that can be applied to xray data arrays.

As xray is designed to make it straightforward to work with high dimensional arrays, it would be particularly convenient if bottleneck had fast functions for N > 3 dimensions (see pydata/bottleneck/issues/84) but we should wrap bottleneck regardless for functions like rolling_mean, rolling_sum, rolling_min, etc.

@shoyer shoyer changed the title Wrap bottleneck for fast moving window and missing value operations Wrap bottleneck for fast moving window functions and NA aware aggregations Aug 3, 2014
@shoyer shoyer added this to the 1.0 milestone Aug 3, 2014
@shoyer shoyer changed the title Wrap bottleneck for fast moving window functions and NA aware aggregations Wrap bottleneck for fast moving window aggregations Feb 18, 2015
@shoyer
Copy link
Member Author

shoyer commented Feb 18, 2015

Wrapping bottleneck for operations like nanmean was fixed in #304.

@shoyer
Copy link
Member Author

shoyer commented Jun 29, 2015

Some further thoughts:

  • I would probably implement these as methods on Dataset/DataArray (e.g., ds.rolling_mean(dim='time', window=5) instead of functions. This is different from pandas, but seems slightly more in line with how everything else in pandas/xray operates.
  • I don't have a strong opinion whether we use the name rolling (like pandas) or moving/move (like bottleneck). Maybe rolling is slightly more obvious?
  • These functions from bottleneck could be easily extended to work on dask arrays by using using ghost: http://dask.pydata.org/en/latest/ghost.html

@shoyer
Copy link
Member Author

shoyer commented Aug 19, 2015

There's been some related discussion about a better API for the rolling aggregation functions in pandas recently: pandas-dev/pandas#10702

@jhamman
Copy link
Member

jhamman commented Feb 20, 2016

closed via #668

@jhamman jhamman closed this as completed Feb 20, 2016
keewis pushed a commit to keewis/xarray that referenced this issue Jan 17, 2024
* Fix pointer not at the start of the file error

* whatsnew

Co-authored-by: William Roberts <wroberts@hadamard.ssec.wisc.edu>
Co-authored-by: Tom Nicholas <thomas.nicholas@columbia.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants