Skip to content

ENH: add quantile method to resample #15023

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jorisvandenbossche opened this issue Dec 31, 2016 · 5 comments · Fixed by #22304
Closed

ENH: add quantile method to resample #15023

jorisvandenbossche opened this issue Dec 31, 2016 · 5 comments · Fixed by #22304
Milestone

Comments

@jorisvandenbossche
Copy link
Member

Problem description

The quantile method is currently not supported on resample. While resample(...).agg(lambda x: x.quantile(..)) works fine, it would be nice to add the resample(..).quantile(..) shortcut.

Code Sample

In [19]: ts = pd.Series(range(20), index=pd.date_range('2016-01-01', periods=20))

In [21]: ts.resample('W').agg(lambda x: x.quantile(0.75))
Out[21]: 
2016-01-03     1.5
2016-01-10     7.5
2016-01-17    14.5
2016-01-24    18.5
Freq: W-SUN, dtype: float64

So it would be nice the following would give the same result:

In [22]: ts.resample('W').quantile(0.75)
/home/joris/miniconda3/envs/dev/bin/ipython:1: FutureWarning: 
.resample() is now a deferred operation
You called quantile(...) on this deferred object which materialized it into a series
by implicitly taking the mean.  Use .resample(...).mean() instead
  #!/home/joris/miniconda3/envs/dev/bin/python
Out[22]: 14.25

The fact that this currently implicitly takes the mean before calculating the quantile (ts.resample('W').mean().quantile(0.75)) would make this change slightly API breaking.

Using pandas master, 0.19.0+289.g1bf94c8

@jorisvandenbossche jorisvandenbossche added this to the Next Major Release milestone Dec 31, 2016
@jorisvandenbossche
Copy link
Member Author

BTW, using grouby this also works (and on rolling works as well):

In [23]: ts.groupby(pd.Grouper(freq='W')).quantile(0.75)
Out[23]: 
2016-01-03     1.5
2016-01-10     7.5
2016-01-17    14.5
2016-01-24    18.5
Freq: W-SUN, dtype: float64

In general should maybe check whether there are also other methods not working on resample.

@jreback
Copy link
Contributor

jreback commented Dec 31, 2016

yeah, we have specificially defined Resampler methods (as we do for groupby).

@srinivasreddy
Copy link
Contributor

I am taking it

@jorisvandenbossche
Copy link
Member Author

@jreback can you remember if there was a specific reason that quantile was left out? or just an oversight?

@jreback
Copy link
Contributor

jreback commented Jan 2, 2017

IIRC, some methods (e.g. cython defined ones, and some others), are specifically defined in .groupby, so these are 'named' methods and were transfered to .resample. However other methods are dispatched (IOW, on the whitelist but not defined as above). I don't think I audited all of these for suitability.

So in theory, we should evaluate these and define if appropriate (or possibly make a general method to make these definitions).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants