Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rolling.quantile does not behave like quantile and np.percentile. #16211

Closed
guillemborrell opened this issue May 3, 2017 · 3 comments
Closed
Labels
Numeric Operations Arithmetic, Comparison, and Logical operations Reshaping Concat, Merge/Join, Stack/Unstack, Explode

Comments

@guillemborrell
Copy link
Contributor

guillemborrell commented May 3, 2017

Code Sample

>>> import numpy as np
>>> import pandas as pd
>>> arr = np.arange(100)
>>> s = pd.Series(arr)
>>> q1 = s.quantile(0.1)
>>> q2 = s.rolling(100).quantile(0.1).iloc[-1]
>>> q3 = s.quantile(0.1, interpolation='lower')
>>> q4 = s.rolling(100).apply(np.percentile, args=(10,)).iloc[-1]
>>> q5 = s.rolling(100).apply(np.percentile, args=(10, None, None, False, 'lower')).iloc[-1]
>>> np.isclose(q2, q1)
False
>>> np.isclose(q2, q3)
True
>>> np.isclose(q2, q4)
False
>>> np.isclose(q2, q5)
True

Problem description

Current implementation of quantile as a method of the rolling window behaves like the lower interpolation of Series.quantile and np.percentile. It would be nice if the default behavior was the same.

Expected Output

>>> import numpy as np
>>> import pandas as pd
>>> arr = np.arange(100)
>>> s = pd.Series(arr)
>>> q1 = s.quantile(0.1)
>>> q2 = s.rolling(100).quantile(0.1).iloc[-1]
>>> q3 = s.quantile(0.1, interpolation='lower')
>>> q4 = s.rolling(100).apply(np.percentile, args=(10,)).iloc[-1]
>>> q5 = s.rolling(100).apply(np.percentile, args=(10, None, None, False, 'lower')).iloc[-1]
>>> np.isclose(q2, q1)
True
>>> np.isclose(q2, q3)
False
>>> np.isclose(q2, q4)
True
>>> np.isclose(q2, q5)
False
INSTALLED VERSIONS ------------------ commit: None python: 3.6.0.final.0 python-bits: 64 OS: Linux OS-release: 4.10.8-1-ARCH machine: x86_64 processor: byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: en_US.UTF-8

pandas: 0.19.2
nose: None
pip: 9.0.1
setuptools: 34.4.1
Cython: 0.25.2
numpy: 1.12.1
scipy: 0.19.0
statsmodels: None
xarray: None
IPython: 5.3.0
sphinx: 1.5.4
patsy: None
dateutil: 2.6.0
pytz: 2017.2
blosc: None
bottleneck: None
tables: 3.3.0
numexpr: 2.6.2
matplotlib: 2.0.0
openpyxl: 2.4.5
xlrd: 1.0.0
xlwt: 1.2.0
xlsxwriter: 0.9.6
lxml: 3.7.3
bs4: None
html5lib: 0.999999999
httplib2: None
apiclient: None
sqlalchemy: 1.1.9
pymysql: None
psycopg2: 2.7.1 (dt dec pq3 ext lo64)
jinja2: 2.9.6
boto: None
pandas_datareader: 0.3.0.post

@jreback
Copy link
Contributor

jreback commented May 3, 2017

dupe of #9413 yes this is pretty easy to fix if you'd like to submit a PR. see the duped issue.

@jreback jreback closed this as completed May 3, 2017
@jreback jreback added this to the No action milestone May 3, 2017
@jreback jreback added Numeric Operations Arithmetic, Comparison, and Logical operations Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels May 3, 2017
@guillemborrell
Copy link
Contributor Author

Thanks. I'll take a look at it.

@guillemborrell
Copy link
Contributor Author

Please, see PR #16216

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Numeric Operations Arithmetic, Comparison, and Logical operations Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

No branches or pull requests

2 participants