Skip to content

BUG: FutureWarning on pct_change() suggests a solution that does not always work. #54981

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
3 tasks done
karrmagadgeteer2 opened this issue Sep 3, 2023 · 1 comment · Fixed by #54983
Closed
3 tasks done
Labels
Bug Warnings Warnings that appear or should be added to pandas
Milestone

Comments

@karrmagadgeteer2
Copy link

karrmagadgeteer2 commented Sep 3, 2023

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd


if __name__ == "__main__":
    arrays_a = [
        [1, 101],
        [2, 102],
        [3, None],
        [4, 104],
        [5, 105],
    ]
    dfa = pd.DataFrame(arrays_a)
    dfa1 = dfa.pct_change()                  # warning as expected here
    dfa2 = dfa.ffill().pct_change()          # This is the case covered by warning

    arrays_b = [
        [1, None],
        [2, 102],
        [3, 103],
        [4, 104],
        [5, 105],
    ]
    dfb = pd.DataFrame(arrays_b)
    dfb1 = dfb.pct_change()                  # warning as expected here
    dfb2 = dfb.ffill().pct_change()          # <- would not expect warning here
    dfb3 = dfb.ffill().bfill().pct_change()  # Adding bfill obvious solution

Issue Description

When the missing value is at the beginning of a column the solution proposed in the warning has no effect.

Expected Behavior

Ideally the warning would not suggest a solution that may not work. Perhaps a solution would be that a check is made if there are one or more missing values at the beginning of any of the arrays in the axis, and based on this make an alternate suggestion.

Installed Versions

INSTALLED VERSIONS

commit : ba1cccd
python : 3.10.11.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.22621
machine : AMD64
processor : Intel64 Family 6 Model 142 Stepping 10, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : Swedish_Sweden.1252

pandas : 2.1.0
numpy : 1.25.2
pytz : 2023.3
dateutil : 2.8.2
setuptools : 65.5.0
pip : 23.2.1
Cython : None
pytest : 7.4.1
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : 3.1.2
lxml.etree : 4.9.3
html5lib : 1.1
pymysql : None
psycopg2 : None
jinja2 : 3.1.2
IPython : 8.14.0
pandas_datareader : 0.10.0
bs4 : 4.12.2
bottleneck : None
dataframe-api-compat: None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : 3.7.2
numba : None
numexpr : None
odfpy : None
openpyxl : 3.1.2
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : 1.0.10
s3fs : None
scipy : 1.11.2
sqlalchemy : None
tables : None
tabulate : 0.9.0
xarray : None
xlrd : 2.0.1
zstandard : None
tzdata : 2023.3
qtpy : None
pyqt5 : None

@karrmagadgeteer2 karrmagadgeteer2 added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Sep 3, 2023
@phofl phofl added Warnings Warnings that appear or should be added to pandas and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Sep 3, 2023
@phofl phofl added this to the 2.1.1 milestone Sep 3, 2023
@torext
Copy link
Contributor

torext commented Sep 5, 2023

@phofl I arrived here because I was about to submit a bug report on this issue. However, looking at the commited change above I notice it won't fix the issue for DataFrames: the 3 changed lines in pandas/core/generic.py only work if self is a Series, not when it's a DataFrame, I believe.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Warnings Warnings that appear or should be added to pandas
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants