Skip to content

DEPR: Compatibility with NumPy in ops #48277

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rhshadrach opened this issue Aug 26, 2022 · 4 comments
Closed

DEPR: Compatibility with NumPy in ops #48277

rhshadrach opened this issue Aug 26, 2022 · 4 comments
Labels
Compat pandas objects compatability with Numpy or Python functions Deprecate Functionality to remove in pandas Needs Discussion Requires discussion from core team before further action

Comments

@rhshadrach
Copy link
Member

In #47836 the *args/**kwargs were deprecated in rolling, expanding, and ewm ops. They were used to raise a specific error message to users who tried to use NumPy functions on pandas objects. For example:

ser = Series([1, 2, 3])
np.sum(ser.rolling(2))

raises

pandas.errors.UnsupportedFunctionCall: numpy operations are not valid with window objects. Use .rolling(...).sum() instead

Other Series / DataFrame ops also have kwargs for NumPy compatibility, but unlike the above, the compatibility is to allow the use of NumPy functions to work on pandas objects:

ser = Series([1, 2, 3])
np.sum(ser)

This succeeds with the value 6. If one removes the kwargs on NDFrame.sum, you instead get TypeError: sum() got an unexpected keyword argument 'out' with the above code.

For sum, the only acceptable calls using kwargs pass a subset of {'dtype': None, 'axis': None, 'keepdims': False, 'initial': None}. For any key out side of this, or any value that differs from this, we will raise.

Using NumPy functions (especially sum) on pandas objects seems somewhat odd to me, but perhaps there are use cases where one can accomplish a computation that can't be readily done using pandas alone? I've frequently used np.exp(ser), but one can use the alternative ser.apply(np.exp) and to my surprise, the apply version has a little less overhead (but performs the same on Series of 1 million rows or more). Also - even with kwargs removed, np.exp(ser) will still work, so it doesn't suffice as a useful example.

@rhshadrach rhshadrach added Compat pandas objects compatability with Numpy or Python functions Deprecate Functionality to remove in pandas Needs Discussion Requires discussion from core team before further action labels Aug 26, 2022
@jbrockmendel
Copy link
Member

i think as long as we support __array_ufunc__ we might as well support other numpy functions. ideally id like to implement __array_function__ to support all(ish) of them

@rhshadrach
Copy link
Member Author

Thanks @jbrockmendel - does this mean you think we're going the wrong direction in #47836? Or would this only be on Series/DataFrame, and other pandas objects (GroupBy, Rolling, Window) would not have support?

@jbrockmendel
Copy link
Member

Or would this only be on Series/DataFrame, and other pandas objects (GroupBy, Rolling, Window) would not have support?

Only Series/DataFrame (and Index and EA I guess) seem clearly array-like-if-you-squint to me

@rhshadrach
Copy link
Member Author

That makes sense to me. I'm going to close this issue and open a new one for improving the documentation on these only-used-for-NumPy kwargs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Compat pandas objects compatability with Numpy or Python functions Deprecate Functionality to remove in pandas Needs Discussion Requires discussion from core team before further action
Projects
None yet
Development

No branches or pull requests

2 participants