Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

REF: consolidate object dtype handling for any/all #42175

Closed
mzeitlin11 opened this issue Jun 21, 2021 · 1 comment
Closed

REF: consolidate object dtype handling for any/all #42175

mzeitlin11 opened this issue Jun 21, 2021 · 1 comment
Labels
Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff Dtype Conversions Unexpected or buggy dtype conversions Refactor Internal refactoring of code

Comments

@mzeitlin11
Copy link
Member

From #42085 (comment), for groupby any/all we have bool coercing logic in

if is_object_dtype(vals):
# GH#37501: don't raise on pd.NA when skipna=True
if skipna:
vals = np.array([bool(x) if not isna(x) else True for x in vals])
else:
vals = np.array([bool(x) for x in vals])

Would be nice to share this with handling in

values, _, _, _, _ = _get_values(values, skipna, fill_value=True, mask=mask)
# For object type, all won't necessarily return
# boolean values (numpy/numpy#4352)
if is_object_dtype(values):
values = values.astype(bool)

though complicated by shape requirement differences and the complicated nature of _get_values

@mzeitlin11 mzeitlin11 added Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff Dtype Conversions Unexpected or buggy dtype conversions Refactor Internal refactoring of code labels Jun 21, 2021
@mzeitlin11 mzeitlin11 added this to the Contributions Welcome milestone Jun 21, 2021
@mroeschke mroeschke removed this from the Contributions Welcome milestone Oct 13, 2022
@jbrockmendel
Copy link
Member

I'm on board in principle, but it isn't obvious to me how to move forward with this. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff Dtype Conversions Unexpected or buggy dtype conversions Refactor Internal refactoring of code
Projects
None yet
Development

No branches or pull requests

3 participants