BUG: Validate numeric_only parameter in groupby aggregations #62842

Zrahay · 2025-10-25T20:43:06Z

closes BUG: groupby.<reduction>(numeric_only=) does not validate non-bool arguments #62778
Tests added and passed (https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
All code checks passed (https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit)
Added type annotations (https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

Description

This PR fixes issue #62778 where groupby aggregation methods (like mean(), sum(), std(), etc.) were incorrectly accepting non-boolean values for the numeric_only parameter.

Problem

The numeric_only parameter was only being checked for truthiness/falsiness, allowing invalid inputs like lists, strings, or integers to be passed without raising an error.

Solution

Added explicit type validation in _cython_agg_general() method in pandas/core/groupby/groupby.py
Raises ValueError with message "numeric_only accepts only Boolean values" when a non-boolean value is provided
Added comprehensive test case in pandas/tests/groupby/test_reductions.py

Example

Before (incorrect behavior):

import pandas as pd
df = pd.DataFrame({"A": range(5), "B": range(5)})
df.groupby(["A"]).mean(["B"])

After (correct behavior):

df.groupby(["A"]).mean(["B"])

Valid usage still works:

df.groupby(["A"]).mean()  # Works
df.groupby(["A"]).mean(numeric_only=True)  
df.groupby(["A"]).mean(numeric_only=False)

- Add type check for numeric_only parameter in _cython_agg_general - Raise ValueError if numeric_only is not a boolean - Add test case for validation - Closes pandas-dev#62778

Zrahay · 2025-10-25T20:47:24Z

If there are any feedbacks or issues here, please do let me know. It's my first time contributing to this repo!

rhshadrach

Thanks for the PR! Can you also add a note to the whatsnew for 3.0.

I've also updated your title to adhere to the contribution guidelines: https://pandas.pydata.org/pandas-docs/dev/development/contributing.html#making-a-pull-request

rhshadrach · 2025-10-25T20:51:02Z

pandas/core/groupby/groupby.py

+        if(isinstance(numeric_only, bool)):
+            data = self._get_data_to_aggregate(numeric_only=numeric_only, name=how)


Can you change negate this condition and raise here; then the rest of this function can go untouched which makes the diff smaller. It also decreases the amount of indentation needed, improving readability.

Also use is_bool from pandsa.core.dtypes.common. We should accept e.g. np.bool here.

rhshadrach · 2025-10-25T20:51:17Z

pandas/core/groupby/groupby.py

+        # that goes through SeriesGroupBy

-        data = self._get_data_to_aggregate(numeric_only=numeric_only, name=how)
+        # Check to confirm numeric_only is fed either True or False and no other data type


This comment repeats the code, can you remove it.

rhshadrach · 2025-10-25T20:51:28Z

pandas/core/groupby/groupby.py

    ):
        # Note: we never get here with how="ohlc" for DataFrameGroupBy;
-        #  that goes through SeriesGroupBy
+        # that goes through SeriesGroupBy


Can you revert this change.

Still needs to be reverted in order to minimize the diff.

…mment on line 1759 and reverted the comment change on line 1757

Zrahay · 2025-10-25T21:52:47Z

Hey there @rhshadrach ! I've made all the 3 changes you had requested. Please let me know if there's any other issue with anything in my PR.

Thanks!

Zrahay · 2025-10-25T22:00:01Z

Thanks for the PR! Can you also add a note to the whatsnew for 3.0.

I'm sorry to ask but where do I find whatsnew for 3.0?

rhshadrach · 2025-10-25T22:12:03Z

I'm sorry to ask but where do I find whatsnew for 3.0?

No problem! See here: https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#documenting-your-code

Zrahay · 2025-10-25T22:47:49Z

@rhshadrach Just added the note in the respective file. Do let me know if there's something else left from my end.

Cheers!

rhshadrach · 2025-10-26T12:51:03Z

pandas/core/groupby/groupby.py

    ):
        # Note: we never get here with how="ohlc" for DataFrameGroupBy;
-        #  that goes through SeriesGroupBy
+        # that goes through SeriesGroupBy


Still needs to be reverted in order to minimize the diff.

rhshadrach · 2025-10-26T12:51:52Z

pandas/tests/groupby/test_reductions.py

+    """
+    Test that numeric_only parameter only accepts boolean values.
+    See GH#62778
+    """


Suggested change

"""

Test that numeric_only parameter only accepts boolean values.

See GH#62778

"""

# GH#62778

rhshadrach · 2025-10-26T12:52:24Z

pandas/tests/groupby/test_reductions.py

+    """
+    df = pd.DataFrame({"A": range(5), "B": range(5)})
+
+    # These test cases should raise a ValueError


Can you remove this comment, it repeats the code.

rhshadrach · 2025-10-26T12:52:57Z

pandas/tests/groupby/test_reductions.py

+    # These test cases should work absolutely fine
+    df.groupby(["A"]).mean()
+    df.groupby(["A"]).mean(numeric_only=True)
+    df.groupby(["A"]).mean(numeric_only=False)


Can you remove these; the test suite has many tests for numeric_only being specified or not specified, these are not increasing our test coverage.

So should I remove this whole function from the test file or just these 3 lines of code?

Just these four lines.

@rhshadrach Made the suggested changes. I had made the changes in the comment inside groupby.py file but forgot to add it while doing git add, hence, it didn't show up in the changes here.

This version should be fine. Please let me know if any other changes are there.

… from test_reductions.py

rhshadrach

Looking good!

rhshadrach · 2025-10-26T16:20:43Z

doc/source/whatsnew/v3.0.0.rst

 - Bug in :meth:`.DataFrameGroupBy.groups` and :meth:`.SeriesGroupBy.groups` would fail when the groups were :class:`Categorical` with an NA value (:issue:`61356`)
 - Bug in :meth:`.DataFrameGroupBy.groups` and :meth:`.SeriesGroupby.groups` that would not respect groupby argument ``dropna`` (:issue:`55919`)
 - Bug in :meth:`.DataFrameGroupBy.median` where nat values gave an incorrect result. (:issue:`57926`)
+- Bug in :meth:`.DataFrameGroupBy` reductions where boolean-valued inputs were mishandled in the Cython aggregation path (``_cython_agg_general``); adding an ``is_bool`` check fixes incorrect results for some bool inputs. (:issue:`62778`)


Only mention public things in the whatsnew. Also "Boolean-valued inputs" makes it sounds like there was an issue with True and False.

Suggested change

- Bug in :meth:`.DataFrameGroupBy` reductions where boolean-valued inputs were mishandled in the Cython aggregation path (``_cython_agg_general``); adding an ``is_bool`` check fixes incorrect results for some bool inputs. (:issue:`62778`)

- Bug in :meth:`.DataFrameGroupBy` reductions where non-Boolean values were allowed for the ``numeric_only`` argument; passing a non-Boolean value will now raise (:issue:`62778`)

Sure, let me correct that.

@rhshadrach Made the required changes!

rhshadrach

lgtm, ping on green.

Zrahay added 2 commits October 25, 2025 23:50

BUG: Validate numeric_only parameter in groupby aggregations

52e07b5

BUG: Validate numeric_only parameter in groupby aggregations

b2f18ef

- Add type check for numeric_only parameter in _cython_agg_general - Raise ValueError if numeric_only is not a boolean - Add test case for validation - Closes pandas-dev#62778

Zrahay requested a review from rhshadrach as a code owner October 25, 2025 20:43

rhshadrach requested changes Oct 25, 2025

View reviewed changes

rhshadrach changed the title ~~[BUG]: Validate numeric_only parameter in groupby aggregations~~ BUG: Validate numeric_only parameter in groupby aggregations Oct 25, 2025

rhshadrach added Groupby Bug Error Reporting Incorrect or improved errors from pandas labels Oct 25, 2025

rhshadrach added this to the 3.0 milestone Oct 25, 2025

Removed the if condition and added is_bool. Removed the unecessary co…

9655a9a

…mment on line 1759 and reverted the comment change on line 1757

Added a note to the whatsnew 3.0 with respect to my issue 62778

6ef5696

rhshadrach requested changes Oct 26, 2025

View reviewed changes

Added the suggested changes. Removed the redundant comments and tests…

5a72c79

… from test_reductions.py

rhshadrach requested changes Oct 26, 2025

View reviewed changes

Change made to the comments in whatsnew 3.0

8056bb8

rhshadrach approved these changes Oct 26, 2025

View reviewed changes

		if(isinstance(numeric_only, bool)):
		data = self._get_data_to_aggregate(numeric_only=numeric_only, name=how)

	- Bug in :meth:`.DataFrameGroupBy` reductions where boolean-valued inputs were mishandled in the Cython aggregation path (``_cython_agg_general``); adding an ``is_bool`` check fixes incorrect results for some bool inputs. (:issue:`62778`)
	- Bug in :meth:`.DataFrameGroupBy` reductions where non-Boolean values were allowed for the ``numeric_only`` argument; passing a non-Boolean value will now raise (:issue:`62778`)

Uh oh!

BUG: Validate numeric_only parameter in groupby aggregations #62842

Are you sure you want to change the base?

BUG: Validate numeric_only parameter in groupby aggregations #62842

Conversation

Zrahay commented Oct 25, 2025

Description

Problem

Solution

Example

Uh oh!

Zrahay commented Oct 25, 2025

Uh oh!

rhshadrach left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Zrahay commented Oct 25, 2025

Uh oh!

Zrahay commented Oct 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rhshadrach commented Oct 25, 2025

Uh oh!

Zrahay commented Oct 25, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rhshadrach Oct 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rhshadrach left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rhshadrach left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

rhshadrach left a comment •

edited

Loading

Zrahay commented Oct 25, 2025 •

edited

Loading

rhshadrach Oct 26, 2025 •

edited

Loading