Skip to content

API: allow single element boolean Series to improve on numpy behavior( related GH4657) #4738

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Oct 1, 2013
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions doc/source/basics.rst
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,7 @@ Boolean Reductions

.. _basics.reductions:

Furthermore, you can apply the reduction functions: ``any()`` and ``all()`` to provide a
Furthermore, you can apply the reductions: ``empty``, ``any()``, ``all()``, and ``bool()`` to provide a
way to summarize these results.

.. ipython:: python
Expand All @@ -233,7 +233,7 @@ You can reduce to a final boolean value.

(df>0).any().any()

Finally you can test if a pandas object is empty, via the ``empty`` property.
You can test if a pandas object is empty, via the ``empty`` property.

.. ipython:: python

Expand Down Expand Up @@ -262,6 +262,15 @@ Finally you can test if a pandas object is empty, via the ``empty`` property.
ValueError: The truth value of an array is ambiguous. Use a.empty, a.any() or a.all().


To evaluate single-element pandas objects in a boolean context, use the method ``.bool()``:

.. ipython:: python

Series([True]).bool()
Series([False]).bool()
DataFrame([[True]]).bool()
DataFrame([[False]]).bool()

See :ref:`gotchas<gotchas.truth>` for a more detailed discussion.


Expand Down
9 changes: 9 additions & 0 deletions doc/source/gotchas.rst
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,15 @@ or return if ``any`` value is ``True``.
print("I am any")
>>> I am any

To evaluate single-element pandas objects in a boolean context, use the method ``.bool()``:

.. ipython:: python

Series([True]).bool()
Series([False]).bool()
DataFrame([[True]]).bool()
DataFrame([[False]]).bool()

See :ref:`boolean reductions<basics.reductions>` for more examples.

Bitwise boolean
Expand Down
2 changes: 1 addition & 1 deletion doc/source/release.rst
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ API Changes

- Infer and downcast dtype if ``downcast='infer'`` is passed to ``fillna/ffill/bfill`` (:issue:`4604`)
- ``__nonzero__`` for all NDFrame objects, will now raise a ``ValueError``, this reverts back to (:issue:`1073`, :issue:`4633`)
behavior.
behavior. Add ``.bool()`` method to ``NDFrame`` objects to facilitate evaluating of single-element boolean Series
- ``DataFrame.update()`` no longer raises a ``DataConflictError``, it now
will raise a ``ValueError`` instead (if necessary) (:issue:`4732`)
- ``Series.isin()`` and ``DataFrame.isin()`` now raise a ``TypeError`` when
Expand Down
18 changes: 16 additions & 2 deletions doc/source/v0.13.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -55,10 +55,14 @@ API changes
index.set_names(["bob", "cranberry"], inplace=True)

- Infer and downcast dtype if ``downcast='infer'`` is passed to ``fillna/ffill/bfill`` (:issue:`4604`)
- Remove deprecated ``Factor`` (:issue:`3650`)
- Remove deprecated ``set_printoptions/reset_printoptions`` (:issue:``3046``)
- Remove deprecated ``_verbose_info`` (:issue:`3215`)
- ``__nonzero__`` for all NDFrame objects, will now raise a ``ValueError``, this reverts back to (:issue:`1073`, :issue:`4633`)
behavior. See :ref:`gotchas<gotchas.truth>` for a more detailed discussion.
behavior. Added the ``.bool()`` method to ``NDFrame`` objects to facilitate evaluating of single-element boolean Series
See :ref:`gotchas<gotchas.truth>` for a more detailed discussion.

This prevent behaviors like (which will now all raise ``ValueError``)
This prevents behaviors like (which will now all raise ``ValueError``)

.. code-block:: python

Expand All @@ -68,6 +72,16 @@ API changes
df1 and df2
s1 and s2


To evaluate single-element pandas objects in a boolean context, use the method ``.bool()``:

.. ipython:: python

Series([True]).bool()
Series([False]).bool()
DataFrame([[True]]).bool()
DataFrame([[False]]).bool()

- All non-Index NDFrames (``Series``, ``DataFrame``, ``Panel``, ``Panel4D``,
``SparsePanel``, etc.), now support the entire set of arithmetic operators
and arithmetic flex methods (add, sub, mul, etc.). ``SparsePanel`` does not
Expand Down
17 changes: 16 additions & 1 deletion pandas/core/generic.py
Original file line number Diff line number Diff line change
Expand Up @@ -596,10 +596,25 @@ def empty(self):
return not all(len(self._get_axis(a)) > 0 for a in self._AXIS_ORDERS)

def __nonzero__(self):
raise ValueError("The truth value of an array is ambiguous. Use a.empty, a.item(), a.any() or a.all().")
raise ValueError("The truth value of a {0} is ambiguous. "
"Use a.empty, a.bool(), a.item(), a.any() or a.all().".format(self.__class__.__name__))

__bool__ = __nonzero__

def bool(self):
""" Return the bool of a single element PandasObject
This must be a boolean scalar value, either True or False

Raise a ValueError if the PandasObject does not have exactly
1 element, or that element is not boolean """
v = self.squeeze()
if isinstance(v, (bool,np.bool_)):
return bool(v)
elif np.isscalar(v):
raise ValueError("bool cannot act on a non-boolean single element {0}".format(self.__class__.__name__))

self.__nonzero__()

def __abs__(self):
return self.abs()

Expand Down
2 changes: 1 addition & 1 deletion pandas/core/series.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

import operator
import types
import warnings

from numpy import nan, ndarray
import numpy as np
Expand Down Expand Up @@ -913,7 +914,6 @@ def to_string(self, buf=None, na_rep='NaN', float_format=None,
"""

if nanRep is not None: # pragma: no cover
import warnings
warnings.warn("nanRep is deprecated, use na_rep", FutureWarning)
na_rep = nanRep

Expand Down
10 changes: 5 additions & 5 deletions pandas/io/tests/test_pytables.py
Original file line number Diff line number Diff line change
Expand Up @@ -2662,18 +2662,18 @@ def test_select_dtypes(self):
df = DataFrame(np.random.randn(5,2), columns =['A','B'])
df['object'] = 'foo'
df.ix[4:5,'object'] = 'bar'
df['bool'] = df['A'] > 0
df['boolv'] = df['A'] > 0
_maybe_remove(store, 'df')
store.append('df', df, data_columns = True)

expected = df[df.bool == True].reindex(columns=['A','bool'])
expected = df[df.boolv == True].reindex(columns=['A','boolv'])
for v in [True,'true',1]:
result = store.select('df', Term('bool == %s' % str(v)), columns = ['A','bool'])
result = store.select('df', Term('boolv == %s' % str(v)), columns = ['A','boolv'])
tm.assert_frame_equal(expected, result)

expected = df[df.bool == False ].reindex(columns=['A','bool'])
expected = df[df.boolv == False ].reindex(columns=['A','boolv'])
for v in [False,'false',0]:
result = store.select('df', Term('bool == %s' % str(v)), columns = ['A','bool'])
result = store.select('df', Term('boolv == %s' % str(v)), columns = ['A','boolv'])
tm.assert_frame_equal(expected, result)

# integer index
Expand Down
37 changes: 35 additions & 2 deletions pandas/tests/test_generic.py
Original file line number Diff line number Diff line change
Expand Up @@ -205,11 +205,31 @@ def test_get_numeric_data_preserve_dtype(self):

def test_nonzero_single_element(self):

# allow single item via bool method
s = Series([True])
self.assertRaises(ValueError, lambda : bool(s))
self.assert_(s.bool() is True)

s = Series([False])
self.assertRaises(ValueError, lambda : bool(s))
self.assert_(s.bool() is False)

# single item nan to raise
for s in [ Series([np.nan]), Series([pd.NaT]), Series([True]), Series([False]) ]:
self.assertRaises(ValueError, lambda : bool(s))

for s in [ Series([np.nan]), Series([pd.NaT])]:
self.assertRaises(ValueError, lambda : s.bool())

# multiple bool are still an error
for s in [Series([True,True]), Series([False, False])]:
self.assertRaises(ValueError, lambda : bool(s))
self.assertRaises(ValueError, lambda : s.bool())

# single non-bool are an error
for s in [Series([1]), Series([0]),
Series(['a']), Series([0.0])]:
self.assertRaises(ValueError, lambda : bool(s))
self.assertRaises(ValueError, lambda : s.bool())


class TestDataFrame(unittest.TestCase, Generic):
_typ = DataFrame
Expand All @@ -220,6 +240,19 @@ def test_rename_mi(self):
index=MultiIndex.from_tuples([("A",x) for x in ["a","B","c"]]))
result = df.rename(str.lower)

def test_nonzero_single_element(self):

# allow single item via bool method
df = DataFrame([[True]])
self.assert_(df.bool() is True)

df = DataFrame([[False]])
self.assert_(df.bool() is False)

df = DataFrame([[False, False]])
self.assertRaises(ValueError, lambda : df.bool())
self.assertRaises(ValueError, lambda : bool(df))

def test_get_numeric_data_preserve_dtype(self):

# get the numeric data
Expand Down