COMPAT: box int/floats in __iter__

[jreback]

xref #13236 as .map was recently clarified to box ints/floats (and all other dtypes) to python/pandas types (rather than numpy scalars). __iter__ should do the same (it already is done for datetimelikes), need to add for int/floats.

Furthermore Series.tolist() also returns python types (#10904)

This will make it more consistent with pandas iteration strategy.

We also already do this for object types

In [12]: type(list(Series([Timestamp('20130101'),Timestamp('20130101',tz='US/Eastern'),1,1.0,'foo']))[2])
Out[12]: int

.tolist()

In [3]: type(Series([1,2,3]).tolist()[2])
Out[3]: int

let's fix for pure types

In [14]: type(list(Series([1,2,3]))[2])
Out[14]: numpy.int64

When solving this issue, also have to look at:

• Inconsistent conversion in to_dict to numpy/python scalar (Inconsistent type casting between DataFrame and Series #14216)

[jreback]

cc @glaucouri
cc @gliptak
@sinhrks

[jreback]

In [4]: arr = np.array([1,2,3])

In [7]: type(list(arr)[0])
Out[7]: numpy.int64

In [9]: type(arr.tolist()[0])
Out[9]: int

interesting that numpy converts with .tolist() but NOT with list(..); seems very odd to me.

[jreback]

cc @njsmith do you know why numpy is like this? (my last comment)
@shoyer

[njsmith]

There was a big debate in the early days about whether numpy indexing should return python scalars or what. The compromise was that it doesn't, but there is also this bolted on notion of "convert to a Python style object" put in to appease the other side. This python conversion API lost the battle for users and mindshare, and these days no one uses intentionally or even realizes that it's in there, but it's still in there. Specifically the python object API is .item(...) for index-and-convert-to-python, and .tolist() as a shorthand for, well, .item(...) (I.e. passing in a literal ellipsis analogous to arr[...], so, converting the whole array recursively to python objects.) That's what you're invoking with .tolist(). The names of these functions are very misleading.

OTOH list(arr) just uses list's usual convert-from-iterable logic, so you get the expected thing.

[glaucouri]

To be honest, i'm puzzled as Jeff, the question is: Why in numpy algo you must convert a ndarray to a python list? Is this unequivocally to go in a python types (so is better to convert to native data types) ?

I think is more clear to mantain always native types, without do any automatic conversion.
"""Explicit is better than implicit"""

Eventually if someone need to exit from this logic, he know exaclty what is doing, and why.
Probably is more clear a method for this purpose , something like:
to_py_list()
iter_py()

I appreciate this approach used in pandas for Timestamp.

my 50 cents.
Gla

[sinhrks]

+1. When we iterate numpy dtype, can use .values