BUG: IndexError: .iloc requires numeric indexers, got [0 1 2 3]

[anbjork]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

>>> df = pd.DataFrame([[0,1,2,3,4],[5,6,7,8,9]])
>>> df.iloc[:, pd.Series([0,1,2,3], dtype = pd.Int64Dtype())]
IndexError: .iloc requires numeric indexers, got [0 1 2 3]

# From stack trace
#    1475 # check that the key has a numeric dtype
#    1476 if not is_numeric_dtype(arr.dtype):
# -> 1477     raise IndexError(f".iloc requires numeric indexers, got {arr}")
#    1479 # check that the key does not exceed the maximum size of the index
#    1480 if len(arr) and (arr.max() >= len_axis or arr.min() < -len_axis):

>>> df.iloc[:, pd.Series([0,1,2,3])]
# Works

# Check dtypes. Note int64 vs Int64Dtype
>>> pd.Series([0,1,2,3]).dtype                                 
dtype('int64')                                                                         
>>> pd.Series([0,1,2,3], dtype = pd.Int64Dtype()).dtype        
Int64Dtype()

# Series.iloc works
pd.Series([1,2,3,4,5,6]).iloc[pd.Series([0,1,2,3], dtype = pd.Int64Dtype())]

Issue Description

Pandas.DataFrame.iloc doesn't understand that Pandas pd.Int64Dtype is numeric, and refuses to index with it.
Pandas.Series.iloc works.

Expected Behavior

That pd.DataFrame.iloc would recognise that pd.Int64Dtype is numeric and correctly index with it.

Installed Versions

INSTALLED VERSIONS

commit : 91111fd
python : 3.9.6.final.0
python-bits : 64
OS : Linux
OS-release : 5.10.16.3-microsoft-standard-WSL2
Version : #1 SMP Fri Apr 2 22:23:49 UTC 2021
machine : x86_64
processor :
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.5.1
numpy : 1.21.2
pytz : 2021.1
dateutil : 2.8.2
setuptools : 58.3.0
pip : 22.3
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.9.1
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 3.0.1
IPython : 8.5.0
pandas_datareader: None
bs4 : None
bottleneck : None
brotli : None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : 3.5.1
numba : None
numexpr : None
odfpy : None
openpyxl : 3.0.9
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.7.1
snappy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
zstandard : None
tzdata : None

[mzeitlin11]

Thanks for reporting this @anbjork! Looks like the same issue as #39133?

[phofl]

I think this is a bit different. The op should work already

[topper-123]

.iloconly accepts integers, but IntegerArray can hold pd.NA and supplying pd.NA to .iloc is supposed to raise.

>>> import pandas as pd
>>> df = pd.DataFrame([[0,1,2,3,4],[5,6,7,8,9]])
>>> iarr = pd.array([0,1,2,pd.NA], dtype = pd.Int64Dtype())
>>> df.iloc[:, pd.Series(iarr)]
IndexError: ...
>>> df.iloc[:, iarr)
IndexError: ...

So IntegerArray cannot be used inplace for np.ndarray with an int dtype currently. Also notice that iarr._data is array([1, 2, 1] which would definitely give a wrong result.

Also notice that the docs don't mention integer indexing with IntegerArray, thought it insinuates that is does here. The docs should be explicit about if they accept IntegerArray in .iloc and, if accepted, how it deals with arrays with pd.NA values.

I lean toward not allowing integer indexing with IntegerArray and that the current behavior is correct, but the error message and docs should be improved. Maybe someone could make an explicit proposal where it's decided how pd.NA should be treated in .iloc operations to see if that makes sense.

[phofl]

No, as long as numpy dtype Series are supposed to work, the ea series should work as well as long as no missing values are present. "Int64" without missing values are integers as well, so no reason to disallow. This gets more important since we are able to return nullable dtypes from I/O where everything is nullable, not only if nulls are present.

Btw using Int64 as row indexer worked already.

[topper-123]

Ok, Didn't know that. Maybe it could be more explicit in the docs, for example here?

[phofl]

Can you make a suggestion? Personally, the sentence

Operations involving an integer array will behave similar to NumPy arrays

tells me, that I can use both interchangeable as long as no missing values are present

[topper-123]

Yeah, to me e.g. this is unclear: "...and the data will be coerced to another dtype if needed." In the case of .iloc indexing, it was not clear to me what coerced meant (or could mean).

I think a warning at the end of this paragraph about using Int64 for .iloc indexing, given that it's up to the user to ensure the array doesn't have a pd.Na?