REGR: slicing an irregular DatetimeIndex with NaT fails with AssertionError

[jorisvandenbossche]

This seems to be triggered in a very specific case, i.e. an irregular time series and a missing value as last element in the index:

increment = pd.Series(pd.to_timedelta([30.0, 30.1, 29.9, 30.0], unit='s')).cumsum()
increment[-1] = None
index = pd.Timestamp("2012-01-01 09:00") + increment
df = pd.Series(range(len(index)), index=index).to_frame()

In [11]: df["2012-01-01":"2012-01-05"]
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-11-ded270432cfc> in <module>
----> 1 df["2012-01-01":"2012-01-05"]

~/miniconda3/envs/pandas11/lib/python3.8/site-packages/pandas/core/frame.py in __getitem__(self, key)
   2883             # either we have a slice or we have a string that can be converted
   2884             #  to a slice for partial-string date indexing
-> 2885             return self._slice(indexer, axis=0)
   2886 
   2887         # Do we have a (boolean) DataFrame?

~/miniconda3/envs/pandas11/lib/python3.8/site-packages/pandas/core/generic.py in _slice(self, slobj, axis)
   3552         Slicing with this method is *always* positional.
   3553         """
-> 3554         assert isinstance(slobj, slice), type(slobj)
   3555         axis = self._get_block_manager_axis(axis)
   3556         result = self._constructor(self._mgr.get_slice(slobj, axis=axis))

AssertionError: <class 'numpy.ndarray'>

[CloseChoice]

Do you know in which version this still worked?

[jorisvandenbossche]

In 1.0.x it still works, so regression in 1.1.x

[simonjayhawkins]

first bad commit: [4ac1e5f] CLN: assorted cleanups (#31938) where the assertion was added cc @jbrockmendel

[simonjayhawkins]

duplicate of #35509 so closing

[jorisvandenbossche]

It might be the same underlying cause, it's a different use case, so reopening to ensure we also test this when fixing the bug.

[simonjayhawkins]

@jorisvandenbossche in that case, probably need to make the code sample more minimal and reproducible, i.e. remove the randomness or seed.

[jorisvandenbossche]

But @simonjayhawkins thanks a lot for finding the cause! (seems just the assert that was added that we might be able to remove)

[jorisvandenbossche]

The randomness doesn't matter really; I think I quite clearly described the case when this occurs: irregular (no freq) and NaT as last element?

[jorisvandenbossche]

Updated the top post with some fixed numbers instead of the np.random.randn()