Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Indexing datetime broken with pandas 1.1.0 #4306

Closed
leifdenby opened this issue Aug 4, 2020 · 2 comments
Closed

Indexing datetime broken with pandas 1.1.0 #4306

leifdenby opened this issue Aug 4, 2020 · 2 comments

Comments

@leifdenby
Copy link
Contributor

Code below works with pandas<=1.0.5 and is broken with the most recent version (pandas==1.1.0) independent of xarray version (tried 0.16.0 and 0.15.0)

import pandas as pd
import xarray as xr
import numpy as np

dates = pd.date_range("2000-01-01", periods=5)
ds = xr.Dataset(coords=dict(dates=dates))
ds['v'] = ("dates"), np.arange(ds.dates.count())

ds.sel(dates=ds.dates.values[2])
The `.sel` operation produces a KeyError in `pandas/core/indexes/datetimes.py`:
Traceback (most recent call last):
  File "datetime_problem.py", line 11, in 
    ds.sel(dates=ds.dates.values[2])
  File "/Users/leifdenby/miniconda3/envs/lagtraj/lib/python3.8/site-packages/xarray/core/dataset.py", line 2101, in sel
    pos_indexers, new_indexes = remap_label_indexers(
  File "/Users/leifdenby/miniconda3/envs/lagtraj/lib/python3.8/site-packages/xarray/core/coordinates.py", line 396, in remap_label_indexers
    pos_indexers, new_indexes = indexing.remap_label_indexers(
  File "/Users/leifdenby/miniconda3/envs/lagtraj/lib/python3.8/site-packages/xarray/core/indexing.py", line 270, in remap_label_indexers
    idxr, new_idx = convert_label_indexer(index, label, dim, method, tolerance)
  File "/Users/leifdenby/miniconda3/envs/lagtraj/lib/python3.8/site-packages/xarray/core/indexing.py", line 189, in convert_label_indexer
    indexer = index.get_loc(
  File "/Users/leifdenby/miniconda3/envs/lagtraj/lib/python3.8/site-packages/pandas/core/indexes/datetimes.py", line 622, in get_loc
    raise KeyError(key)
KeyError: 946857600000000000

Environment:

Output of xr.show_versions() INSTALLED VERSIONS ------------------ commit: None python: 3.8.5 | packaged by conda-forge | (default, Jul 24 2020, 01:06:20) [Clang 10.0.1 ] python-bits: 64 OS: Darwin OS-release: 18.0.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: en_GB.UTF-8 LANG: None LOCALE: en_GB.UTF-8 libhdf5: 1.10.5 libnetcdf: 4.6.3

xarray: 0.16.0
pandas: 1.1.0
numpy: 1.19.1
scipy: 1.5.2
netCDF4: 1.5.4
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: None
cftime: 1.2.1
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: 2.22.0
distributed: None
matplotlib: 3.3.0
cartopy: None
seaborn: None
numbagg: None
pint: None
setuptools: 49.2.0.post20200712
pip: 20.1.1
conda: None
pytest: 6.0.1
IPython: 7.16.1
sphinx: None

Apologies if this is a know issue. I tried to work out whether this is an issue with pandas or xarray (I assume it is with pandas), but couldn't find the right piece of code. Happy to fix the issue if someone could show me what needs to change.

@keewis
Copy link
Collaborator

keewis commented Aug 4, 2020

Thanks for the report. I think we're already working on this, see #4283 and #4292.

@leifdenby
Copy link
Contributor Author

Ah, sorry. I tried to look for an existing issue. I'll close this in that case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants