Same files in open_mfdataset() unclear error message

[antscloud]

When using xr.open_mfdataset() with two exact same files by mistake, it causes an unclear error message

What happened:

With of course the time dimension existing :

ds=xr.open_mfdataset(["some_file.nc","some_file.nc"],concat_dim="time",engine="netcdf4")

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)

~/.local/src/miniconda/envs/minireobs/lib/python3.8/site-packages/xarray/backends/api.py in open_mfdataset(paths, chunks, concat_dim, compat, preprocess, engine, lock, data_vars, coords, combine, parallel, join, attrs_file, **kwargs)
    966             # Redo ordering from coordinates, ignoring how they were ordered
    967             # previously
--> 968             combined = combine_by_coords(
    969                 datasets,
    970                 compat=compat,

~/.local/src/miniconda/envs/minireobs/lib/python3.8/site-packages/xarray/core/combine.py in combine_by_coords(datasets, compat, data_vars, coords, fill_value, join, combine_attrs)
    762     concatenated_grouped_by_data_vars = []
    763     for vars, datasets_with_same_vars in grouped_by_vars:
--> 764         combined_ids, concat_dims = _infer_concat_order_from_coords(
    765             list(datasets_with_same_vars)
    766         )

~/.local/src/miniconda/envs/minireobs/lib/python3.8/site-packages/xarray/core/combine.py in _infer_concat_order_from_coords(datasets)
    106 
    107     if len(datasets) > 1 and not concat_dims:
--> 108         raise ValueError(
    109             "Could not find any dimension coordinates to use to "
    110             "order the datasets for concatenation"

ValueError: Could not find any dimension coordinates to use to order the datasets for concatenation

What you expected to happen:
A warning saying that we are using the same dataset ? A more explicit error message (exact same dimensions) ? No error and no concatenation, remove duplicated datasets?

Environment:

Output of xr.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.8.5 (default, Sep 4 2020, 07:30:14)
[GCC 7.3.0]
python-bits: 64
OS: Linux
OS-release: 5.4.0-72-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: fr_FR.UTF-8
LOCALE: fr_FR.UTF-8
libhdf5: 1.10.4
libnetcdf: 4.7.3

xarray: 0.17.0
pandas: 1.1.1
numpy: 1.19.2
scipy: 1.5.2
netCDF4: 1.5.3
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: None
cftime: 1.2.1
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: 2021.04.0
distributed: 2021.04.0
matplotlib: 3.3.1
cartopy: None
seaborn: None
numbagg: None
pint: None
setuptools: 49.6.0.post20200814
pip: 20.2.2
conda: None
pytest: 6.1.1
IPython: 7.18.1
sphinx: None

[TomNicholas]

Thanks for reporting this @antscloud - the error is from the combine_by_coords function that can be used inside open_mfdataset, but I think something in the logic has gone wrong to give you this particularly unhelpful error. I'll have a look now

[antscloud]

Thank you for the comment @TomNicholas, if it can help i think this error comes from this particular line : https://github.com/pydata/xarray/blob/master/xarray/core/combine.py#L81
from a missing else statement. Indeed, the concat_dims is empty due to initialization and then raise the error.

[TomNicholas]

Okay so I found the problem. If you also pass combine='nested' then it should work - that's what you're supposed to do to open your two datasets.

We currently get an unhelpful error because the default argument combine='by_coords' should be incompatible with passing any value for concat_dim (because the function combine_by_coords doesn't accept any concat_dim argument), but right now that possibility is not checked in any way (it used to be, but got lost in a deprecation cycle...)

To fix this I would simply add an input check that raises an error like
"passing a value for concat_dim can only be used with combine='nested', not combine='by_coords'"