FutureWarning when using open_mfdataset()

[arthur-e]

What is your issue?

When I attempt to use open_mfdataset() with a GLOB expression, to open multiple files:

import xarray as xr

ds = xr.open_mfdataset('data_raw/NLDAS/*NOAH*.nc')

This works but generates a FutureWarning:

/usr/local/python-env/ScienceCore/lib/python3.10/site-packages/xarray/core/concat.py:532: FutureWarning: unique with argument that is not not a Series, Index, ExtensionArray, or np.ndarray is deprecated and will raise in a future version.
  common_dims = tuple(pd.unique([d for v in vars for d in v.dims]))

My concern about the FutureWarning is that this will, at best, confuse my students and, at worst, potentially break my curriculum after some "future version." However, it's not clear that I have any control over what to do differently here.

I'm using xarray version '2023.1.0'. The netCDF4 files I'm using could be downloaded automatically (given a NASA Earthdata account) with:

import earthaccess
import xarray as xr

# Log-in, presumably using a .netrc file
auth = earthaccess.login()

# Get data from August for every year from 2008 up to (but not including) 2018
results = []
for year in range(2008, 2018):
    search = earthaccess.search_data(
        short_name = 'NLDAS_NOAH0125_M',
        version = '2.0',
        temporal = (f'{year}-08', f'{year}-08'))
    results.extend(search)

earthaccess.download(results, 'data_raw/NLDAS')

[welcome]

Thanks for opening your first issue here at xarray! Be sure to follow the issue template!
If you have an idea for a solution, we would really welcome a Pull Request with proposed changes.
See the Contributing Guide for more.
It may take us a while to respond here, but we really value your contribution. Contributors like you help make xarray better.
Thank you!

[TomNicholas]

Thanks for raising this @arthur-e . This shouldn't really have happened, but I expect this should be a simple fix.

My concern about the FutureWarning is that this will, at best, confuse my students and, at worst, potentially break my curriculum after some "future version."

That's totally understandable. Just for context, it's unlikely that your curriculum would get broken because we test against upstream development versions of packages, so as soon as a new pandas commit turns this warning into an error, we would see it as a test failure in our CI (and fix it before pandas releases).

We are however guilty of allowing our test suites to pass even they generate (lots of) warnings, which is why this one made it through to be seen by users. (see also #7164 and #8494)

[TomNicholas]

Looks like the warning is raised here, and should be fixable by wrapping the list comprehension with a np.array() call. Alternatively there is probably a neater way to write this line that doesn't require pandas (or maybe numpy).

If you (or any of your students) would like to have a go at fixing it we would welcome a pull request, else we can do it 🙂

[itsdivya1309]

Hii! I am new to xarray, but can I work on this issue?

[TomNicholas]

Hii! I am new to xarray, but can I work on this issue?

Yes, anyone is welcome to work on any issue, just submit a pull request. We don't "reserve" issues for one person to work on.

[dcherian]

@arthur-e Thanks for the report. Can you try this on latest xarray please?

This line now uses OrderedSet

xarray/xarray/core/concat.py

Line 550 in ca4f121

common_dims = tuple(utils.OrderedSet(d for v in vars for d in v.dims))

so I'm pretty sure this warning isn't being raised any more.

[arthur-e]

Thanks @dcherian, @TomNicholas. It looks like this is fixed in the latest xarray release, 2024-01-1. I not longer get a FutureWarning.