Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

open_mfdataset does not handle single pathlib.Path objects but handles single str just fine #5881

Closed
pmav99 opened this issue Oct 21, 2021 · 2 comments · Fixed by #5884
Closed
Labels

Comments

@pmav99
Copy link
Contributor

pmav99 commented Oct 21, 2021

What happened:

An exception was raised

What you expected to happen:

That a dataset object would be created.

Minimal Complete Verifiable Example:

from pathlib import Path

import xarray as xr

dataset = "xarray/tests/data/example_1.nc"

with xr.open_mfdataset(dataset) as actual:
    assert isinstance(actual, xr.Dataset)

with xr.open_mfdataset(Path(dataset)) as actual:
    assert isinstance(actual, xr.Dataset)  # this fails

with the following traceback

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/panos/Prog/git/xarray/xarray/backends/api.py", line 869, in open_mfdataset
    paths = [str(p) if isinstance(p, Path) else p for p in paths]
TypeError: 'PosixPath' object is not iterable

Anything else we need to know?:

This seems to be fixing it, but I am not familiar with the codebase to know if this is the proper way to fix it

diff --git a/xarray/backends/api.py b/xarray/backends/api.py
index 6d73946b..596ac249 100644
--- a/xarray/backends/api.py
+++ b/xarray/backends/api.py
@@ -838,6 +838,8 @@ def open_mfdataset(
     .. [1] http://xarray.pydata.org/en/stable/dask.html
     .. [2] http://xarray.pydata.org/en/stable/dask.html#chunking-and-performance
     """
+    if isinstance(paths, Path):
+        paths = paths.as_posix()
     if isinstance(paths, str):
         if is_remote_uri(paths) and engine == "zarr":
             try:

Environment:

Output of xr.show_versions()
>>> xr.show_versions()

INSTALLED VERSIONS
------------------
commit: 07de257c5884df49335496ee6347fb633a7c302c
python: 3.9.7 (default, Aug 31 2021, 13:28:12) 
[GCC 11.1.0]
python-bits: 64
OS: Linux
OS-release: 5.14.12-arch1-1
machine: x86_64
processor: 
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: ('en_US', 'UTF-8')
libhdf5: 1.12.0
libnetcdf: 4.7.4

xarray: 999
pandas: 1.3.4
numpy: 1.21.3
scipy: None
netCDF4: 1.5.7
pydap: None
h5netcdf: None
h5py: None
Nio: None
zarr: None
cftime: 1.5.1
nc_time_axis: None
PseudoNetCDF: None
rasterio: None
cfgrib: None
iris: None
bottleneck: None
dask: 2021.09.1
distributed: None
matplotlib: None
cartopy: None
seaborn: None
numbagg: None
fsspec: 2021.10.1
cupy: None
pint: None
sparse: None
setuptools: 57.4.0
pip: 21.3
conda: None
pytest: 6.2.5
IPython: None
sphinx: None
@TomNicholas
Copy link
Member

Hi @pmav99, thanks for raising this!

I would probably just fix it with

diff --git a/xarray/backends/api.py b/xarray/backends/api.py
index 6d73946b..00619d68 100644
--- a/xarray/backends/api.py
+++ b/xarray/backends/api.py
@@ -865,6 +865,8 @@ def open_mfdataset(
             )
         else:
             paths = sorted(glob(_normalize_path(paths)))
+    elif isinstance(paths, Path):
+        paths = [str(paths)]
     else:
         paths = [str(p) if isinstance(p, Path) else p for p in paths]

to emulate the list comprehension we already have.

Would you be interested in submitting this little fix as a PR? Or would you rather I did it?

@TomNicholas
Copy link
Member

In fact in light of #5879, it should probably be

+    elif isinstance(paths, os.PathLike):
+        paths = [str(paths)]

instead.

pmav99 added a commit to pmav99/xarray that referenced this issue Oct 22, 2021
pmav99 added a commit to pmav99/xarray that referenced this issue Oct 22, 2021
TomNicholas pushed a commit that referenced this issue Oct 26, 2021
* Handle single `PathLike` objects in `open_mfdataset()`

Fixes #5881

* Update doc/whats-new.rst

Co-authored-by: Illviljan <14371165+Illviljan@users.noreply.github.com>
Co-authored-by: Deepak Cherian <dcherian@users.noreply.github.com>
snowman2 pushed a commit to snowman2/xarray that referenced this issue Feb 9, 2022
* Handle single `PathLike` objects in `open_mfdataset()`

Fixes pydata#5881

* Update doc/whats-new.rst

Co-authored-by: Illviljan <14371165+Illviljan@users.noreply.github.com>
Co-authored-by: Deepak Cherian <dcherian@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants