Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_cloud_optimize fails with TypeError: 'int' object is not iterable #1593

Closed
hendrikmakait opened this issue Nov 13, 2024 · 2 comments
Closed

Comments

@hendrikmakait
Copy link
Member

CI Run: https://github.com/coiled/benchmarks/actions/runs/11800653821/job/32872068359#step:9:60 based on git-tip
Cluster:
This still works on the baseline (https://github.com/coiled/benchmarks/actions/runs/11801205071/job/32873962659#step:8:54)

________________________ test_cloud_optimize[execution] ________________________
[gw1] linux -- Python 3.10.15 /home/runner/miniconda3/envs/test/bin/python3.10

scale = 'small', s3 = <s3fs.core.S3FileSystem object at 0x7f30be934a[60](https://github.com/coiled/benchmarks/actions/runs/11800653821/job/32872068359#step:9:61)>
s3_url = 's3://coiled-runtime-ci/test-scratch/test_cloud_optimize-46aa3420d8b24a36a882d633311ba030'
setup_benchmark = <function setup_benchmark.<locals>._ at 0x7f30bcf952d0>
cluster_kwargs = ***'region': 'us-west-2', 'workspace': 'dask-benchmarks'***
scale_kwargs = ***'large': ***'n_workers': 200***, 'medium': ***'n_workers': 100***, 'small': ***'n_workers': 10***

    def test_cloud_optimize(
        scale,
        s3,
        s3_url,
        setup_benchmark,
        cluster_kwargs=***
            "workspace": "dask-benchmarks",
            "region": "us-west-2",
        ***,
        scale_kwargs=***
            "small": ***"n_workers": 10***,
            "medium": ***"n_workers": 100***,
            "large": ***"n_workers": 200***,
        ***,
    ):
        with setup_benchmark(
            **scale_kwargs[scale], **cluster_kwargs
        ) as benchmark:  # noqa: F841
>           benchmark(cloud_optimize, scale, s3fs=s3, storage_url=s3_url)

tests/geospatial/test_cloud_optimize.py:22: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/geospatial/conftest.py:73: in benchmark_execution
    func(*args, **kwargs).compute()
tests/geospatial/workloads/cloud_optimize.py:[72](https://github.com/coiled/benchmarks/actions/runs/11800653821/job/32872068359#step:9:73): in cloud_optimize
    ds = xr.open_mfdataset(
../../../miniconda3/envs/test/lib/python3.10/site-packages/xarray/backends/api.py:1539: in open_mfdataset
    paths = _find_absolute_paths(paths, engine=engine, **kwargs)
../../../miniconda3/envs/test/lib/python3.10/site-packages/xarray/backends/common.py:148: in _find_absolute_paths
    return _normalize_path_list(paths)
../../../miniconda3/envs/test/lib/python3.10/site-packages/xarray/backends/common.py:139: in _normalize_path_list
    return [
../../../miniconda3/envs/test/lib/python3.10/site-packages/xarray/backends/common.py:143: in <listcomp>
    else _normalize_path_list(p)
../../../miniconda3/envs/test/lib/python3.10/site-packages/xarray/backends/common.py:139: in _normalize_path_list
    return [
../../../miniconda3/envs/test/lib/python3.10/site-packages/xarray/backends/common.py:143: in <listcomp>
    else _normalize_path_list(p)
../../../miniconda3/envs/test/lib/python3.10/site-packages/xarray/backends/common.py:139: in _normalize_path_list
    return [
../../../miniconda3/envs/test/lib/python3.10/site-packages/xarray/backends/common.py:143: in <listcomp>
    else _normalize_path_list(p)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

lpaths = 137

    def _normalize_path_list(
        lpaths: NestedSequence[str | os.PathLike],
    ) -> NestedSequence[str]:
>       return [
            (
                _normalize_path(p)
                if isinstance(p, str | os.PathLike)
                else _normalize_path_list(p)
            )
            for p in lpaths
        ]
E       TypeError: 'int' object is not iterable

../../../miniconda3/envs/test/lib/python3.10/site-packages/xarray/backends/common.py:139: TypeError
@phofl
Copy link
Contributor

phofl commented Nov 15, 2024

pydata/xarray#9784

@phofl
Copy link
Contributor

phofl commented Nov 15, 2024

PR opened here: pydata/xarray#9785

@phofl phofl closed this as completed Nov 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants