Skip to content

xarray.open_datatree is taking too long to open datatree in a s3 bucket #9376

Closed
@aladinor

Description

@aladinor

What is your issue?

Hi all,

I was trying to open a datatree stored in a s3 bucket but it is taking too long.

from xarray.backends.api import open_datatree

URL = 'https://js2.jetstream-cloud.org:8001/'
path = f'pythia/radar/erad2024'
fs = s3fs.S3FileSystem(anon=True, client_kwargs=dict(endpoint_url=URL))
file = s3fs.S3Map(f"{path}/zarr_radar/erad_2024.zarr", s3=fs)
dt = open_datatree(file, engine='zarr', consolidated=True)

When digging around, I discovered some parameters/arguments such as mode, consolidated, ..., were not being passed to ZarrStore.open_store function here.

stores = ZarrStore.open_store(filename_or_obj, group=parent)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions