Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.download() fails on TiledDataset #494

Open
Cadair opened this issue Jan 16, 2025 · 2 comments
Open

.download() fails on TiledDataset #494

Cadair opened this issue Jan 16, 2025 · 2 comments
Labels

Comments

@Cadair
Copy link
Member

Cadair commented Jan 16, 2025

import dkist
from dkist.data.sample import VBI_AJQWW

ds = dkist.load_dataset(VBI_AJQWW)
ds.files.download()
INFO: Fetching updated search values for the DKIST client to /home/stuart/.local/share/dkist/api_search_values.json [dkist.net.attrs_values]
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[4], line 1
----> 1 ds.files.download()

File ~/Git/DKIST/dkist/dkist/io/file_manager.py:430, in FileManager.download(self, path, destination_endpoint, progress, wait, label)
    427 from dkist.net.helpers import _orchestrate_transfer_task
    429 if self._ndcube is None:
--> 430     raise ValueError(
    431         "This file manager has no associated Dataset object, so the data can not be downloaded."
    432     )
    434 inv = self._ndcube.meta["inventory"]
    435 path_inv = path_format_inventory(humanize_inventory(inv))

ValueError: This file manager has no associated Dataset object, so the data can not be downloaded.
@Cadair Cadair added the bug label Jan 16, 2025
@Cadair
Copy link
Member Author

Cadair commented Jan 16, 2025

Also assigning .base_path silently fails as you need to iterate over all the sub-datasets.

@Cadair
Copy link
Member Author

Cadair commented Jan 21, 2025

It looks like the FileManager object we dynamically construct for the TiledDataset object needs a reference to the parent cube, but the only thing that's used for is accessing ds.meta["inventory"] so the best fix would be replacing the reference to the cube with an inventory dict (passed by reference).

In the short term adding a fm._ndcube = namedtuple(meta={"inventory": inventory}) here:

return FileManager(
StripedExternalArray(
fileuris=fileuris,
target=1,
dtype=dtype,
shape=shape,
loader=AstropyFITSLoader,
basepath=basepath,
chunksize=chunksize
)
)

before the return would probably work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant