No such file or directory: ref_african_crops_kenya_02_source.tar.gz

[xen0f0n]

Description

I've been trying to download the Kenya Crop Type dataset but I was getting a no such file or directory error for the tar.gz files. I believe there is a discrepancy between the dataset folder structure as it's downloaded from Radiant MLHub and the folder structure assumed in CV4AKenyaCropType. Eventually, I managed to iterate through the DataLoader by compressing the source and labels folders in ref_african_crops_kenya_02_labels.tar.gz and ref_african_crops_kenya_02_source.tar.gz.

This is the folder structure I originally had after downloading the dataset:

- root dir
- - ref_african_crops_kenya_02
- - - ref_african_crops_kenya_02_source
- - - ref_african_crops_kenya_02_labels
- - - catalog.json
- - - err_report
- - - mlhub_stac_assets.db

And this is the one after tweaking it to agree with the code... I think that setting checksum=False doesn't extract the tar.gz files and when set to True it does...

- root dir
- - ref_african_crops_kenya_02_labels
- - ref_african_crops_kenya_02_source
- - ref_african_crops_kenya_02_labels.tar.gz
- - ref_african_crops_kenya_02_source.tar.gz

Steps to reproduce

from torchgeo.datasets import CV4AKenyaCropType
from torch.utils.data import DataLoader

_radiant_earth_api_key = '************'

dataset = CV4AKenyaCropType(
    root='./torchgeo_datasets',
    download=True,
    api_key=_radiant_earth_api_key,
    checksum=True
    )

dataloader = DataLoader(dataset, batch_size=1)
for sample in dataloader:
    image = sample["image"]
    target = sample["mask"]

Version

0.3.1

[adamjstewart]

I'm unable to reproduce this issue. What version of radiant-mlhub are you using? TorchGeo only supports radiant-mlhub 0.2.1–0.4, 0.5+ won't work, see #711. @KennSmithDS has there been any progress on radiant-mlhub 0.5+ support?

[xen0f0n]

@adamjstewart it's the radiant-mlhub version then... I'm using 0.5.3 :/ Cheers!

[adamjstewart]

Glad we solved that mystery. I'm going to close this issue, feel free to track this in #711.