Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EuroSAT: fix SSL certificate issues #8563

Merged
merged 4 commits into from
Aug 6, 2024

Conversation

adamjstewart
Copy link
Contributor

@adamjstewart adamjstewart commented Jul 30, 2024

When downloading the EuroSAT dataset, users encounter the following error message:

> python3
>>> from torchvision.datasets import EuroSAT
>>> EuroSAT('data', download=True)
...
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)>

This leads a lot of users to dangerous StackOverflow suggestions to disable SSL verification entirely.

This PR switches the download URL to a stable mirror of the download on Hugging Face. This dataset was rehosted with permission from the original author of EuroSAT: phelber/EuroSAT#10. The zip file is unmodified and has the same MD5 and SHA256 checksums.

Also added a link to a data loader for the multispectral (MS) version of the dataset for any other remote sensing researchers who need it.

Copy link

pytorch-bot bot commented Jul 30, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/8563

Note: Links to docs will display an error until the docs builds have been completed.

❌ 7 New Failures, 2 Unrelated Failures

As of commit e135137 with merge base c8c496d (image):

NEW FAILURES - The following jobs have failed:

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@@ -53,7 +56,7 @@ def download(self) -> None:

os.makedirs(self._base_folder, exist_ok=True)
download_and_extract_archive(
"https://madm.dfki.de/files/sentinel/EuroSAT.zip",
"https://huggingface.co/datasets/torchgeo/eurosat/resolve/c877bcd43f099cd0196738f714544e355477f3fd/EuroSAT.zip",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could use hf.co or main to make this URL shorter if you want. I don't plan on modifying this file, but the commit hash ensures that even if I do, the checksum still works.

Copy link
Member

@NicolasHug NicolasHug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot @adamjstewart

@adamjstewart
Copy link
Contributor Author

Are the prototype datasets being abandoned?

@NicolasHug
Copy link
Member

Yes, pretty much (https://github.com/pytorch/vision/tree/main/torchvision/prototype/datasets).
The underlying datapipes are being deprecated as well pytorch/data#1196 (comment)

@NicolasHug NicolasHug merged commit 5242d6a into pytorch:main Aug 6, 2024
51 of 60 checks passed
@adamjstewart adamjstewart deleted the datasets/eurosat branch August 6, 2024 12:57
facebook-github-bot pushed a commit that referenced this pull request Aug 7, 2024
Differential Revision: D60903714

fbshipit-source-id: cddb425b64f5b813fc96827bd839f013e2afeea5

Co-authored-by: Nicolas Hug <contact@nicolas-hug.com>
Co-authored-by: Nicolas Hug <nh.nicolas.hug@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants