Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security] Use SHA-256 checksums from trusted mirrors only #493

Merged
merged 10 commits into from
Mar 7, 2022

Conversation

ddalcino
Copy link
Contributor

@ddalcino ddalcino commented Feb 27, 2022

This change causes aqt to use SHA-256 checksums for all downloads, and only download them from a list of trusted mirrors. By default, this list includes only https://download.qt.io, but the settings.ini file may be used to modify this list. The settings.ini file can also be used to set the number of times to retry the failed download of a checksum.

Partially addresses #488

Todo:

  • Require checksums for Updates.xml files

aqt/helper.py Outdated Show resolved Hide resolved
ddalcino added 5 commits March 6, 2022 17:36
To keep this commit small, `hashurl` was removed from QtPackage, and
`get_hash` constructs the hash url based on the url of the 7z archive
to download. I think that in the future, QtArchive and QtPackage could
be refactored to construct this url more appropriately. However, this
would be a complicated change that doesn't belong in this commit.
This uses a fixture to load the default settings file before each test.
Tests that require a different settings file (ie test_settings) can be
configured to skip this step using this line of code:

    @pytest.mark.load_default_settings(False)

This change is necessary to be able to run these tests in isolation from
each other. The `altlink`, `getUrl` and `downloadBinaryFile` functions
are all dependent on Settings, and you cannot test them without loading
a Settings file first. Without this change, when these tests ran, they
used a Settings object loaded up in some previous test, which could have
loaded some Settings file that we do not want. If we try to run these
tests without loading a Settings file, they will just fail to run
because Settings doesn't exist yet.
This splits QtArchive.archive_url into two new datamembers: archive_path
and base_url. Ultimately, base_url should be removed from QtPackage
entirely.
ddalcino added 4 commits March 6, 2022 17:56
aqt.helper.MyConfigParser.getlist fails to retrieve a fallback list when
the `section` parameter does not exist in the `settings.ini` file.
This ensures that the fallback is used when that key is missing.
`MetadataFactory.fetch_http` must often download HTML pages, not
Updates.xml files. download.qt.io does not store checksums for these
files, so this particular function must be allowed to download these
pages without using a checksum.
@ddalcino ddalcino marked this pull request as ready for review March 7, 2022 02:52
@miurahr
Copy link
Owner

miurahr commented Mar 7, 2022

The change looks good and all the installation/compilation test are passed, and I observed that these tests downloads SHA256 hash files for check.
I'd like to merge it, push release v2.1.0, and ask users for feedback.

@miurahr miurahr merged commit 9010df2 into miurahr:master Mar 7, 2022
@ddalcino ddalcino deleted the retry_checksums branch March 8, 2022 16:13
@ddalcino
Copy link
Contributor Author

ddalcino commented Mar 8, 2022

I can’t help but worry that there’s some security flaw that I’m overlooking here. This sonatype output shows several preexisting flaws that we should probably fix before a full v2.1.0 release (maybe this deserves a separate issue):
https://lift.sonatype.com/results/github.com/miurahr/aqtinstall/01FXH2M8SHR30EFH8N7K0KZX9D

On the other hand, I agree that this change is important enough to let users begin using it as soon as possible. I prefer to release it as v2.1.0rc1.

@miurahr miurahr added this to the version 2.1 milestone Apr 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants