Downloads should have a way to verify that they are official/authentic

[omajid]

Describe the Problem

I can now download full sources from the VMR:

Unfortunately, if I pass along the tarball to someone, there's no way for them to verify that the download is authentic and I didn't tamper with it.

Describe the Solution

Source-build should publish checksums (sha512, or similar) for release assets, similar to dotnet/installer#15803

As a further enhancement, gpg signatures would be even better, since they would guarantee that the tarball is a blessed and official release without any tampering (even, say GitHub staff, Microsoft staff other than those authorized to handle releases).

Additional Context

This ties into https://learn.microsoft.com/en-us/nuget/concepts/security-best-practices and https://docs.fedoraproject.org/en-US/packaging-guidelines/#_source_file_verification

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[crummel]

[Planning meeting notes]

• Signing the tag and the tarball are two different things (we should probably do both).
• Typical open-source practice tends to be a detached GPG signature on the entire tarball - this is not the same as a catalog signature which would attest to individual files inside the tarball.
• Nothing in our current process uses GPG signatures but it shouldn't be impossible.
• But we don't really have a way to establish trust for our GPG key - no CA infra for GPG keys.

[mmitche]

@omajid Is the debian-recommended process something like what you're looking for?

[omajid]

Yup, that's exactly it.

Though that document refers to auto-generated tarballs. I think we are still trying to get auto-generated tarballs to work with the VMR? The current workaround is to generate the tarballs via release-automation and attach it to the release.

Related, GitHub recently broken the bit-by-bit-reproducible guarantees with the auto-generated tarballs, so that makes custom tarballs more appealing for signing via GPG.

[MichaelSimons]

FYI, We will be working on getting the auto-generated tarballs to work in preview 4.

[MichaelSimons]

@premun, @mmitche - Where are we at on this?

[premun]

We have not started working on adding a signature file onto the release. We only did the work to make the auto-generated tarball work by requiring the user to supply the extra git information (or using the generated release manifest).

We can prioritize this then but my understanding is that we'll need to figure out where to take the key (ESRP?) and where is its public part published so that people can verify the signature.

[omajid]

There was a question offline about what the public key might look like. The guidance we have (which is targeted to a packager packaging up a project for Fedora) is described at https://docs.fedoraproject.org/en-US/packaging-guidelines/#_source_file_verification

The verification method requires an OpenPGP keyring file with one or more public keys from the upstream project. The keyring shall contain all the keys that are trusted to certify the authenticity of the sources, and MUST NOT contain any other keys.

Ideally the upstream project publishes such a keyring as a downloadable file. You shall download that file and do everything you reasonably can to verify that it is authentic. Then you shall add it unmodified to the package SCM, and provide its URL in the spec file so that others can verify it. The URL MUST use HTTPS or a similarly authenticated protocol if at all possible.

(The "package SCM" is the Fedora-side source code repo that contains the package build recipe.)

Any detached signature file (e.g. foo.tar.gz.asc or foo.tar.gz.sig) must be uploaded to the package lookaside cache alongside the source code, while the keyring must be committed directly to the package SCM.

(The package lookaside cache is the place where we upload the upstream source code archive)

If the upstream project does not publish a keyring file (for example if they publish only a fingerprint on their website and refer to a keyserver network for downloading the key), then you may need to create a keyring after you have verified the key. In this case there is no upstream URL to the keyring, so instead you should document how you created the keyring in a comment in the spec file

[mmitche]

Thanks @omajid

[mmitche]

@omajid Can you take a look at https://github.com/dotnet/dotnet/releases/tag/v8.0.0 for the current implementation of the PGP signed releases and let me know what you think?

[omajid]

The current implementation looks good to me:

# Matching files
$ /usr/lib/rpm/redhat/gpgverify --keyring=release-key-2023.asc --signature=dotnet-8.0.0.tar.gz.sig --data=dotnet-v8.0.0.tar.gz 
gpgv: Signature made Tue 14 Nov 2023 02:31:55 PM EST
gpgv:                using RSA key FDBF53C24DB4872E
gpgv: Good signature from "onnxcoredev@microsoft.com"

# Non matching files
$ /usr/lib/rpm/redhat/gpgverify --keyring=release-key-2023.asc --signature=dotnet-8.0.0.tar.gz.sig --data=dotnet-v8.0.0-rc.1.23419.4.tar.gz 
gpgv: Signature made Tue 14 Nov 2023 02:31:55 PM EST
gpgv:                using RSA key FDBF53C24DB4872E
gpgv: BAD signature from "onnxcoredev@microsoft.com"
gpgverify: Signature verification failed.

[MichaelSimons]

Adding clarity to the scope of this PR. This applies to the public GH release as well as the internal handoff to partners. Both versions of the tarballs should be signed. The same reasons why we sign the GH releases applies to the internal handoff artifacts.

[mmitche]

Understood, sounds good. We just got a green on the original PR. I need to go through it one more time though.

[MichaelSimons]

@mmitche, @oleksandr-didyk, @mthalman - Can this be closed now?

[mmitche]

Still need to complete the work for the early handoff tarball.

[MichaelSimons]

@oleksandr-didyk - Is this something you can pickup for the early handoff tarball?

[oleksandr-didyk]

@oleksandr-didyk - Is this something you can pickup for the early handoff tarball?

I could, but would be a bit lost as to what to do exactly. I could chat with @mmitche about it if he does not have something close to finishing and take it over from him

[MichaelSimons]

@oleksandr-didyk, @mmitche - please work together on this. I would like to see this done for the March servicing release which means this should be completed this week.

[mmitche]

https://dev.azure.com/dnceng/internal/_git/dotnet-release/pullrequest/37506