Add checksums (SHA256) to GitHub releases for all files

[o-l-a-v]

Related command

Add checksums (SHA256) to releases for all files, be it files hosted on GitHub (MSI, source code), and files from azcliprod.blob.core.windows.net (ZIP, MSI etc.).

Is your feature request related to a problem? Please describe.

• Integrity, know that what you downloaded hasn't been corrupted.
• Make it faster for package managers like Scoop to get checksums for new releases without having to download the artifact(s).
  • Which is an even bigger pain with Azure CLI because azcliprod.blob.core.windows.net is so slow.
    • 2021-09-03 [Feature Request] Use CDN to distribute MSI package #19447
    • 2024-04-30 Add ZIP release to GitHub releases too #28856

Describe the solution you'd like

Add a list to GitHub releases with SHA256, like Beaver Notes does:

• https://github.com/Beaver-Notes/Beaver-Notes/releases/tag/3.5.0

Describe alternatives you've considered

None.

Additional context

None.

[yonzhan]

Thank you for opening this issue, we will look into it.

[jiasli]

Hi @o-l-a-v, I noticed https://github.com/Beaver-Notes/Beaver-Notes/releases/tag/3.5.0 puts SHA256s in the release description:

https://github.com/PowerShell/PowerShell/releases/tag/v7.4.4 uses a different format:

How does Scoop extract this information from plaintext?

[o-l-a-v]

I have a PR on Beaver Notes for Scoop that shows the info Scoop needs to fetch the SHA256 checksums:

• https://github.com/ScoopInstaller/Extras/pull/13661/files

Here's the manifest for pwsh, looks like it fetches SHA256 info from release info too:

• https://github.com/ScoopInstaller/Main/blob/master/bucket/pwsh.json

Here's info on the Scoop app manifest:

• https://github.com/ScoopInstaller/Scoop/wiki/App-Manifests

Here's Scoop main manifest repos, see if you can find more examples there:

• main (command line programs only): https://github.com/ScoopInstaller/Main/tree/master/bucket
• extras (can be GUI too): https://github.com/ScoopInstaller/Extras/tree/master/bucket

---

Edit: And I believe the logic for getting new versions is found inside here:

• https://github.com/ScoopInstaller/Scoop/blob/master/bin/checkver.ps1

Which calls Invoke-AutoUpdate from here:

• https://github.com/ScoopInstaller/Scoop/blob/master/lib/autoupdate.ps1

[jiasli]

Thanks @o-l-a-v, but I think you misunderstand my question.

In other words, how is 6461dd3fda39fc65e30c7642f863b9e1dabe32885043094e1d8a79dffcef1dcb extracted from https://github.com/PowerShell/PowerShell/releases/tag/v7.4.4 to https://github.com/ScoopInstaller/Main/blob/75c4fa28f734a4849633a8bd920013bf7d93911a/bucket/pwsh.json#L16 ? Is this a manual process? Or is there a standard format? I would like to know which format you want us to use to provide the SHA256s.

BTW, I noticed Scoop computes the SHA256 of https://azcliprod.blob.core.windows.net/zip/azure-cli-2.63.0-x64.zip by itself: https://github.com/ScoopInstaller/Main/blob/75c4fa28f734a4849633a8bd920013bf7d93911a/bucket/azure-cli.json#L13

[o-l-a-v]

As the feature request states: Scoop can calculate SHA256 by downloading the artifact. But if checksum is already made available and Scoop is told were to look, it does not have to 1) download the artifact and 2) calculate hash.

Scoop has logic to autoupdate manifests. If we add hash ("autoupdate":{"hash": {}}) property to the manifest JSON, Scoop will try to find the checksums depending on the logic. So for Beaver-Notes https://github.com/ScoopInstaller/Extras/pull/13661/files:

{
    ...
    "autoupdate": {
        "architecture": {
            "64bit": {
                "url": "https://github.com/Beaver-Notes/Beaver-Notes/releases/download/$version/Beaver-notes.$version.portable.exe#/dl.7z"
            },
            "arm64": {
                "url": "https://github.com/Beaver-Notes/Beaver-Notes/releases/download/$version/Beaver-notes.$version.portable.arm64.exe#/dl.7z"
            }
        },
        "hash": {
            "url": "https://github.com/Beaver-Notes/Beaver-Notes/releases/tag/$version",
            "regex": "$sha256.*?$basename"
        }
    }
}

And for pwsh https://github.com/ScoopInstaller/Main/blob/master/bucket/pwsh.json:

{
    ...
    "autoupdate": {
        "architecture": {
            "64bit": {
                "url": "https://github.com/PowerShell/PowerShell/releases/download/v$version/PowerShell-$version-win-x64.zip"
            },
            "32bit": {
                "url": "https://github.com/PowerShell/PowerShell/releases/download/v$version/PowerShell-$version-win-x86.zip"
            },
            "arm64": {
                "url": "https://github.com/PowerShell/PowerShell/releases/download/v$version/PowerShell-$version-win-arm64.zip"
            }
        },
        "hash": {
            "url": "$baseurl/hashes.sha256"
        }
    }
}

[o-l-a-v]

There are som find_hash_in_x functions inside here:

• https://github.com/ScoopInstaller/Scoop/blob/master/lib/autoupdate.ps1

You could probably also just add checksum files to the GitHub release instead of adding it as plain text, if you prefer that.

[jiasli]

Thank you for the detailed explanation. find_hash_in_textfile looks a little bit fragile. I think creating a hashes.sha256 with sha256sum --binary * is a more formal and reliable implementation. We will consider this as a feature request.

BTW, we can reuse the code from https://github.com/PowerShell/PowerShell/blob/a1774fd9332925f7635e0832b64b2d158e3a3745/.pipelines/templates/release-githubtasks.yml#L88-L104