Suggestion: Improve tooling for mismatched checksums when bumping

[andig]

During Homebrew/homebrew-core#78666 and other's we've seen a number of checksum/sha issues. As per comment:

how do we determine if the checksum should be updated?

We ask upstream if they retagged their release and/or if the file change is intentional. We err on the side of caution here because it would be very bad if we shipped compromised software.

This process is pretty tedious and it would be nice to automatically open an upstream issue on mismatching checksums. Could be either additional brew command (raise-checksum or issue-checksum) or maybe integrated with brew install --build-from-source (--raise-upstream).

[SMillerDev]

This process is pretty tedious and it would be nice to automatically open an upstream issue on mismatching checksums.

Since not all projects use GitHub issues this is practically impossible to script and even if they do you'd have to script their issue templates too. You'd also still need to follow up with the required information for upstream. I don't think this has as much benefit as you think right now.

[andig]

Since not all projects use GitHub issues this is practically impossible to script

Many do, most I'm working with and I'd suggest to scope it to GH only for the beginning

and even if they do you'd have to script their issue templates too.

AFAIKS issue templates are not mandatory. Projects might accept well-formulated issues by cornerstone projects like homebrew

You'd also still need to follow up with the required information for upstream.

For sure. We still need to ask first though

I don't think this has as much benefit as you think right now.

Imho one might even automate that to the point where maintainers could /sha in Github to have an action open issues against all affected repos. Even if not, it would speed up creating these issues. The go1.16.5 ticket has shown that the manual process is tedious and requires someone to do it, plus that it's a recurring task.

I would still maintain that there's benefit doing so.

[SMillerDev]

I would still maintain that there's benefit doing so.

I think the real benefit would be explaining people that even the git manual says re-tagging is "the insane thing" to do. And convince people to stop doing it.

[andig]

See influxdata/telegraf#9334 (comment). People do re-release and even intentionally. It is unreasonable to expect to educate all of them.

The current release process is highly painful (go1.16.4, go1.16.5) und releases take weeks to make it through homebrew. Imho improving the tooling here (or changing the process of how new releases are picked up alternatively) will make homebrew better.

[andig]

influxdata/telegraf#9334 (comment) actually indicates, that people are re-releasing on purpose which means that the current homebrew-distributed version does not affect the official release (broken as that release process might be).

[iMichka]

that people are re-releasing on purpose

Sounds like they are moving a pre-release tag, not re-release.
Also, look at: influxdata/telegraf#9334 (comment), upstream sort of admits their process is sub-optimal.

In case of pre-releases, upstreams should use tags like version-rc, or b/beta, a/alpha: we will not attempt to ship these.

We only ship stable tags.

[gromgit]

What you're suggesting only streamlines maybe the first 5% of the process (filing a query with upstream). It absolutely does not help with:

1. tracking that query, along with the necessary back-and-forth to confirm the details (I had to do this some months back for a non-GitHub project whose only contact was via email)
2. filing a subsequent PR to fix the checksum issue (which should include the necessary details for the permanent record)
3. tracking that PR to ensure that it and all its dependents build correctly (because what upstream thinks is a harmless change is sometimes...not)
4. rerunning CI on the original PR

It should also be noted that whoever runs brew raise-checksum is also automatically responsible for at least step 1 above (and likely steps 2 and 3 too), since it's their ID on the upstream issue.

[andig]

It should also be noted that whoever runs brew raise-checksum is also automatically responsible for at least step 1 above (and likely steps 2 and 3 too), since it's their ID on the upstream issue.

I didn't know about brew raise-checksum- seems its not documented in brew commands?

[SMillerDev]

brew raise-checksum is your suggested command name

[andig]

First, let me say thanks that we're having this discussion. My intent is making it easier for contributors to fix outstanding brew formula issues.

What you're suggesting only streamlines maybe the first 5% of the process (filing a query with upstream). It absolutely does not help with:

I'm aware of that.

1. tracking that query, along with the necessary back-and-forth to confirm the details (I had to do this some months back for a non-GitHub project whose only contact was via email)

My recent experience is that this is a often straigt-forward and a good part of maintainers does reply. If not we can always deprecate the formula.

2. filing a subsequent PR to fix the checksum issue (which should include the necessary details for the permanent record)

I was gonna suggest to have a command for that, too- seems it already exists? The manual process for doing so ist imho a waste of time.
UPDATE: Sorry, just noticed I had already suggested that with the initial post.

3. tracking that PR to ensure that it and all its dependents build correctly (because what upstream thinks is a harmless change is sometimes...not)

Following up is not a problem for me personally. I'm doing that for all my PRs.

4. rerunning CI on the original PR

Of course.

It should also be noted that whoever runs brew raise-checksum is also automatically responsible for at least step 1 above (and likely steps 2 and 3 too), since it's their ID on the upstream issue.

I maintain my position that this would be a helpful addition.

[carlocab]

You can probably build this tooling for yourself: https://docs.brew.sh/External-Commands

It doesn't even have to be in Ruby if you don't want it to be. If it proves as useful as you believe it will be, then it may be integrated into brew.