3.12.1.0 — a minor release of Cabal and friends

[ulysses4ever]

The 3.12.1.0 release of Cabal and related packages, including cabal-install, is supposed to follow a (yet to be released) major release of Cabal and Cabal-syntax -- 3.12.0.0. Below is a checklist for the release taken from the wiki page on releasing. /cc @Mikolaj @ffaf1 But first and foremost:

• Release 3.12.0.0. See: Release checklist for GHC 9.10 #9729
  • One optional point that wasn't done there is: extending CI for GHC 9.10; I started this work in #9914 but got a little stuck: I need more time to handle it
• check if any more backports are needed for 3.12.1.0; current plans:
  • update base bounds for desupported ghc 8.2 (backport #9964) #9980
  • Render project configs as clean list in provenance message #9985
  • together with @Kleidukos we will go through the list of PRs merged into master since March, and see if anything is worth of backports to 3.12 but didn't make it (We use this search query.)
    • As discussed on Matrix, backports should be limited to three things: bugfixes, documentation, CI (except bootstrap). Rationale: it's a slow process, and we have a growing demand for a cabal-install that can work with Cabal-3.12.

When ticking out the boxes below, please, try to add links to respective PRs.

A.1 Before the release

• Pre-flight Checks Update licence list #9837 Bump dependencies and cabal-version #10082
• Changelog generation Changelogs for 3.12.1.0 #10124
• Bump version numbers Release 3.12.1.0 version bumps #10130
• Upload release candidates
• Publish artifacts
• Tweak things as needed, merge patches to fix regressions
• Re-upload the candidates on Hackage based on the latest tag, and publish them

A.2 After the release

• prepare a list of contributors for the release, e.g., git log --use-mailmap --pretty=format:%an Cabal-v3.6.1.0...Cabal-v3.8.1.0 | sort -u and don't forget to mention "reviewers, QA testers, devops and others"
• advertise on discourse, haskell-cafe and cabal-devel (how to install using ghcup, e.g. ghcup install cabal --url https://downloads.haskell.org/~cabal/cabal-install-3.8.1.0-rc1/cabal-install-3.8.1.0-x86_64-linux-deb10.tar.xz 3.8.1.0), changelogs, how great our community is and how we love and cherish collaborators, known important bugs and that if anybody is keen to see a particular bug go before 3.8.1.0, the window is now very narrow but we will try to help the interested person squash the bug)
• ask kind souls to cross-post to Discord, cabal-devel mailing list, Twitter, Reddit, LinkedIn, copy the links to #hackage so that others can upvote
• Move the tickets that haven't made it to the release to the next one.
• party 🥳

[ulysses4ever]

I added another checkbox in the very beginning: to release 3.12.0.0 because as Mikolaj suggests on Matrix, a lot is blocked on some of it.

[ulysses4ever]

Possible blocker:

• Regression: local repository file+noindex ignores noindex part #9891

[ulysses4ever]

Added a reference to #9729 by the "Release 3.12.0.0" todo.

[ulysses4ever]

In the issue description above, I added a suggestion to add links to respective PRs when ticking out the checkboxes.

[ulysses4ever]

I added another point that goes before the main list of steps: to check if any more backports are needed for 3.12.1.0.

[ulysses4ever]

Seems like two PR's are worth of backporting:

• update base bounds for desupported ghc 8.2 (backport #9964) #9980 (already a backport PR; I'm working on landing this now)
• Render project configs as clean list in provenance message #9985 (not merged into master yet, needs a review; I plan to do it as soon as I'm done with the first one)

[Mikolaj]

Sounds good. @ffaf1, @Kleidukos?

[phadej]

Is there an ETA when release will happen? The lack of GHC-9.10 compatible cabal-install is (softly) blocking the work on adding support for GHC-9.10

[ulysses4ever]

There's no ETA that I know of but thank you for signaling this concern: this will help us to prioritize things.

[ulysses4ever]

We are going through backports slowly. As discussed on Matrix, backports will have to be limited to three things: bugfixes, documentation, CI (except bootstrap). Rationale: it's a slow process, and we have a growing demand for a cabal-install that can work with Cabal-3.12.

[Bodigrim]

How is it going? There is always going to be one more backport to do, but are they that critical?

[ulysses4ever]

There were some critical CI fixes that we were backporting over last week. We want to have one last pass over the list of actual bugfixes, and then get to actually do the release. We still don't have a tentative date. All who's in need of a newer cabal-install (e.g. because they use Custom setups), should consider using the prerelease in the meantime: https://discourse.haskell.org/t/ann-cabal-3-12-0-0-released/9504#how-to-get-the-cabal-install-pre-release-3

One issue that we potentially want to be resolved, fixed on master and backported is #9917 and it's in very early stages...

[Bodigrim]

One issue that we potentially want to be resolved, fixed on master and backported is #9917 and it's in very early stages...

You might want to change things in future so that it does not happen again in time for GHC 9.12, but #9917 per se would be immediately resolved by releasing cabal-install-3.12.

There were some critical CI fixes that we were backporting over last week.

But CI fixes are not a part of cabal-install package anyway, how do they happen to be critical?

[ulysses4ever]

But CI fixes are not a part of cabal-install package anyway, how do they happen to be critical?

Well we consider green CI a prerequisite for a release.

You might want to change things in future so that it does not happen again in time for GHC 9.12, but #9917 per se would be immediately resolved by releasing cabal-install-3.12.

We want to change this before 3.12 so that when Cabal-3.14 is release we don'tget into the same trouble.

[Bodigrim]

My opinion that it's important to have a release of cabal-install within a few weeks after a respective GHC release, and delaying it even if for a noble cause does more harm than good to the state of ecosystem. A guidance for timeline would be much appreciated.

[Mikolaj]

@ulysses4ever: How about timeboxing #9917? Once all other backports are merged, give it 2 more days? (CC @andreasabel)

[ulysses4ever]

Sounds good to me.

[andreasabel]

I do not have time to work on #9917 now, emitting where and how to emit the warning needs more understanding of the code base. I am paid to work on Agda and I need a working Haskell infrastructure, but digging so deep into Cabal is somewhat out of scope. But also, there seems to be no one who is paid to do bugfixing work on Cabal, is that right? From our meetings, I get the impression that all is volunteer work, and @Mikolaj has a part-time job to orchestrate this work...

[Mikolaj]

@andreasabel: I can't speak for others (AFAIK, some other companies are permitting or encouraging their employees to contribute to cabal) but that's about right regarding my humble person and the key point is we crucially depend on contributions from cabal users. But I did not suggest you fix the problem. I CCd you, because you are an interested party and you did cabal a favour by coordinating this particular issue, so you should be aware of the constraints as they emerge. As soon as there's consensus about the issue, maybe we could advertise it to find an implementer?

[ulysses4ever]

Discussion over at benz0li/ghc-musl#8 (comment) reminded me that it'd be great to update cabal-install so that it builds with GHC 9.10 before the 3.12.1.0 release. So #9914, which is currently stuck with some test failures due to 9.10.

[ulysses4ever]

#9914 (support for GHC 9.10) has landed but its backport to 3.12 currently has some test failures:

• CI: add GHC 9.10 (backport #9914) #10058

[Mikolaj]

I've added some more info in the backport, but now somebody has to do the actual work.

[ulidtko]

Hi @ulysses4ever, kindly reconsider #9917 for 3.12.1.0.

It is wrecking thick havoc onto custom-setup packages with GHC 9.10.

custom-setup
  setup-depends: base,      Cabal  < 3.13

👇 👇 👇

[__2] rejecting: simple-example:setup.Cabal-3.12.0.0/installed-be90
(constraint from maximum version of Cabal used by Setup.hs requires <3.12)

In effect, cabal just unconditionally overrides bounds set by user explicitly in .cabal.

---

Additionally, getting cabal-install 3.12 released would be great.

[Mikolaj]

@ulidtko: hi! I'm afraid the ticket you linked is an issue, not a (doubly reviewed) PR. As such, it can't be considered by the release manager. On the bright side, we do plan to release cabal-install as part of 3.12.1.0, which should fix your problem for the time being, if I understand anything from that ticket. If you have some spare time, any help with the release tasks, or any other cabal contributions, would be very welcome.

[ulysses4ever]

There are two sticky points that I can see now:

• Does cabal-3.10 impose setup.Cabal < 3.12? #9917 -- an issue that is far from being resolved
• Use Directory.canonicalizePath to resolve --with-compiler #10113 -- a PR that has unresolved discussion.

I propose to pass on both of these for the release and get them for a future point release, hopefully.

One thing that we probably want to get in before proceeding:

• Depend on tar-0.6.3.0 #10120
  • Specifically, the bump to index-state that we use to release so that we build the release with the newer tar and rip the benefits of certain optimizations in that package.

[ulysses4ever]

Changelogs PR: #10124 (second step in the release process, see the whole list of steps in the OP).

[geekosaur]

Does anything remain to be done on the hackage-security side? Its GitHub repo is already updated, but I don't think it's on Hackage yet.

[ffaf1]

Does anything remain to be done on the hackage-security side? Its GitHub repo is already updated, but I don't think it's on Hackage yet.

Does bootstrapping work?

[geekosaur]

No, but not for that reason.

hilfy «cabal*version-bumps-3.12.1.0» Z$ make bootstrap-jsons
cabal build --project-file=cabal.bootstrap.project --with-compiler=ghc-9.0.2 --dry-run cabal-install:exe:cabal
Warning: Requested index-state 2024-06-17T00:00:01Z is newer than
'hackage.haskell.org'! Falling back to older state (2024-06-16T21:13:16Z).
Resolving dependencies...
Error: cabal: Could not resolve dependencies:
[__0] trying: Cabal-3.12.1.0 (user goal)
[__1] next goal: Cabal-hooks (user goal)
[__1] rejecting: Cabal-hooks-0.1 (conflict: Cabal==3.12.1.0, Cabal-hooks =>
Cabal>=3.13 && <3.15)
[__1] fail (backjumping, conflict set: Cabal, Cabal-hooks)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: Cabal, Cabal-hooks

make: *** [Makefile:232: bootstrap-json-9.0.2] Error 1
zsh: exit 2     make bootstrap-jsons

Looks like we have a leak from master? Cabal-hooks isn't supposed to be in 3.12, AIUI.

Also, what's with the index-state warning?

Downloading the latest package list from hackage.haskell.org
Package list of hackage.haskell.org has been updated.
The index-state is set to 2024-06-20T22:19:34Z.
To revert to previous state run:
    cabal v2-update 'hackage.haskell.org,2024-06-13T13:22:25Z'

Looks newer than the requested index-state to me…. (ETA: forgot to switch back from cabal 3.10.3.0, and doesn't happen with my 3.12 build)

[geekosaur]

FWIW, git history is a bit difficult because the file was renamed, but it does indeed look like either the rename (which is faked by git, and determined post hoc by heuristics) or backporting #9565 inadvertently brought in Cabal-hooks.

[jasagredo]

Warning: Requested index-state 2024-06-17T00:00:01Z is newer than
'hackage.haskell.org'! Falling back to older state (2024-06-16T21:13:16Z).

This should not show up in latest cabal, it was demoted to a warning (and reworded unless I am misremembering). #8944

[geekosaur]

Bootstrap jobs seem to be passing with updated bootstrap JSONs.

[ulysses4ever]

We're moving to the next step on the list — uploading release candidates.

[Mikolaj]

This should not show up in latest cabal, it was demoted to a warning (and reworded unless I am misremembering). #8944

Is this point cleared up? If so, in which PR?

[ulysses4ever]

@jasagredo what do you mean by "latest cabal"? None of the recent changes around index state have been released, I think. Do you mean master? @geekosaur which version of cabal did you use when you got that message?

[geekosaur]

3.10.3.0, because I'd forgotten to switch back after probing a different issue. When I switched to my usual cabal (dogfooding HEAD of 3.12 branch as of a couple days prior) it went away.

[ulysses4ever]

Good, thank you. We can proceed then.

[geekosaur]

FTR we're currently blocked on #10136 which is breaking CI. (ETA: merged and backported, we can proceed.)

[ulysses4ever]

We're done with the candidates:

• https://hackage.haskell.org/package/Cabal-syntax-3.12.1.0/candidate
• https://hackage.haskell.org/package/Cabal-3.12.1.0/candidate
• https://hackage.haskell.org/package/cabal-install-3.12.1.0/candidate
• https://hackage.haskell.org/package/cabal-install-solver-3.12.1.0/candidate

We're about to move on to the next step -- creating binaries and uploading the artifacts (binaries and tarballs) to the website. (As a part of this step, we'll push the git tags.) But before going there we need to merge one last fix for how we create the release:

• Disable arch-native flag when building releases that we distribute #10142

[ulysses4ever]

Tags are up (as a part of the artifacts step), and they point at the same commit as usual:

• https://github.com/haskell/cabal/tree/Cabal-syntax-v3.12.1.0
• https://github.com/haskell/cabal/tree/Cabal-v3.12.1.0
• https://github.com/haskell/cabal/tree/cabal-install-v3.12.1.0
• https://github.com/haskell/cabal/tree/cabal-install-solver-v3.12.1.0

[ulysses4ever]

The release artifacts have been uploaded to https://downloads.haskell.org/~cabal/cabal-install-latest/ It's not an official announcement yet (see the checklist in the OP) but if all goes well, that should be final artifacts.

[ulysses4ever]

Hackage released is up. Announcement on Discourse: https://discourse.haskell.org/t/cabal-install-3-12-1-0-and-accompanying-libraries-released/

[ulysses4ever]

All checkboxes appear to be ticked. Closing...