New release

[NorfairKing]

Hi @mpilgrem.
Any chance we could cut a release before the new nixos release is cut this month?

[mpilgrem]

I do not know what triggers a new stack release - I am more janitor than architect. Looking at past releases, perhaps @borsboom (linked to past releases) can advise. It would be good if a new release could take into account #5697 (move to GHC 9.0.2) and #5732 (upgrade MYSY2). Perhaps it is important also first to understand better #5721 (stack making much larger executables than before).

[NorfairKing]

Hey Manny, long time no see!
@borsboom Could we get a release and/or a look at these issues?

[hasufell]

There's a plethora of unattended PRs from contributors, e.g.:

• Pass 'with-gcc:' specified compiler to 'stack setup', fixes #5493 #5609
• Fix how cpp-options are passed to ghci wrt #5532 #5608
• Use capi calling convention for ioctl #5699
• Avoid doing slow IO twice when running stack path #5645
• etc.

[nh2]

There's a plethora of unattended PRs from contributors, e.g.:

I have reviewed those 4, they all look good to me.

Also as per linked issue freebsd/freebsd-ports-haskell#62 above, a new release might also help FreeBSD.

[NorfairKing]

@nh2 Excellent! Could you ping Manny ? :D

[arrowd]

Any movement on this?

[hasufell]

From my side I think #5609 should be merged and then there should be a release in the near future, so that ghcup can make use of the new GHC installation hooks and pre-install stack by default.

[mpilgrem]

#5609 is now merged. Personally, I would like to move Stack to GHC 9.0.2 (current LTS) or GHC 9.2.3 (currently nightly)/GHC9.2.4 (released 28 July 2022, resolver=ghc-9.2.4 now available). @psibi considers that haskell/process#251 is not a block in that regard, but I would be happier if the integration tests passed on macOS. I also think it would be 'neater' to have an extra-dep that is not tied to a particular commit of process to cover the resolution of haskell/process#250 (but to a yet-to-come version of process released on Hackage).

[psibi]

@psibi considers that haskell/process#251 is not a block in that regard, but I would be happier if the integration tests passed on macOS.

Yeah, the integration test passes with the workaround we have for MacOS. But it seems it's currently failing now, but I'm hoping that it should be easy to get it resolved.

I also think it would be 'neater' to have an extra-dep that is not tied to a particular commit of process to cover the resolution of haskell/process#250 (but to a yet-to-come version of process released on Hackage).

Agree, but I think it's okay. :-) But there is one more issue that needs to get resolved in the process package: haskell/process#251 - We would need to remove the workaround we have Stack in now once that get's resolved. But I don't think we should hold release of stack for it.

[mpilgrem]

The fix for haskell/process#250 is now on Hackage, and I hope I've put that through #5763 (waiting on the CI).

[mpilgrem]

I am flagging this discussion to @snoyberg and @borsboom, in case they are willing and able to assist with a new release of Stack.

Thanks to @psibi, Stack has now been moved to GHC 9.2.4. There may be others better placed to release a new version of Stack binaries than me, but I am willing to work through the steps at https://github.com/commercialhaskell/stack/blob/master/doc/maintainers/releases.md. However, I would need some help/guidance for certain steps, as identified below (I am a Windows 11 user, with access to Windows Subsystem for Linux 2. My practical experience of Linux is limited, largely to CI scripts on GitHub).

I understand that all of the steps from https://github.com/commercialhaskell/stack/blob/master/doc/maintainers/releases.md#build-linux-static-binary-distribution-with-nix are no longer relevant, given the GitHub workflows in integration-tests.yml and arm64-release.yml.

[1] What is 'your GPG key', how do you get a GPG key, and how do you sign your key so that it is 'authorised'?

I have no knowledge or experience of GPG (GNU Privacy Guard).

This relates to step:

Push signed Git tag. For final releases the tag should be vX.Y.Z (where X.Y.Z matches the version in package.yaml from the previous step); for release candidates it should be rc/vX.Y.Z.A. e.g.: git tag -u <YOUR-GPG-KEY> -m vX.Y.Z vX.Y.Z && git push origin vX.Y.Z. [RC]

I have also read:

• https://git-scm.com/book/en/v2/Git-Basics-Tagging on annotated Git tagging
• https://git-scm.com/docs/git-tag on the git tag command
• https://github.com/commercialhaskell/stack/blob/master/doc/SIGNING_KEY.md which refers to signing a key to 'authorise' it
• keyserver of signing key seems unavailable #5693, which identifies that https://sks-keyservers.net/ is a dead link in Stack's documentation
• parts of https://gnupg.org/, but it is a large site and I find it difficult to distinguish what is relevant to the step from what is irrelevant

[2] How do you 'activate versions' on https://readthedocs.org/projects/stack/versions/?

I think @borsboom may be the only Maintainer of that Read the Docs account (see https://readthedocs.org/projects/stack/).

This relates to step:

Activate version for new release tag, on readthedocs.org

[3] How do you update the '/stable and /upgrade rewrite rules' at https://raw.githubusercontent.com/commercialhaskell/stack/stable/etc/scripts/get-stack.sh?

This relates to step:

Update get.haskellstack.org /stable and /upgrade rewrite rules with the new version

The URL in the step above is dud, for me. https://get.haskellstack.org redirects to https://raw.githubusercontent.com/commercialhaskell/stack/stable/etc/scripts/get-stack.sh. I do not know to what part of the script reference is being made.

[4] What is 'ArgoCD', and how do you 'sync an application' in it?

This relates to step:

sync the application in ArgoCD

The URL in the step above is dud, for me.

It is not obvious to me what this step is intended to do or if it is still required.

I have read:

• https://argo-cd.readthedocs.io/en/stable/ "Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes." but I have no experience of Kubernetes, Git Ops, or continuous delivery tools.

[5] Are all of the mailing lists haskell-cafe@haskell.org, haskell-stack@googlegroups.com, and commercialhaskell@googlegroups.com still valid, or have any of them been overtaken/replaced by the Haskell Community?

This relates to step:

Announce to haskell-cafe@haskell.org; haskell-stack@googlegroups.com; commercialhaskell@googlegroups.com mailing lists, subject ANN: stack-X.Y.Z (or ANN: first release candidate for stack-X.Y.x), containing the release description from Github. [RC]

[6] What is a Docker image, can you create a relevant one on Windows and, if so, how?

I have very limited knowledge and no practical experience of Docker. @psibi somehow solved the Docker-related aspects of moving Stack to GHC 9.2.4.

This relates to step:

Update fpco/stack-build Docker images with new version

• Under commercialhaskell/stackage/automated/dockerfiles, add lts-X.Y/Dockerfile (where X.Y is the latest stackage LTS version), containing (note where X.Z is the previous LTS version, and X.Y.Z is the newly released stack version)

   FROM $DOCKER_REPO:lts-X.Z
   ARG STACK_VERSION=X.Y.Z
   RUN wget -qO- https://github.com/commercialhaskell/stack/releases/download/v$STACK_VERSION/stack-$STACK_VERSION-linux-x86_64.tar.gz | tar xz --wildcards --strip-components=1 -C /usr/local/bin '*/stack'
  

• Run ./build.sh lts-X.Y and test that the new image has the new version of Stack (e.g. docker run --rm fpco/stack-build:lts stack --version).

• Run ./build.sh --push lts-X.Y && ./build.sh --push --small lts-X.Y to push the new image to the registry.

I have also read:

• The introduction to https://github.com/commercialhaskell/stack/blob/master/doc/docker_integration.md which states 'Windows does not work at all'.
• https://docs.docker.com/get-started/, which indicates that a product 'Docker Desktop' does exist for Windows.

[7] What is required to include builds for ARM64/AArch64 in the release?

This relates to this issue: #5709, which calls for ARM64/AArch64 binary versions of Stack.

I understand:

• ARM64 and AArch64 refer to the same thing
• The GitHub workflow arm64-release builds binary distributions and uploads them as artifacts, and it currently passes.

[mpilgrem]

I am keen for there to be a new binary release of Stack. As there has been no response here to my personal sticking-points above, I am going to propose on the Slack channel that I 'have a go' at making one (even though there is a risk I will not succeed).

With the passage of time, I now understand sticking-point [1] (GPG signing). I think I have to email dev@fpcomplete.com to ask for my GPG key to be 'authorised'. EDIT: which I have now done.

[2] (Read The Docs) versions can be sorted out post-release. I assume 'stable' and 'latest' would 'just work'.

[3] (the Unix-like OS installation script) I do not know that the script would need any changes for new version, if the GitHub workflow handles the uploads.

[4] (ArgoCD) I suspect this may be out of date.

[5] (Announcements) can be sorted out post-release. News would get out, by one channel or another.

[6] (Docker images) This is still a black box and (assuming Docker images are important) a sticking-point, as far as me making a release is concerned.

[7] I understand the last part of this comment to mean that the AArch64/ARM64 builds of Stack do 'work' as intended.

[jsoref]

[2] yes. https://readthedocs.org/projects/stack/versions/ appears to link to https://docs.haskellstack.org/en/latest/README/ / https://docs.haskellstack.org/en/stable/README/ which are tied to https://github.com/commercialhaskell/stack/tree/stable/doc/README.md and https://github.com/commercialhaskell/stack/blob/stable/doc/README.md respectively (and their siblings and niblings). I suspect someone will have to activate the version per https://docs.readthedocs.io/en/stable/tutorial/#versioning-documentation to get the versioned version to appear prominently

[3] How do you update the '/stable and /upgrade rewrite rules' -- I think it's saying that https://get.haskellstack.org/stable/* items need to point to the newly released artifacts. There's an nginx server somewhere fronting https://get.haskellstack.org/stable, but if you visit it redirects to https://github.com/commercialhaskell/stack/releases/download/v2.7.5/stack-2.7.5- so, it's possible that things will magically work. https://github.com/commercialhaskell/stack/actions/runs/1941263181 seems to have been the workflow run that triggered this. I think you want to try generating a release candidate, a la https://github.com/commercialhaskell/stack/actions/runs/799949776, and seeing what happens.

[4] Probably. -- If one could find https://gitlab.com/fpco/operations/kube/fpcomplete-sites-project/-/blob/master/fpcomplete-redirects/get-haskellstack_virtualservice.yaml, it'd make a bit of sense, but I can't find that file. After chasing things a bit, I found https://github.com/fpco/dockerfile-argocd-deploy/blob/master/Dockerfile -- the current release of argocd is https://github.com/argoproj/argo-cd/releases/tag/v2.4.11 -- it seems like asking someone @fpcomplete.com should lead to some insight. Fwiw, there's a https://v7.fpcomplete.com/
-- Fwiw, you're looking for something like https://github.com/commercialhaskell/devops/tree/master/helm/commercialhaskell-com-redirect/templates

[6] You can do things w/ docker using Rancher Desktop. It will let you build images or run them.

[7] As someone who just spent a bit of time w/ stack on m1, yes, stack can work as intended, and would work much better if a new release were to be cut.

Please allow me to add an item to the checklist:
[8] Update actions list haskell/actions@0eee2e8

I'll say hi on Slack...

[borsboom]

Sorry it's taken a while to get back to you... any time not spend doing paid work in the latter half the summer is usually spent trying to squeeze in as much outdoor time as possible before the clouds and rain and darkness return.

As a general point, the release process as documented is overly complex, and I've started trying to simplify it for recent releases. I think be possible to move to making releases directly from the master branch rather than all the extra branching business (although, for readthedocs documentation purposes, keeping the stable branch pointing to the latest release would be best). It'd be nice to skip release candidates too, but with the fairly major updates to dependencies etc. it'd probably be a good idea to make a release candidate (but maybe this can be done from master if we have a moratorium on non-bugfix merges until the final release).

[1] (GPG signing). I think I have to email dev@fpcomplete.com to ask for my GPG key to be 'authorised'. EDIT: which I have now done.

I believe you've already discovered that this isn't necessary since Github Actions takes care of it.

[2] (Read The Docs) versions can be sorted out post-release. I assume 'stable' and 'latest' would 'just work'.

Yes, since those come from the stable and master branches (respectively) they will be automatic. If you send me your readthedocs.org username (you can sign up for one on the site) I can make you a maintainer there.

[3] (the Unix-like OS installation script) I do not know that the script would need any changes for new version, if the GitHub workflow handles the uploads.

Yes, usually this only needs to be updated if we change the officially supported bindist versions

[4] (ArgoCD) I suspect this may be out of date.

I think that's still mostly up to date, but it's internal to fpcomplete. We use that to manage a bit of infrastructure for haskellstack.org. For the moment I can handle updating those. It'd be nice to move this somewhere more public where we could give outside people permissions to update the redirects. Doesn't look like github pages supports HTTP redirects, not sure what the easiest option would be.

[6] (Docker images) This is still a black box and (assuming Docker images are important) a sticking-point, as far as me making a release is concerned.

It should be relatively straightforward, and can be done after the rest of the release. Docker for Windows should work fine for building the images (it just won't work if you try to use stack --docker, but that's not necessary for a release).

[7] I understand the last part of #5709 (comment) to mean that the AArch64/ARM64 builds of Stack do 'work' as intended.

If you can build an arm64 binary, it'll work. The problem is that we're not building and uploading arm64 bindists as part of the Github Actions release process (which uses etc/scripts/release.hs). I think I mentioned in another ticket that it may well be easier to do away with release.hs and just use simple shell commands in the Github Actions workflow to package up the bindists.

[jsoref]

Indeed GitHub pages does not support redirects. For web content, one could cheat w/ html meta http-equiv tags, but that wouldn't work for binaries.

[mpilgrem]

@borsboom, many thanks!

On Read the Docs, I am mpilgrem too (https://readthedocs.org/profiles/mpilgrem/).

On ARM64/AArch64, I've been looking over the long UK weekend at the ARM64 GitHub workflow (see #5847). I had the idea that:

(a) I might be able to update it, so it was in line GHC versions etc being used in the other workflows (which I think I have now done in that pull request, subject to the self-hosted runner seeming to me to have run out of disk space); and

(b) that it could then be integrated as just another job in integration-tests.yml, so that job github-release would 'need' both integration-tests and that ARM64 job.

[borsboom]

I've sent an invite on readthedocs to become a maintainer.

[mpilgrem]

I have (I hope) followed the instructions accurately and published a first release candidate for Stack 2.9.1. See: https://discourse.haskell.org/t/ann-first-release-candidate-for-stack-2-9-1/5015.

[mpilgrem]

I have published, but not yet announced, a final release of Stack 2.9.1. The pause on announcement is that the redirect of https://get.haskellstack.org/stable/linux-x86_64.tar.gz (etc) does not appear to be 'automatic'. @borsboom, you mentioned that you might be able to assist with that step. (I've also messaged @snoyberg on Slack about this step.)

Finally, I have not done the 'Update Docker images' step.

[borsboom]

@mpilgrem I've updated the redirects, so https://get.haskellstack.org/stable/linux-x86_64.tar.gz now redirects to https://github.com/commercialhaskell/stack/releases/download/v2.9.1/stack-2.9.1-linux-x86_64.tar.gz

[mpilgrem]

@borsboom, many thanks! I'll move to the announcement.

[hasufell]

I can't build the release: https://gitlab.haskell.org/maerwald/stack/-/jobs/1176868#L1360

It is using the shipped cabal.project.

Additionally, I don't understand why GHC 9.2.4 is a requirement to build. That's very uncommon on hackage to require such a new base version.

[mpilgrem]

We don't (EDIT: currently; this could change) test systematically that the Stack executable can be built with Cabal (the tool) and the release of fsnotify-0.4.0.0 on 15 September 2022 broke my previous attempts to get Stack to build with Cabal. I've added a first revision to the Cabal file on Hackage for Stack 2.9.1 to add an upper bound on fsnotify. Locally, that seemed to work.

The executable does not aim to have backwards compatability with older versions of GHC - see https://github.com/commercialhaskell/stack/blob/master/CONTRIBUTING.md#backwards-compatability (EDIT: I see that needs to be updated for the detail of Stack 2.9.1 - I'll do that). The previous constraint on base was >= 0.13 (GHC >= 8.8.1). It got bumped when Stack moved to GHC 9.2.4 - perhaps unnecessarily, from a strict point of view. However, to see if the lower bound could be relaxed, somebody would have to try to build against unsupported versions of GHC.

[hasufell]

So I need to cherry pick 92b9622

Wrt base lower bound: 8.10.7 is still widely used in the community, so setting such an extreme lower bound will break cabal install stack for a lot of users. And I know that this is used commonly.

[mpilgrem]

@hasufell, Stack's maintainers' guide to releases refers to 'Hackage-only dependency compatibility patch releases' (https://docs.haskellstack.org/en/stable/maintainers/releases/#use-of-a-fourth-component) and they have been used in the past. Would a stack-2.9.1.1 on Hackage be a solution for what you identify? Work commitments mean that I can't however, pay much attention to Stack before the coming weekend.

[hasufell]

I don't think there is a need for a patch release. Hackage revisions here should be enough. Maybe a 2.9.1 branch where it's applied (not the tag).

[hasufell]

Build failure on alpine for me: https://gitlab.haskell.org/maerwald/stack/-/jobs/1178339#L1482

I have a feeling this will take some time to get into ghcup.

[mpilgrem]

@hasufell, I'm sorry to hear that. Something has been going on with the process package and posix_spawnp on macOS for a while. I wonder if Alpine Linux is somehow similarly affected.

[mpilgrem]

Returning to the lower bound on base, I've reduced it in the master branch to base >= 4.14.3.0 (GHC 8.10.7) after testing with Cabal (the tool) version 3.6.2.0 on Windows 11. If somebody could check it works also on a Unix-like operating system, then I'll make a corresponding revision on Hackage for stack-2.9.1.

[mpilgrem]

As Stack 2.9.1 is released, I am going to close this issue and open a new issue for the post-release matters. I find it easier to keep track of discrete things under their own issue headings.