Move to a weekly release scheme

[jiribenes]

Resolves #533

Was extensively tested in jiribenes/effekt-ci-test

Description

1. [autorelease.yml] Every Monday at 03:00 UTC (= 05:00 Berlin time as of writing this) or on click [workflow-dispatch], the following happens:

   • check if the test pass (with 2 retries) [run-tests]
   • if so, bump the minor version (X.Y.Z => X.{Y + 1}.0), [calculate-version-and-tag]
   • update versions in sbt, npm, and mvn files, [calculate-version-and-tag]
   • and create a git tag [calculate-version-and-tag]

2. [deploy.yml] Whenever a git tag starting with v gets pushed or when autorelease.yml succeeds:

   • figure out the correct version from context [build-jar]
   • assemble the JAR file [build-jar]
   • generate the NPM package [build-jar]
   • generate nice release notes from the diff & PRs via GitHub API [release]
   • create a GitHub release & upload the artefacts [release]
   • publish the NPM package to the NPM registry [publish-npm]

---

Current version format: X.Y.Z

• the CI always bumps Y so the resulting version is X.{Y + 1}.0 :)

Caveats/Wishlist (of the current draft)

• What if we want to release a version manually? Do we instead want to use something like M.YY.WW.N where N is zero by default for automatically released versions and nonzero for manual hot-fixes / patches?
• We lose the nice invariant that every version has a nice git tag like v0.2.2. However, we'll still know which git commit the version is from as GitHub Releases remembers the commit.
  • I think there is a way to tell GitHub to make a version tag automatically, but I don't see how right now. EDIT: The bigger problem is: how do we ensure that a git tag push doesn't trigger this again. If one CI event can react to another, then it might be easier to automatically push a tag, and only then do the release the same way as it was -- this would simplify things a lot!
  • if it's not possible to trigger it implicitly, we should be able to use https://github.blog/changelog/2022-09-08-github-actions-use-github_token-with-workflow_dispatch-and-repository_dispatch/
• It would be nice to automatically push the package to npm as well.
• It would be really useful for jiribenes/effekt-nix to provide a sha256 hash for each released .tgz archive. EDIT: We'll see if we can manage without it.

[jiribenes]

https://github.com/peter-evans/create-pull-request/blob/main/docs/concepts-guidelines.md#triggering-further-workflow-runs
Here are more details on actions triggering other actions automatically:

When you use the repository's [default] GITHUB_TOKEN to perform tasks, events triggered by the GITHUB_TOKEN will not create a new workflow run. This prevents you from accidentally creating recursive workflow runs. For example, if a workflow run pushes code using the repository's GITHUB_TOKEN, a new workflow will not run even when the repository contains a workflow configured to run when push events occur.

There are a few ways to go around this: we could for example add a personal token to be used in CI instead.

[b-studios]

Since this auto-releases to npm (which is great!), can we also change the README to tell people to install effekt via the npm registry?

[jiribenes]

Since this auto-releases to npm (which is great!), can we also change the README to tell people to install effekt via the npm registry?

It's severely untested now (the CI never ran!), but if everything goes well, then we can change the README in a few weeks (once we're confident in the release process). The installer in the VSCode plugin effekt-lang/effekt-vscode#20 is also relying on npm as it's much easier to get the list of versions, check if a version is up to date, etc.

By the way, we'll need to add the NPM secrets to the repository.

[jiribenes]

I'm testing this CI in https://github.com/jiribenes/effekt-ci-test (except for the NPM part for now).
The cronjob should run autorelease every day at 12:00Z UTC to simulate the weekly release schedule a bit quicker.

---

I'll log my observations here:

• The manual way of pushing a tag works perfectly fine as before.
• The release gets created same as before.
• A lot of things need tweaking/changing, see the repo. I'll set the interval to 4x a day and observe it for a bit before backporting the changes; EDIT(2024-08-30 16:20): Just backported a bunch of changes from the test repo.

[jiribenes]

✨ what fun ✨
the CI managed to stumble into the "text file busy" problem twice in a single run!
https://github.com/effekt-lang/effekt/actions/runs/10634552922/job/29482682022

[b-studios]

A controversial question: does the CI that deploys also need to run (all) the tests again?

[jiribenes]

does the CI that deploys also need to run (all) the tests again?

Au contraire, I think it's the only CI that actually really *does* have to pass all the tests.

I understand what you mean, technically not as long as we maintain the invariant that everything on the main branch went through a successful PR. But we have been known to merge PRs without CI finishing or commit directly, both of which resulted in failing test on the main branch.

I'd keep it so that we watch for flaky tests and similar "oops, failing master" escapades, especially since it runs at 3AM and we don't really care how long it takes as long as the new version is ready when we wake up :)

EDIT: The version currently here & in my test repo does not run tests.

[jiribenes]

It's been a month and yet two problems remain:

1. I haven't tested the npm deployment at all
2. sbt test sometimes fails... what do we do then? by that time, we've already created a git tag with a new version, we just never release it

I think the correct way to proceed is:

1. Just merge it, if there's an error during npm deployment, we deploy by hand and then fix the CI.
2. Either disable the tests and agree not to push to master, or maybe doing the tests in CI that bumps a version? But what do we do then if it fails? (then we don't have a release at all)

WDYT?

[b-studios]

I think moving the tests to the ci that creates the version makes sense. Can we automatically rerun the tests a second time? If they still fail it would be good to be notified and abort the release.

[jiribenes]

I've moved the tests to autorelease.yml (= before the tag is created).
Tomorrow, I'll investigate if we can somehow rerun the tests on failure (esp. if the failure is "text file busy").
EDIT: See commit & note below.

[jiribenes]

If we don't want to depend on an external action, we could also add something like the following:

    - name: Run tests (Attempt 1)
      id: test_attempt_1
      continue-on-error: true
      run: sbt clean test

    - name: Run tests (Attempt 2)
      id: test_attempt_2
      if: steps.test_attempt_1.outcome == 'failure'
      continue-on-error: true
      run: sbt clean test

    - name: Run tests (Attempt 3)
      id: test_attempt_3
      if: steps.test_attempt_2.outcome == 'failure'
      run: sbt clean test

[jiribenes]

This PR should now be completely ready for review / discussion. I've modified the PR description to reflect the current state and what the autorelease & deploy CI actually do.