Changelog Fragments

[epage]

Deferring documentation to the end of a task isn't ideal. When its a changelog

• The release manager just wants to get a release done and this is viewed as a chore
• The release manager will likely have less context on what happened

Ideally, the changelog would be written as part of the relevant PRs but how do we that?

If people directly edit CHANGELOG.md, then you end up with merge conflicts.

You could infer it from the commits, but when a commit spans multiple packages, the "breaking change" in it might only apply to one of the packages with the other being just an internal change to update to the new API. The feedback process on commit messages is also not that great.

You could have a tool diff APIs but that leaves out behavior changes, documentation, etc.

What if instead you worked with machine-generated, hand edited changelog fragments per PR that got merged into the CHANGELOG.md?

[epage]

Tools in this space

• cargo-changelog
• unclog
  • doesn't remove files; instead changelog is fully generated
• changesets
  • rust implementation
  • Deals with package relationships
  • Deals with version bump level
• changie
• semverlog
• Changelog Fragments Creator
• scriv
• towncrier
• reno
• changelogging

[epage]

I feel like my ideal solution is some combination of the above

Vision

Idealized workflows

• Create a PR, and a workflow automatically generates a changelog fragment and push it to your branch
  • Contributor and reviewers can decide to tweak this as needed
  • Ideally, the Action would post or update a comment on the PR that showed the rendered version
• Once merged, a release PR is created automatically that collects all of the releases that can be made until someone merges it
  • While other tools might handle the release PR, this tool can support it with
    • Reporting the version bump needed for the changelog fragments
    • Rendering the changelog fragments into a changelog

Design

For now, I'm going to call this "annalist". Unsure if this should be a git subcommand or not. In an ideal world, this would work well across VCS, across VCS hosts, and across build tools but the more we do that, the more inherent complexity and bloat there is.

annalist record

• Discover all project files
  • Discovering a Cargo.toml will stop walking for those files and will run cargo metadata to do discovery
• If multiple project files in a directory, prefer the one with a config section
• Read config
  • For Cargo.toml, read it from package.metadata.annalist, either explicitly or implicitly inheriting from workspace.metadata.annalist
• Modes
  • --auto (default): based on git range, if no entry exists, write an inferred entry
  • --render: based on git range, infer entries and render them to screen
  • --append: based on git range, add inferred entries to the existing file
  • --diff: based on git range, perform a diff between a rendering of an existing entry and the inferred entries
• Option to make a commit of this from a template that includes the fragments introduced in git range
• Random notes
  • Commits are assigned to a package based on having the most specific project file
    • Maybe have a config setting to filter things by cargo package --list
  • Unassigned commits get put into a bucket for consideration

annalist

• Render all .changelog/unreleased

annalist compatibility

• Discovers all .changelog/unreleased
• For each package, list whether the unreleased fragments represent a major or minor incompatibility

annalist publish

• Takes the .changelog/unreleased and moves them to **/.changelog/<current-version>
• May also imply annalist collect
• Option to make a commit of this from a template that includes the formerly-unreleased sections rendered

annalist collect

• Discovers all **/.changelog/*
  • Writes the changelog from all of the files
• Has a --check mode to prevent drift
• Ignores GIT_DIR/.changelog/unreleased
• Option to make a commit of this from a template

File format

[[change]]
# foo dependents are bar, baz
# bar dependents are bob
packages = ["foo", "bar"]
compatibility = "major"
scope = "parser"
type = "feat"
description = '''
Track compatibility across for releases
'''

This was written as-if it was from annalist record --append

• Dependents are listed for the user so they can decide if the change applies to them as well and add them to the packages
• Packages can constrain the set of scopes allowed
• Projects can constrain the set of types allowed
• If a breaking change is assigned to two packages but one is just the update for the other, a user can just copy/paste the block and rewrite them as needed
• Should we have a dependents = [] field which would inform the tool that a version req bump is needed for those dependents?

See https://hachyderm.io/@epage/111189054535306058 for some discussion

[epage]

Something I do with clap's changelog is have Highlights and Migration Guide sections.

Couple of thoughts

• Pull change summaries out as entries and use the summary/body for highlights
• The above but have a highlight = true
• Have a way to flag special entries to go in these sections

[Byron]

My comment only refers to the following:

Idealized workflows

• Create a PR, and a workflow automatically generates a changelog fragment and push it to your branch

  • Contributor and reviewers can decide to tweak this as needed
  • Ideally, the Action would post or update a comment on the PR that showed the rendered version

• Once merged, a release PR is created automatically that collects all of the releases that can be made until someone merges it

  • While other tools might handle the release PR, this tool can support it with

    • Reporting the version bump needed for the changelog fragments
    • Rendering the changelog fragments into a changelog

In general, I'd love to have a cargo release-kind of tool that can also handle changelogs and helps with version bumps of complex workspaces. cargo smart-release is nothing I want, only something I need, and maintenance is thorny already with required features that I keep postponing until the pain is big enough.

With that said, I think an alternative shouldn't be too tied to GitHub PRs and ideally have (at least) a plan for how workflows would look like without GitHub integration, just operating locally. Maybe it's even possible to design such workflow with just git or VCS support and then add goodies in case a branch is also linked to a PR (e.g. to add comments with nicer rendering if a bot is configured).

In any case, let me express my gratitude for your work in this space and my excitement for any kind of improvement this brings, knowing that one day all pieces may come together so cargo smart-release can be retired.

[epage]

I think a lot of this can work with other hosts and maybe VCS; it just might require some more manual work. The most magical / integrted parts would likely live in the Action itself, leveraging lower level tools to compose it.

[thomaseizinger]

I've also toyed with something like this and wrote a PoC at https://github.com/thomaseizinger/semverlog.

[thomaseizinger]

I've thought a lot about release automation for https://github.com/libp2p/rust-libp2p and where I am currently at is that I'd actually like to move away from release PRs and towards having main always in a releasable state. That includes the changelog.

What this doesn't solve is avoiding merge conflicts within changelogs if they are touched by multiple PRs. We squash-merge all PRs so at least for us, it would be ok if there was a bot that automatically amends the changelog and / or fixes the conflicts. Additionally, I am wondering if you couldn't avoid many of these conflicts with e.g. enforced formatting of the changelog. Git at least has the concept of merge drivers so perhaps that could also be used (by a bot) to automatically resolve some of the conflicts?

But even with the possible changelog conflicts, I find that being able to just always release is still better than having to faff around with "prepare release" PRs. Unless you have a branching setup like git-flow, there is always the risk that another PR merges into main after you've branched off your "prepare release" PR. At that point, you now need to coordinate again, which PRs can merge. That doesn't scale well with busier repositories.

Keen to hear about other people's experiences.

[epage]

I've also toyed with something like this and wrote a PoC at https://github.com/thomaseizinger/semverlog.

Looks like this uses md files with yaml frontmatter. Someone was suggesting that on Mastadon, I'm assuming so that the editor workflow is focused on the markdown. My concern is I want it to be easy to split a changelog entry into two which would be easiest if they are in the same file. I'm also hoping most authorship is done by extracting conventional commits.

I've thought a lot about release automation for https://github.com/libp2p/rust-libp2p and where I am currently at is that I'd actually like to move away from release PRs and towards having main always in a releasable state. That includes the changelog.

Something has to bump versions, whether its at the start or end of a release cycle, likely it'll be in the same release tool. With cargo-release, I got a lot of feedback that post-release bumps causes problems with patching dependencies because cargo requires the versions to be the same, so we dropped support for it (it also made other release steps easier later on). Now, patching might not apply everywhere but I'm a strong proponent for making projects consistent to make it easier to scale up development, even if it isn't as ideal.

If you have to bump the version at some point, I feel like generating the changelog and updating other related version bump files isn't too big of a deal.

But even with the possible changelog conflicts, I find that being able to just always release is still better than having to faff around with "prepare release" PRs. Unless you have a branching setup like git-flow, there is always the risk that another PR merges into main after you've branched off your "prepare release" PR. At that point, you now need to coordinate again, which PRs can merge. That doesn't scale well with busier repositories.

I've raised these race condition concerns with release-plz and hopefully we can come up with a solution. My idea is to walk the commit history and do the publishes on the commits the version bumps happened on but that requires using merge commits (which solving this problem requires using merge commits anyways and I feel like squashing is an anti-pattern)

[thomaseizinger]

I've thought a lot about release automation for libp2p/rust-libp2p and where I am currently at is that I'd actually like to move away from release PRs and towards having main always in a releasable state. That includes the changelog.

Something has to bump versions, whether its at the start or end of a release cycle, likely it'll be in the same release tool. With cargo-release, I got a lot of feedback that post-release bumps causes problems with patching dependencies because cargo requires the versions to be the same, so we dropped support for it (it also made other release steps easier later on). Now, patching might not apply everywhere but I'm a strong proponent for making projects consistent to make it easier to scale up development, even if it isn't as ideal.

If you have to bump the version at some point, I feel like generating the changelog and updating other related version bump files isn't too big of a deal.

The idea is that the versions must be bumped together with the change that is making them. See libp2p/rust-libp2p#4620 for example. Together with cargo semver-checks in our CI, that should hopefully ensure that we are making the correct bump!

What I liked about semverlog is that it would play nicer with reverting certain commits. If you record in a changelog entry or some other metadata, that this particular change is a breaking change, it is easier to later revert that again and the inferred version bump will change to non-breaking if that was the only breaking change.

Having said that, we currently use a model where breaking changes as batched up in a milestone and "wait" until we decide to make a release with breaking changes. I think for more established libraries, it is pretty common to do this kind of "planning" of releases. Hence, most of our PRs and also releases are patch-releases and those are the ones I want to automate / make easy. Consequently, I am no longer that concerned about inferred the bump-version from the changelog as the default should be that it is not breaking and you'd typically not interleave breaking and non-breaking changes much.

But even with the possible changelog conflicts, I find that being able to just always release is still better than having to faff around with "prepare release" PRs. Unless you have a branching setup like git-flow, there is always the risk that another PR merges into main after you've branched off your "prepare release" PR. At that point, you now need to coordinate again, which PRs can merge. That doesn't scale well with busier repositories.

I've raised these race condition concerns with release-plz and hopefully we can come up with a solution. My idea is to walk the commit history and do the publishes on the commits the version bumps happened on but that requires using merge commits (which solving this problem requires using merge commits anyways and I feel like squashing is an anti-pattern)

Yeah that is fair. I think using merge commits and clever branching strategies can definitely solve this, see https://github.com/thomaseizinger/github-action-gitflow-release-workflow for example. It is a different kind of complexity in my eyes and I'd like to avoid it if we can. There is something really nice about a linear history on master and each commit being tested, linted and ready to release / deploy.

[weihanglo]

Thanks for the writeup and discssions here people!

When editing changelog for Cargo, I often combine relevant pull requests into one changelog entry. This is a thing need to consider.

The other thing is the ability to post-edit. I guess this can be achieved by editing files under .changelog/ manually?

[epage]

With this, the workflow is that PR authors would write their entry and we'd review them. And yes, we can always change the fragments later.