Skip to content

Commit

Permalink
Change the release flow to use release branches (#2253)
Browse files Browse the repository at this point in the history
* What happens if we comment out the runtime crate version from gradle.properties?

* Allow running the release and the CI workflows from an arbitrary commit.

* Does a fake version work?

* Pass `git_ref` from the release workflow.

* It needs to be a valid semver version.

* Sketch new command to upgrade version in gradle.properties

* Command implementation

* Plug the new publisher command into the `release` action.

* Plumb end-to-end

* Fix copyright header.

* Fix lint.

* Temporarily comment out the sanity check.

* Ignore sanity check

* Add a command that prints out the template for CHANGELOG.next.toml

* Add branch check + empty TOML generation.

* Add copyright headers.

* Fix imports.

* Remove sanity check.

* Move script to a file.

* Add a check to validate the tag.

* Remove second build step.

* Move to .github/scripts folder.

* Make the script easier to run locally

* Fail if anything fails.

* Add comment.

* Update .github/scripts/get-or-create-release-branch.sh

Co-authored-by: david-perez <d@vidp.dev>

* Update .github/scripts/get-or-create-release-branch.sh

Co-authored-by: david-perez <d@vidp.dev>

* Update .github/scripts/get-or-create-release-branch.sh

Co-authored-by: david-perez <d@vidp.dev>

* Update .github/workflows/ci.yml

Co-authored-by: david-perez <d@vidp.dev>

* Remove touch.

* Fix indentation and branch name.

* Update .github/workflows/ci.yml

Co-authored-by: david-perez <d@vidp.dev>

* Update .github/workflows/release.yml

Co-authored-by: david-perez <d@vidp.dev>

* Update .github/workflows/release.yml

Co-authored-by: david-perez <d@vidp.dev>

* Explicit flags.

* Use the path that was provided.

* Format

---------

Co-authored-by: david-perez <d@vidp.dev>
  • Loading branch information
LukeMathWalker and david-perez authored Feb 1, 2023
1 parent 7a0bcc2 commit d9c0b2e
Show file tree
Hide file tree
Showing 14 changed files with 337 additions and 81 deletions.
82 changes: 82 additions & 0 deletions .github/scripts/get-or-create-release-branch.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
#!/bin/bash
#
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
#
set -e

# Compute the name of the release branch starting from the version that needs to be released ($SEMANTIC_VERSION).
# If it's the beginning of a new release series, the branch is created and pushed to the remote (chosen according to
# the value $DRY_RUN).
# If it isn't the beginning of a new release series, the script makes sure that the commit that will be tagged is at
# the tip of the (pre-existing) release branch.
#
# The script populates an output file with key-value pairs that are needed in the release CI workflow to carry out
# the next steps in the release flow: the name of the release branch and a boolean flag that is set to 'true' if this
# is the beginning of a new release series.

if [ -z "$SEMANTIC_VERSION" ]; then
echo "'SEMANTIC_VERSION' must be populated."
exit 1
fi

if [ -z "$1" ]; then
echo "You need to specify the path of the file where you want to collect the output"
exit 1
else
output_file="$1"
fi

# Split on the dots
version_array=(${SEMANTIC_VERSION//./ })
major=${version_array[0]}
minor=${version_array[1]}
patch=${version_array[2]}
if [[ "${major}" == "" || "${minor}" == "" || "${patch}" == "" ]]; then
echo "'${SEMANTIC_VERSION}' is not a valid semver tag"
exit 1
fi
if [[ $major == 0 ]]; then
branch_name="smithy-rs-release-${major}.${minor}.x"
if [[ $patch == 0 ]]; then
echo "new_release_series=true" >"${output_file}"
fi
else
branch_name="smithy-rs-release-${major}.x.y"
if [[ $minor == 0 && $patch == 0 ]]; then
echo "new_release_series=true" >"${output_file}"
fi
fi

if [[ "${DRY_RUN}" == "true" ]]; then
branch_name="${branch_name}-preview"
fi
echo "release_branch=${branch_name}" >"${output_file}"

if [[ "${DRY_RUN}" == "true" ]]; then
git push --force origin "HEAD:${branch_name}"
else
commit_sha=$(git rev-parse --short HEAD)
if git ls-remote --exit-code --heads origin "${branch_name}"; then
# The release branch already exists, we need to make sure that our commit is its current tip
branch_head_sha=$(git rev-parse --verify --short "refs/heads/${branch_name}")
if [[ "${branch_head_sha}" != "${commit_sha}" ]]; then
echo "The release branch - ${branch_name} - already exists. ${commit_sha}, the commit you chose when "
echo "launching this release, is not its current HEAD (${branch_head_sha}). This is not allowed: you "
echo "MUST release from the HEAD of the release branch if it already exists."
exit 1
fi
else
# The release branch does not exist.
# We need to make sure that the commit SHA that we are releasing is on `main`.
git fetch origin main
if git branch --contains "${commit_sha}" | grep main; then
# We can then create the release branch and set the current commit as its tip
git checkout -b "${branch_name}"
git push origin "${branch_name}"
else
echo "You must choose a commit from main to create a new release series!"
exit 1
fi
fi
fi
15 changes: 15 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,13 @@ on:
required: false
default: false
type: boolean
git_ref:
description: |
The git reference that all checks should be run against. It can be a branch, a tag or a commit SHA.
If unspecified, it will default to the git reference or SHA that triggered the execution of this workflow.
required: false
type: string
default: ""

env:
rust_version: 1.62.1
Expand All @@ -39,6 +46,7 @@ jobs:
- uses: actions/checkout@v3
with:
path: smithy-rs
ref: ${{ inputs.git_ref }}
# The models from aws-sdk-rust are needed to generate the full SDK for CI
- uses: actions/checkout@v3
with:
Expand Down Expand Up @@ -84,6 +92,7 @@ jobs:
- uses: actions/checkout@v3
with:
path: smithy-rs
ref: ${{ inputs.git_ref }}
- name: Run ${{ matrix.test.action }}
uses: ./smithy-rs/.github/actions/docker-build
with:
Expand Down Expand Up @@ -113,6 +122,7 @@ jobs:
- uses: actions/checkout@v3
with:
path: smithy-rs
ref: ${{ inputs.git_ref }}
- name: Run ${{ matrix.test.action }}
uses: ./smithy-rs/.github/actions/docker-build
with:
Expand All @@ -128,6 +138,8 @@ jobs:
RUSTFLAGS: -D warnings
steps:
- uses: actions/checkout@v3
with:
ref: ${{ inputs.git_ref }}
# Pinned to the commit hash of v2.1.0
- uses: Swatinem/rust-cache@b894d59a8d236e2979b247b80dac8d053ab340dd
with:
Expand Down Expand Up @@ -190,6 +202,8 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v3
with:
ref: ${{ inputs.git_ref }}
# Pinned to the commit hash of v2.1.0
- uses: Swatinem/rust-cache@b894d59a8d236e2979b247b80dac8d053ab340dd
with:
Expand Down Expand Up @@ -258,6 +272,7 @@ jobs:
- uses: actions/checkout@v3
with:
path: smithy-rs
ref: ${{ inputs.git_ref }}
- name: Run ${{ matrix.actions.action }}
uses: ./smithy-rs/.github/actions/docker-build
with:
Expand Down
135 changes: 106 additions & 29 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,48 +6,44 @@

# Allow only one release to run at a time
concurrency:
group: release-smithy-rs
group: release-smithy-rs-${{ inputs.dry_run }}
cancel-in-progress: true

env:
rust_version: 1.62.1

name: Release smithy-rs
run-name: ${{ github.workflow }} - ${{ inputs.dry_run && 'Dry run' || 'Production run' }}
run-name: ${{ github.workflow }} ${{ inputs.semantic_version }} (${{ inputs.commit_sha }}) - ${{ inputs.dry_run && 'Dry run' || 'Production run' }}
on:
workflow_dispatch:
inputs:
commit_sha:
description: The SHA of the git commit that you want to release (e.g. b2318b0)
required: true
type: string
semantic_version:
description: The semver tag that you want to release (e.g. 0.52.1)
required: true
type: string
dry_run:
description: Dry runs will only produce release artifacts, but will not cut a release tag in GitHub nor publish to crates.io
description: Dry runs will only produce release artifacts, but they will not cut a release tag in GitHub nor publish to crates.io
required: true
type: boolean
default: true

jobs:
main-branch-check:
name: Check that workflow is running in main
runs-on: ubuntu-latest
steps:
- name: Main branch check
if: ${{ github.ref_name != 'main' }}
uses: actions/github-script@v6
with:
script: |
core.setFailed("The release workflow can only be ran on main (current branch: ${{ github.ref_name }})")
# If a release is kicked off before an image is built after push to main,
# or if a dry-run release is kicked off against a non-main branch to test
# automation changes, we'll need to build a base image to work against.
# This job will be a no-op if an image was already built on main.
# We'll need to build a base image to work against if:
# - a release was kicked off before the image build step triggered by a push to the release branch/main completed
# - a dry-run release was kicked off against a feature branch to test automation changes
# This job will be a no-op if an image had already been built.
acquire-base-image:
name: Acquire Base Image
needs:
- main-branch-check
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
path: smithy-rs
ref: ${{ inputs.commit_sha }}
fetch-depth: 0
- name: Acquire base image
id: acquire
Expand All @@ -61,17 +57,74 @@ jobs:

release-ci:
name: Prerelease checks
if: inputs.dry_run == false
needs:
- acquire-base-image
uses: ./.github/workflows/ci.yml
with:
run_sdk_examples: false
git_ref: ${{ inputs.commit_sha }}

get-or-create-release-branch:
name: Get or create a release branch
needs:
- release-ci
outputs:
release_branch: ${{ steps.branch-push.outputs.release_branch }}
new_release_series: ${{ steps.branch-push.outputs.new_release_series }}
steps:
- uses: actions/checkout@v3
with:
ref: ${{ inputs.commit_sha }}
token: ${{ secrets.RELEASE_AUTOMATION_BOT_PAT }}
- name: Get or create release branch
id: branch-push
shell: bash
env:
SEMANTIC_VERSION: ${{ inputs.semantic_version }}
DRY_RUN: ${{ inputs.dry_run }}
run: |
set -e
./.github/scripts/get-or-create-release-branch.sh output
cat output > $GITHUB_OUTPUT
upgrade-gradle-properties:
name: Upgrade gradle.properties
if: inputs.dry_run == false
needs:
- get-or-create-release-branch
outputs:
commit_sha: ${{ steps.gradle-push.outputs.commit_sha }}
release_branch: ${{ needs.get-or-create-release-branch.outputs.release_branch }}
steps:
- uses: actions/checkout@v3
with:
ref: ${{ inputs.release_branch }}
token: ${{ secrets.RELEASE_AUTOMATION_BOT_PAT }}
- name: Upgrade gradle.properties
uses: ./smithy-rs/.github/actions/docker-build
with:
action: upgrade-gradle-smithy-rs-release ${{ inputs.semantic_version }}
- name: Push gradle.properties changes
id: gradle-push
shell: bash
env:
SEMANTIC_VERSION: ${{ inputs.semantic_version }}
DRY_RUN: ${{ inputs.dry_run }}
run: |
if [[ git diff-index --quiet HEAD ]]; then
# The file was actually changed, we need to commit and push the changes
git commit gradle.properties --message "Upgrade the smithy-rs runtime crates version to ${SEMANTIC_VERSION}"
echo "Pushing upgraded gradle.properties commit..."
git push origin
fi
echo "commit_sha=$(git rev-parse --short HEAD)" > $GITHUB_OUTPUT
release:
name: Release
needs:
- acquire-base-image
- release-ci
- upgrade-gradle-properties
runs-on: ubuntu-latest
steps:
- name: Install Rust
Expand All @@ -81,6 +134,7 @@ jobs:
- name: Checkout smithy-rs
uses: actions/checkout@v3
with:
ref: ${{ needs.upgrade-gradle-properties.outputs.release_branch }}
path: smithy-rs
token: ${{ secrets.RELEASE_AUTOMATION_BOT_PAT }}
- name: Generate release artifacts
Expand All @@ -93,13 +147,8 @@ jobs:
shell: bash
working-directory: smithy-rs-release/smithy-rs
run: |
if [[ "${{ inputs.dry_run }}" == "true" ]]; then
echo "Pushing a preview of the release to the smithy-rs-release-preview branch"
git push --force origin HEAD:smithy-rs-release-preview
else
echo "Pushing release commits..."
git push origin
fi
echo "Pushing release commits..."
git push origin
- name: Tag release
uses: actions/github-script@v6
with:
Expand Down Expand Up @@ -133,3 +182,31 @@ jobs:
else
publisher publish -y --location .
fi
trim-changelog-next-on-main:
name: Remove released entries from CHANGELOG.next.toml on the main branch
if: inputs.dry_run == false && needs.get-or-create-release-branch.outputs.new_release_series == true
needs:
- get-or-create-release-branch
- release
runs-on: ubuntu-latest
steps:
- name: Checkout smithy-rs
uses: actions/checkout@v3
with:
ref: ${{ inputs.commit_sha }}
token: ${{ secrets.RELEASE_AUTOMATION_BOT_PAT }}
- name: Empty CHANGELOG.next.toml
uses: ./smithy-rs/.github/actions/docker-build
with:
action: generate-new-changelog-next-toml
- name: Push smithy-rs changes
shell: bash
run: |
# This will fail if other commits have been pushed to `main` after `commit_sha`
# In particular, this will ALWAYS fail if you are creating a new release series from
# a commit that is not the current tip of `main`.
# We can build more refined automation to handle this case in the future - until then, it'll require
# a manual PR to edit the current CHANGELOG.next.toml file and remove the released entries.
git commit CHANGELOG.next.toml --message "Remove released entries from \`CHANGELOG.next.toml\`"
git push origin main
2 changes: 1 addition & 1 deletion gradle.properties
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ rust.msrv=1.62.1
org.gradle.jvmargs=-Xmx1024M

# Version number to use for the generated runtime crates
smithy.rs.runtime.crate.version=0.54.1
smithy.rs.runtime.crate.version=0.0.0-smithy-rs-head

kotlin.code.style=official

Expand Down
16 changes: 16 additions & 0 deletions tools/changelogger/src/init.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
/*
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
* SPDX-License-Identifier: Apache-2.0
*/

use crate::render::EXAMPLE_ENTRY;
use clap::Parser;
use std::io::Write;

#[derive(Parser, Debug, Eq, PartialEq)]
pub struct InitArgs {}

pub fn subcommand_init(_args: &InitArgs) -> anyhow::Result<()> {
writeln!(std::io::stdout(), "{}", EXAMPLE_ENTRY)?;
Ok(())
}
1 change: 1 addition & 0 deletions tools/changelogger/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,6 @@
*/

pub mod entry;
pub mod init;
pub mod render;
pub mod split;
Loading

0 comments on commit d9c0b2e

Please sign in to comment.