chore(ci): introduce provenance and attestation in release

[heitorlessa]

Issue number: #2203

Summary

Introduces SLSA 3+ to increase our supply chain security via verified reproducible builds. It allows customers to publicly verify our builds came from where we claim it came and that it hasn't been tampered (source code + build platform).

• Test release pipeline run https://github.com/heitorlessa/aws-lambda-powertools-test/actions/runs/5509204391/jobs/10041603506
• Test release notes with attestation: https://github.com/heitorlessa/aws-lambda-powertools-test/releases/tag/v2.17.0a8

Changes

Please provide a summary of what's being changed

• Introduce provenance and attestation of our release artifact (SLSA 3+)
• Upgrade poetry-bumpversion plugin due to Pydantic v2 typing_extensions issue
• Upload attestation to current release draft
• Introduce composite action to handle GitHub Release sync + attestation upload
• Refactor code seal with two composite actions to make it more maintainable: seal and seal-restore
• Lower case all job outputs except RELEASE_VERSION
• Update release workflow documentation
• Update CD diagram
• Update release process diagram
• Document how customers can verify our attestation
• Enforce Layer uses release commit + restore

User experience

Please share what the user experience looks like before and after this change

Checklist

If your change doesn't seem to apply, please leave them unchecked.

• Meet tenets criteria
• I have performed a self-review of this change
• Changes have been tested
• Changes are documented
• PR title follows conventional commit semantics

Is this a breaking change?

RFC issue number:

Checklist:

• Migration process documented
• Implement warnings (if it can live side by side)

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

[heitorlessa]

Script used locally to verify release was reproducible and signed:

#!/bin/bash
set -uo pipefail # prevent accessing unset env vars, prevent masking pipeline errors to the next command

#docs
#title              :verify_provenance.sh
#description        :This script will verify a given Powertools for AWS Lambda release build with SLSA Verifier
#author		    :@heitorlessa
#date               :July 1st 2023
#version            :0.1
#usage		    :bash verify_provenance.sh {git_staged_files_or_directories_separated_by_space}
#notes              :Meant to use in GitHub Actions or locally.
#os_version         :Ubuntu 22.04.2 LTS
#todo:              : 1) Receive release version via first input, 2) Update to Prod PyPi after first prod release
#==============================================================================

export readonly ARCHITECTURE=$(uname -m | sed 's/x86_64/amd64/g') # arm64, x86_64 ->amd64
export readonly OS_NAME=$(uname -s | tr '[:upper:]' '[:lower:]')  # darwin, linux
export readonly SLSA_VERIFIER_VERSION="2.3.0"
export readonly SLSA_VERIFIER_CHECKSUM_FILE="SHA256SUM.md"
export readonly SLSA_VERIFIER_BINARY="./slsa-verifier-${OS_NAME}-${ARCHITECTURE}"

export readonly RELEASE_VERSION="2.17.0a8"
export readonly RELEASE_BINARY="aws_lambda_powertools-${RELEASE_VERSION}-py3-none-any.whl"
export readonly ORG="heitorlessa"
export readonly REPO="aws-lambda-powertools-test"
export readonly PROVENANCE_FILE="multiple.intoto.jsonl"

export readonly FILES=("${SLSA_VERIFIER_BINARY}" "${SLSA_VERIFIER_CHECKSUM_FILE}" "${PROVENANCE_FILE}" "${RELEASE_BINARY}")

function debug() {
    TIMESTAMP=$(date -u "+%FT%TZ") # 2023-05-10T07:53:59Z
    echo ""${TIMESTAMP}" DEBUG - $1"
}

function download_slsa_verifier() {
    debug "[*] Downloading SLSA Verifier for - Binary: slsa-verifier-${OS_NAME}-${ARCHITECTURE}"
    curl --location --silent -O "https://github.com/slsa-framework/slsa-verifier/releases/download/v${SLSA_VERIFIER_VERSION}/slsa-verifier-${OS_NAME}-${ARCHITECTURE}"

    debug "[*] Downloading SLSA Verifier checksums"
    curl --location --silent -O "https://raw.githubusercontent.com/slsa-framework/slsa-verifier/main/${SLSA_VERIFIER_CHECKSUM_FILE}"

    debug "[*] Verifying SLSA Verifier binary integrity"
    CURRENT_HASH=$(sha256sum "${SLSA_VERIFIER_BINARY}" | awk '{print $1}')
    if [[ $(grep "${CURRENT_HASH}" "${SLSA_VERIFIER_CHECKSUM_FILE}") ]]; then
        debug "[*] SLSA Verifier binary integrity confirmed"
        chmod +x "${SLSA_VERIFIER_BINARY}"
    else
        debug "[!] Failed integrity check for SLSA Verifier binary: ${SLSA_VERIFIER_BINARY}"
        exit 1
    fi
}

function download_provenance() {
    debug "[*] Downloading attestation for - Release: https://github.com/${ORG}/${REPO}/releases/v${RELEASE_VERSION}"

    curl --location --silent -O "https://github.com/${ORG}/${REPO}/releases/download/v${RELEASE_VERSION}/${PROVENANCE_FILE}"
}

function download_release_artifact() {
    debug "[*] Downloading ${RELEASE_VERSION} release from PyPi"
    # TODO: Once published to Prod, this will become
    # python -m pip download \
    #     --only-binary=:all: \
    #     --progress-bar on \
    #     --no-deps \
    #     --quiet \
    #     aws-lambda-powertools==${RELEASE_VERSION}
    python -m pip download \
        --index-url https://test.pypi.org/simple/ \
        --only-binary=:all: \
        --progress-bar on \
        --no-deps \
        --quiet \
        aws-lambda-powertools==${RELEASE_VERSION}
}

function verify_provenance() {
    debug "[*] Verifying attestation with slsa-verifier"
    "${SLSA_VERIFIER_BINARY}" verify-artifact \
        --provenance-path "${PROVENANCE_FILE}" \
        --source-uri github.com/${ORG}/${REPO} \
        ${RELEASE_BINARY}
}

function cleanup() {
    debug "[*] Cleaning up previously downloaded files"
    rm "${SLSA_VERIFIER_BINARY}"
    rm "${SLSA_VERIFIER_CHECKSUM_FILE}"
    rm "${PROVENANCE_FILE}"
    rm "${RELEASE_BINARY}"
    echo "${FILES[@]}" | xargs -n1 echo "Removed file: "
}

function main() {
    download_slsa_verifier
    download_provenance
    download_release_artifact
    verify_provenance
    cleanup
}

main

# Lessons learned
#
# 1. If source doesn't match provenance
#
# FAILED: SLSA verification failed: source used to generate the binary does not match provenance: expected source 'awslabs/aws-lambda-powertools-python', got 'heitorlessa/aws-lambda-powertools-test'
#
# 2. Avoid building deps during download in Test registry endpoints
#
# FAILED: Could not find a version that satisfies the requirement poetry-core>=1.3.2 (from versions: 1.2.0)
#

[heitorlessa]

Maintainer docs updated: 1/ CD diagram now features Provenance, and 2/ Release process visualize takes Provenance time into account

[heitorlessa]

Ready for review @leandrodamascena - gonna start writing the docs section on how customers can verify our signed builds (and update the script to use our prod GH org+repo)

[sthulb]

Note for later, in your release notes (https://github.com/heitorlessa/aws-lambda-powertools-test/releases/tag/v2.17.0a8) the bot names are weird.

https://github.com/dependabot, https://github.com/dependabot[bot], @github-actions, @github-actions[bot] and @heitorlessa

We should review this later

[sthulb]

Overall, it's looking good. I think the only "extra" work could be to match stanza to labels in the SLSA image.

[heitorlessa]

All addressed

[leandrodamascena]

All addressed

Missing push commits? The name of the repo is still wrong.

https://github.com/aws-powertools/powertools-lambda-python/pull/2746/files/f20f3f9565f958ea2b57bcfeeb4c2df56bcd3f43#diff-20888964593e09b11cc3bf85828e76e19056562e6ab73c138f8da2722f819d5aR33

[heitorlessa]

forgot the last push! great catch!!!

[leandrodamascena]

I have to say that I couldn't cover all these areas, @heitorlessa! This is amazing work and I'm happy to learn from you!!

APPROVED!!