Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overhaul versions.py to handle generic named versions #100

Merged
merged 42 commits into from
Nov 20, 2024

Conversation

timkimadobe
Copy link
Contributor

@timkimadobe timkimadobe commented Nov 14, 2024

Questions for reviewers

  1. Should this bump the versions of iOS and Android workflows to x.0.1? Or since no production repo workflows have adopted this tag yet (and hence created Actions caches etc), should we move the existing x.0.0 tags to the state after this PR is merged?
  2. What tag should be used to version the versions.yml file? It is shared across both iOS and Android workflows. Currently it uses gha-ios-5.0.0

Description

This PR overhauls the versions.py logic to be able to handle more dependency version cases (specifically updating GitHub Action workflow yaml files).

Basically, the previous version update logical hierarchy was dependent on the top level version's file paths + pattern types:

  • File (path)
    • Version (unnamed)
      • Dependencies

With the changes in this PR, dependencies are also able to specify their own completely custom file paths + pattern types:

  • File (path
    • Version (unnamed)
    • Dependencies

Now, the -p --paths flag acts as the default file paths for both the main version and dependencies, and each dependency can specify their own override paths + pattern types that override the default ones.

File changes

versions.py

Instead of process_file_version handling both final pattern construction and applying the patterns to the file, it is now only responsible for the latter.

  • Final pattern construction has been moved to:
    • parse_paths -> generate_versioned_patterns and
    • parse_dependencies -> generate_dependency_patterns
    • Rename motivation for both methods was that it no longer just parses the input string

General logic updates

  • Behavior change: empty string paths are filtered out
    • Motivation: empty string paths cannot refer to anything other than the current directory which cannot also be a file with a version. Instead of throwing an error, it is filtered out for ease of calling the script from a workflow file which can pass an empty string instead of more complicated logic of passing or not passing flags.
  • Regex pattern substitution has been improved by regex escaping strings before substituting
    • This is especially important for the caller workflow provided action names in the dependencies input string
  • RegexTemplate and RegexPattern classes have been enhanced by adding a new version_pattern property which allows specifying an overriding version pattern to use instead of the default semantic version.
  • Strict semantic version requirement for both version-only and named versions has been dropped (to support yaml style versions)

versions.yml

Updates to accommodate Actions-based workflow file changes; a PAT is required to provide the required workflows permission:
https://github.com/orgs/community/discussions/35410#discussioncomment-7645702

  • More specifically, the minimum required permissions for the PAT are:
    • contents:write and workflows:write (and metadata, but this is automatically included)

To this end, the workflow has been updated to use the optionally provided input secret WORKFLOW_TOKEN, and used for the initial checkout of the repo (meaning that the scope of permissions provided to the token will be used for git operations going forward in the workflow)

General logic updates

  • Configure GitHub Actions bot user has been combined with the PR creation step
Linked comment archive

This is pretty high up when searching for the error message, and some of the answers are outdated. As of today (November 2023), I think this is correct:
* `GITHUB_TOKEN` can't be given permissions to modify workflow files. `actions: write` is not sufficient. To modify a workflow file from an actions workflow, you must use a personal access token.

* Fine-grained PATs now work with GraphQL, so they can be used. To make changes to a workflow file and commit it, these repo scopes are required: `contents:write`, `workflows:write`. `metadata:read` is set automatically.

* To authenticate the `git` command line client, use the `token` option on `checkout`

Something like this:

jobs:
  changewf:
    name: Change a workflow file
    runs-on: ubuntu-22.04
    permissions:
      contents: write
      pull-requests: write

    steps:
      - name: Check out repository
        uses: actions/checkout@v4.1.1
        with:
          # Fine-grained PAT with contents:write and workflows:write
          # scopes
          token: ${{ secrets.WORKFLOW_TOKEN }}

      - name: Make change to workflow file
          GITHUB_TOKEN: ${{ github.token }}
        run: |
          # Create new branch
          git switch -c "feature"

          # Edit workflow file
          echo "# I am a change!" >> .github/workflows/workflow.yml

          git config --global user.name "github-actions"
          git config --global user.email \
              "41898282+github-actions[bot]@users.noreply.github.com"

          git add .github/workflows/workflow.yml
          git commit --message "Update workflow"

          # This authenticates with WORKFLOW_TOKEN, because it was
          # used with the checkout action
          git push --set-upstream origin "feature"
          
          # This authenticates with GITHUB_TOKEN, using the scopes
          # set in jobs.changewf.permissions
          gh pr create --title "Update workflow" --body ''

ios-update-workflow-versions.yml

Newly created action taking advantage of the changes in the version workflow and script to update the iOS reusable workflow tags.

Removed

Dependency related classes and logic:

  • DEPENDENCY_FILE_TYPES
  • @dataclass class Dependency

Intermediate processing classes and logic:

  • @dataclass class FilePatternGroup

Usage examples

Version validation

name: Update iOS Reusable Workflow Versions

on:
  workflow_dispatch:
    inputs:
      version:
        description: |
          The version to validate for the iOS reusable workflows (ex: 1.2.3). 
          The workflow will automatically construct the final version (ex: gha-ios-1.2.3).
        type: string
        required: true

jobs:
  update-versions:
    uses: ./.github/workflows/versions.yml
    with:
      version: gha-ios-${{ github.event.inputs.version }}
      branch: ${{ github.event.inputs.branch }}
      dependencies: >
        adobe/aepsdk-commons/.github/workflows/versions.yml gha-ios-${{ github.event.inputs.version }}@.github/workflows/ios-release.yml:yml_uses,
        adobe/aepsdk-commons/.github/workflows/ios-validate-code.yml gha-ios-${{ github.event.inputs.version }}@.github/workflows/ios-build-and-test.yml:yml_uses,
        adobe/aepsdk-commons/.github/workflows/ios-custom-command-build-and-test.yml gha-ios-${{ github.event.inputs.version }}@.github/workflows/ios-build-and-test.yml:yml_uses,
        adobe/aepsdk-commons/.github/actions/ios-setup-dependencies-action gha-ios-${{ github.event.inputs.version }}@.github/workflows/ios-build-and-test.yml:yml_uses;.github/workflows/ios-custom-command-build-and-test.yml:yml_uses;.github/workflows/ios-validate-code.yml:yml_uses
    secrets: inherit

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • I have signed the Adobe Open Source CLA.
  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@timkimadobe timkimadobe requested a review from praveek November 14, 2024 03:20
@praveek praveek merged commit 2b0d879 into adobe:main Nov 20, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants