Make upgrade workflows more accessible to third parties

[ringods]

ThePULUMI_BOT_TOKEN requirement for third parties is not very intuitive and the upgrade workflows fail without this available.

1. Add the option to use PULUMI_PROVIDER_AUTOMATION_TOKEN instead of the PULUMI_BOT_TOKEN as a more descriptive alternative.
2. Add fallback if no PAT is available to use the built-in GitHub Actions token. A permissions block is therefore also added to allow it to push commits, create Github issues and pull requests in the repository.

Using the built-in token has the downside that the created PR will not be automatically built because it was created by an App (see the docs for details).

Internally, we should also switch to using PULUMI_PROVIDER_AUTOMATION_TOKEN instead of PULUMI_BOT_TOKEN as it's a better name that describes its purpose better within the ever-more crowded list of org-level secrets.

Contributes to: #1087

[danielrbradley]

Looks good at first glance. Will take this for a test drive before merging.

[iwahbe]

I remember taking a crack at this earlier, and hitting permission issues with publishing. If you can get it to work, then by all means merge it.

[ringods]

@iwahbe the upgrade workflows aren't publishing anything. The publishing in the release workflow is already working with GITHUB_TOKEN and elevated permissions, for instance:

https://github.com/pulumi/pulumi-cloudflare/blob/803fd6dd7d823d98cdf3e318739ed14080641def/.github/workflows/release.yml#L61-L65

[danielrbradley]

I've set up the xyz provider to work for testing upgrade provider.

I've temporarily merged these changes into main to be able to test the upgrade bridge and provider workflows:

• Upgrade bridge: job, PR

Edit: The changes for the provider upgrade were lost because the xyz provider didn't have the workflow enabled when this PR was opened.

It appears that the PR build is not running automatically. I think this is because GitHub actions disables auto-building for changes pushed by GitHub actions to avoid infinite loops. We need to investigate how to enable these builds.

[danielrbradley]

From the docs:

When you use the repository's GITHUB_TOKEN to perform tasks, events triggered by the GITHUB_TOKEN, with the exception of workflow_dispatch and repository_dispatch, will not create a new workflow run. This prevents you from accidentally creating recursive workflow runs. For example, if a workflow run pushes code using the repository's GITHUB_TOKEN, a new workflow will not run even when the repository contains a workflow configured to run when push events occur.

The recommendation on this docs page is to use a PAT when pushing PRs which should trigger a workflow. Perhaps as an in-between we should use the BOT token when creating the pull request, if it's available, but fall back to the built-in token for third parties who haven't set this up yet.

Alternatively, we might be able to use a workflow_dispatch or repository_dispatch to run the acceptance tests directly, and allow the manual sentinel check to be written back to the correct PR, but this will require a little more engineering. One downside of this approach is that it won't give direct access to the job details on the PR if specific parts fail.

[danielrbradley]

The GITHUB_TOKEN does not have scopes to create repository_dispatch or workflow_dispatch events, so we're back to requiring a PAT at some point here.

I think we might just have to gate the step for the creation of the PR on the PAT being set up. We could allow a less pulumi-bot specific secret name to be used instead. If third parties don't want to configure the PAT, then their only other option is to periodically run the upgrade job locally.

[ringods]

@danielrbradley @t0yv0 @iwahbe linking back to Daniel's comment here: #1128 (comment)

I would like to settle on the name of the PAT token so I can test this upfront using a Pulumiverse provider. Since I don't have admin access anymore on Pulumiverse, I have to ask approval to the Pulumiverse GH org admins when using a fine-grained PAT.

Here are my suggestions:

• PULUMI_PROVIDER_AUTO_PR_TOKEN
• PULUMI_PROVIDER_AUTOMATION_TOKEN

I personally don't mind a longer name if it explains better what it is for.

[danielrbradley]

PULUMI_PROVIDER_AUTOMATION_TOKEN is pretty nice

[t0yv0]

Folks I'm summoned to review but I don't have a good handle on what's going on here. NO objection from me if it keeps working for the Pulumi providers builds 🙏

[ringods]

Verification of a succesful run:

• A PAT linked to my GH account is configured as PULUMI_PROVIDER_AUTOMATION_TOKEN
• The Upgrade Provider workflow ran succesfully for the Pulumiverse Cockroach provider.
• It resulted in PR #55
• By using a PAT, also the Run Acceptance Tests workflow was correctly started.

[danielrbradley]

Testing with no new version to upgrade to: https://github.com/pulumi/pulumi-xyz/actions/runs/11975610349/job/33389373072

Currently, something is not working as it's never detecting the new upstream version.

[iwahbe]

I havn't been following very well. @ringods Are you still developing here or is this ready for review?

[ringods]

@iwahbe this is ready for review. This works for me for my Pulumiverse providers, but it should still be tested on a Pulumi-managed provider.

[danielrbradley]

Next test, I've opened a PR (#148) in pulumi-tf-bridge-boilerplate.

I've then started the upgrade-provider job manually from that temporary branch.

There was an existing upgrade PR that was opened last night. This was not modified by the run.

I've closed the existing PR, deleted the branch and retried the upgrade-provider job from the temporary branch.

This second run correctly opened a new PR 🎉

[danielrbradley]

After testing via the TF bridge boilerplate this looks to be functional 👍

[danielrbradley]

@iwahbe I think this is good to go. I've updated the description and title. I'll merge this afternoon, but wanted to give you the opportunity in case you've got any thoughts to add before this ships.

[iwahbe]

If it works (and we've tested it), it looks good to me.