Seeing sporadic 403s when action tries to install plugins #815
Comments
jbrunton
added
kind/bug
|
I note per this comment that the problem seems to be GitHub's rate limiter? If so, I'm not a fan of the proposed solutions, both of which require maintaining separate lists of the plugins used (or even hosting the artifacts independently if I'm reading them right?) This doesn't seem ideal: since I use the Pulumi action in a few different workflows, I would have to maintain the list of plugins in multiple places. I'm guessing from the described behaviour and proposed solutions that Pulumi is trying to download artifacts from GitHub without authenticating, and thus all users of this action are sharing the same rate limits? If so, a better fix would be for Pulumi to do one of the following:
Unless I misunderstood the problem, I think doing something like this would provide a more robust experience for all users of this action. |
|
I agree with using authentication if possible, but AFAIK this would require a change in the pulumi core as it would require to allow headers set in the plugin installation command (please correct me if I am wrong). |
@philprime: I assume so. In fact, I see there's a related open issue in the core Pulumi repo: pulumi/pulumi#11743 I don't think I have the time or Go experience to contribute to that, so I hope the Pulumi maintainers are able to address this. I really like what Pulumi offers but it's becoming too unreliable when running in GitHub to make it a viable product for me atm. |
|
Actually, I see that the However, I suspect this action isn't correctly setting a GITHUB_TOKEN secret. I'm not 100% certain, but:
The fix for this action may, therefore, be as simple as adding a GITHUB_TOKEN secret. I'd be happy to contribute a fix for that, but may not have time until next weekend. EDIT: Actually, the fix may be as simple as doing this when using the action, since one can set any environment variables for an action: - uses: pulumi/actions@v3
env:
PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
...I'm testing this now, but if this works then it would be helpful to update the README about this. EDIT: It did work. Opened a PR to update the readme. |
|
I partially fixed this by cashing the plugins directory, but as @jbrunton says |
kpitzen
added
impact/usability
|
Likely the same root cause as pulumi/pulumi-metabase#3. |
RobbieMcKinstry
added
good-first-issue
|
I'm convinced @JoseAlban 's suggestion of caching plugins is the way to go, in addition to providing guidance around |
|
@RobbieMcKinstry: Thanks. Btw, a note from a related issue: apparently setting a default GitHub token doesn't work with GitHub Enterprise – see the comment from @Moon1706: pulumi/pulumi#11743 (comment) I assume that's because GitHub Enterprise instances don't share any tokens with vanilla GitHub. If so, this might slightly complicate things, since you might need two tokens (one for downloading plugins, and another for commenting on internal PRs). There are also the discussions in other PRs in this repo about using self-hosted plugins (e.g. #812). |
|
Hello, we have also run into this issue with the So far, we have likely solved it by setting the - uses: pulumi/actions@v5
env:
PULUMI_ACCESS_TOKEN: ${{ secrets.PULUMI_ACCESS_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
... Please, are there any plans to set the |
What happened?
I'm using this action to automate deployments in a few projects. In recent days I've started seeing inconsistent failures when pulumi tries to install plugins (especially the
synced-folderplugin, but I'm unsure if it's specific to that).The error looks like this, apparently caused by a 403 when trying to download the plugin:
However, the behaviour is inconsistent, and a re-run of the workflow usually fixes it.
Steps to reproduce
It doesn't happen all the time so it may be difficult to provide a reproducible example – though I'm happy to try to create a simplified app if it's helpful – but it is happening pretty regularly. You can see an example here:
https://github.com/jbrunton/auth0-test/actions/runs/3802920177/jobs/6468891986
You can also see that attempt #2 was successful:
https://github.com/jbrunton/auth0-test/actions/runs/3802920177/jobs/6468979717
That is, the code works as is – it just occasional fails.
Given the nondeterminism, I'm wondering if I might be encountering rate limiting? That's just a guess, however.
Expected Behavior
For the app to be provisioned.
Actual Behavior
The provisioning fails and I have to re-run the workflow.
Output of
pulumi aboutAdditional context
I noticed a couple of open issues related to plugins, but I think the error I'm seeing is different?
EDIT: See also my comment below regarding the open PRs related to this issue.
Contributing
Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).
The text was updated successfully, but these errors were encountered: