Invalid attestations supplied during upload: Attestations are only supported when using Trusted Publishing

[cpburnz]

Describe the bug

I'm following Producing attestations - PyPI Docs, and PyPI won't accept my attestations using Sigstore & Github. According to the guide,

Attestations are uploaded to PyPI as part of the normal file upload flow.

If you're using twine, you can upload any adjacent attestations with their associated files by passing --attestations to twine upload:

Expected behavior

PyPI should accept my attestations using Sigstore & Github which is a Trusted Publisher.

To Reproduce

Clone my project https://github.com/cpburnz/python-sqlparams, and run the following:

python -m build
python -m pypi_attestations sign ./dist/*
# Log into sigstore via Github account.
twine upload -r sqlparams --attestations --skip-existing ./dist/*

Receive the error:

WARNING  Error during upload. Retry with the --verbose option for more details.                                     
ERROR    HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/                                            
         Invalid attestations supplied during upload: Attestations are only supported when > using Trusted Publishing

My Platform

This bug is not platform dependent.

Additional context

N/A

[di]

Hi, thanks for this issue. This is expected: using Trusted Publishing in this case means publishing from GitHub Actions. See https://docs.pypi.org/trusted-publishers/using-a-publisher/ for an example.

[cpburnz]

This is confusing. Why does the documentation show you how to upload attestations manually under the "The manual way" heading if it doesn't work?

[di]

Did you see this warning in those docs?

[cpburnz]

There's no need to be rude. I'm just trying to understand why this doesn't work.

[di]

Not trying to be rude, I really do want to know if you saw it and thought it didn't apply here or missed it entirely. Either we need to tweak the language of the warning to make it more clear, or remove this guide from our user-facing docs if folks are missing the warning.

[cpburnz]

I saw the warning. The "easy way" section links to another project describing how to set it up using Github's CI. The warning doesn't say this is required.

The "manual way" section looks like a far easier method for signing packages. It gives the impression there's a way to sign the package using attestations without having to tie into a convoluted CI. It seems like it should work because when I use the python -m pypi_attestations sign dist/* command, it verifies my identity from Github (a Trusted Publisher), and it generates the attestations. It's only during the upload that it fails despite having an attestation signed by my Github identity.

[woodruffw]

The "manual way" section looks like a far easier method for signing packages. It gives the impression there's a way to sign the package using attestations without having to tie into a convoluted CI. It seems like it should work because when I use the python -m pypi_attestations sign dist/* command, it verifies my identity from Github (a Trusted Publisher), and it generates the attestations. It's only during the upload that it fails despite having an attestation signed by my Github identity.

That's not exactly what's happening here: when you run python -m pypi-attestations sign locally (i.e. not in CI), you're signing with an user identity that's been attested by some IdP (Google, Microsoft, GitHub, etc.). Some of those services are also Trusted Publishing providers, but the two are separate IdPs: Trusted Publishing requires a CI identity (and corresponding machine IdP), not an email/user identity and the normal OAuth flow that goes with it.

In other words: running that command locally produces an attestation that's signed by your GitHub user identity, but not by the machine identity of a CI process. The latter is what Trusted Publishing uses, and is also a requirement within the current acceptance checks for attestations (since we can verify them against the Trusted Publisher(s) for a project).

The "manual way" steps are documented for use in CI; they don't bypass the requirements that the official workflows maintain internally. But I can see how this could be confusing, and I can look at tweaking the docs to make it clearer that the manual steps still fundamentally require Trusted Publishing at the moment.

TL;DR: Trusted Publishers are always machine identities, not user identities, so signing an attestation with your email address (as verified by GitHub) doesn't bypass the Trusted Publishing requirement for attestations.

[woodruffw]

I just checked, and we do indeed mention in the prereqs that Trusted Publishing is required, and mention that it has to be a "supported CI/CD provider."

We also say in the manual section that the manual process won't bypass any Trusted Publishing requirements:

Given those, I think we're pretty well documented here. However, I could look into updating the Trusted Publishing docs themselves to further clarify the human/machine ID and IdP difference.

[cpburnz]

TL;DR: Trusted Publishers are always machine identities, not user identities, so signing an attestation with your email address (as verified by GitHub) doesn't bypass the Trusted Publishing requirement for attestations.

Thanks for the clarification. That's totally not what I expected when I read the term "Trusted Publisher". As a programmer, I expected the IdP such as Github to be the "publisher" because publishers are colloquially organizations, not machines.

In other words: running that command locally produces an attestation that's signed by your GitHub user identity, but not by the machine identity of a CI process. The latter is what Trusted Publishing uses, and is also a requirement within the current acceptance checks for attestations (since we can verify them against the Trusted Publisher(s) for a project).

That leads me to wonder why does pypi_attestations even allow me to sign using my Github user identity in the first place? It opens up a Sigstore page in my browser requesting access to Github. This step is what really made me think attestations could be done outside of a CI process.

I just checked, and we do indeed mention in the prereqs that Trusted Publishing is required, and mention that it has to be a "supported CI/CD provider."

I read the prerequisites section, and the way it's worded sounds like a polite request to read some boilerplate ToS and disclaimer documents 99% of people won't bother with.

[woodruffw]

Thanks for the clarification. That's totally not what I expected when I read the term "Trusted Publisher". As a programmer, I expected the IdP such as Github to be the "publisher" because publishers are colloquially organizations, not machines.

In the context of PyPI, a "publisher" is the entity that uploads projects to the index. That can be a user, or it can be a machine process like a GitHub Actions workflow. The "trusted" in "Trusted Publsher" refers to the fact that the publisher provides a verifiable proof of identity instead of needing to be given a user-configured API token.

That leads me to wonder why does pypi_attestations even allow me to sign using my Github user identity in the first place? It opens up a Sigstore page in my browser requesting access to Github. This step is what really made me think attestations could be done outside of a CI process.

Under the hood, attestations are a wrapper over Sigstore. Sigstore allows signing with all kinds of identities (including email identities), but PyPI intentionally limits the things it accepts to only things that it can verify. At the present time, that effectively means machine identities that come from Trusted Publishing.

I suppose we could cause pypi_attestations to fail when run in a non-CI context, but the long term plan here is to support non-machine identities -- #17001 has some more details on that.

I read the prerequisites section, and the way it's worded sounds like a polite request to read some boilerplate ToS and disclaimer documents 99% of people won't bother with.

This feedback isn't actionable -- disclaimers are simultaneously boring and important. If you could give us something more actionable here (like "I might not have glossed over this if the order was different or the important technical part was more prominent") that would be helpful.