API feature to request new project-scoped upload API token

[brainwane]

From @takluyver:

Is there any plan for an API to create upload tokens? E.g. I'd like to have a command-line tool prompt me once for my password & 2FA code, then obtain and store a project-scoped token to use for uploads.

This is a followup to #994.

[rachelcipkins]

I will be working on this issue next!

[rachelcipkins]

@brainwane I have a few questions regarding this issue. Should the command line tool be created in Twine? Should this tool create tokens for entire projects or should it implement the new caveats implemented in #6255 ?

[takluyver]

As my request was a bit vague: what I'm really looking for is a public, documented HTTP interface for creating API tokens. I maintain a command line tool, which handles project uploads, and I'd like to extend it to use API tokens. But I'm not aware of any way to create them programmatically. Once the HTTP interface exists, no doubt other tools will use it too.

Thanks for working on it!

[brainwane]

Sorry for the wait in replying to you @rcipkins -- I think @di or @ewdurbin would be better placed to answer you!

[di]

This may be part of (or blocked on) #284 -- we don't currently have any authenticated APIs (aside from the upload API) but it seems reasonable that this would be part of some future JSON API which could support authentication.

[graingert]

@takluyver can't you generate an unscoped macaroon and then narrow the scope of it with Caveats yourself before storing it?
ah this was already mentioned

[woodruffw]

This is not the same thing as #11272, but the work there may provide a template for an API route here.

[takluyver]

I don't suppose we're any closer to this? I guess #284 is a dependency?

We're meant to be moving in the direction of token based uploads, which absolutely makes sense, but as things stand we have to choose between a user-scoped token - which isn't much of an improvement - or project scoped tokens - which require manual click-click-click interaction for every project. Admittedly I have an unusually large number of projects, but it still seems like a poor UX even if you have just a few.

I suppose I could just mimic the requests from the browser when you log in and create a token, but I assume that's not a good idea for the long term.

[woodruffw]

Yes! We're currently working on #11296, which will complement the existing API token behavior. You'll be able to register an OIDC provider for a PyPI project that doesn't exist yet, and publish from GitHub Actions (and other CI services in the future) without having to manually configure an API token.

[takluyver]

Thanks! But for those of us old fashioned people who like to upload packages to PyPI from our local machine, rather than from CI, will this make it any easier to create a token (or get some equivalent security mechanism) programmatically, i.e. without having to go through the PyPI web interface?

[woodruffw]

Not in its current form, unfortunately -- that work is mostly focused on CI providers.

[smheidrich]

Hacky interim solution: https://smheidrich.gitlab.io/pypi-token-client/

[di]

Blocked on #13409.

[takluyver]

Now that 2FA is required, storing a user-scoped token is less secure than storing a password, because the token is sufficient for uploads on its own.

The workflow I'd really like when publishing a package is that the maintainer is prompted for their PyPI password in the terminal (or it's found from a suitable password manager), then prompted to touch a security key or enter a TOTP code, and then the upload goes ahead. There are usually multiple files to upload (sdist + at least one wheel), and these prompts shouldn't be repeated for each file.

It still seems like the obvious way to achieve this is with an API to create a project-scoped upload token. From my perspective, it would also be best if the server could make the token time-limited (e.g. 1 hour) - I know I can add caveats locally, but if the indefinite token is never sent over the wire, it can never leak. Also, if the server knows it created a time-limited token, it can automatically clear it up after it expires, rather than leaving an ever-growing list of tokens behind.

[graingert]

@takluyver my approach is to never create a token and use the direct from GitHub action authentication method instead

[takluyver]

I'm using that on some projects, but I still kind of like being able to manage a project from my own computer, rather than relying ever more on cloud services. 😮‍💨