Add kerberos support to authentication pip (when supported)

[cryvate]

Closes #6708

Kerberos has seen widespread adoption in many academic institutions, corporations and other organisations. By:

• vendoring requests_kerberos
• detecting whether (requests_)kerberos is working by doing a try ... except ImportError
• creating a chain of authenticators based on both prompting (opposite no-input flag) and kerberos flag

I have a server running that has kerberos authentication running (with/without possibility of interactive password authentication) and there is pypi.org that has no authentication (for downloads). Consider the matrix of options:

• py/winkerberos installed (or not)
• kerberos credentials present (or not) (user: testing, password: pip10pip10)
• --no-input option passed (or not)

The possible outcomes are as follows:

• If py/winkerberos is not installed, the behaviour is the same as before
• If kerberos credentials are not present, the apparent behaviour is the same as before, and no network requests involving kerberos will be send (as pykerberos fails before that happens). These errors only show with --verbose
• If kerberos credentials are present, and negotiation fails, it will fall-through to another option (prompting or failure). The errors are only shown with verbose.
• If kerberos credentials are present, and negotiation succeeds, the connection will work with kerberos!

[pradyunsg]

Hi @cryvate! Thanks for this PR! :)

I personally haven't used Kerberos so, I genuinely can't review the code in this PR. The rest looks fine to me.

Could you add documentation regarding this new support -- in some appropriate file in the docs/ directory? There's a tox job for generating the docs so, tox -e docs would build the documentation for you.

[cryvate]

Having a reference to --no-input in the docs is helpful, until now I think you could only find it by looking at the source.

[BrownTruck]

Hello!

I am an automated bot and I have noticed that this pull request is not currently able to be merged. If you are able to either merge the master branch into this pull request or rebase this pull request against master then it will eligible for code review and hopefully merging!

[suic86]

Any chance to have this in the next pip version? Thanks

[cryvate]

I don't see why not, I will have look at rebasing/merging and adding the docs

[forsberg]

Let me cheerfully encourage you to do the rebase/merge job! :-) This would be very useful for us in working with our internal nexus-based pypi repo!

[cryvate]

OK, let's do this then. Better late than never!

[cryvate]

This version number should be changed to the release it will actually be in.

[suic86]

Any updates on this? Thanks

[pradyunsg]

None yet @suic86.

[suic86]

@cryvate and @pradyunsg - can I somehow help to move this forward?

[pradyunsg]

I think #5948 is likely a better way to go about this.

[cryvate]

Will #5948 allow support of Kerberos? I do not know the ins-and-outs of the keyring package nor of Kerberos to be honest, but since we need pykerberos etc. here and keyring does not have those as dependencies I am sceptical it will allow support of Kerberos without other evidence. I cannot find any reference to Kerberos or tickets on the keyring package's source or the mailing list. Maybe it will 'just work' (especially for tokens) but I do not see how?

[BrownTruck]

Hello!

I am an automated bot and I have noticed that this pull request is not currently able to be merged. If you are able to either merge the master branch into this pull request or rebase this pull request against master then it will be eligible for code review and hopefully merging!

[suic86]

IMO kerberos support and keyring aren't the same thing.

[dstufft]

Keyring is effectively a way to have pluggable authentication backends for HTTP requests, so it should be possible to write a keyring backend for Kerberos.

[cryvate]

@dstufft I am not sure I agree. Keyring seems to fundamentally serve as a username-and-password middleware that allows writing and retrieving to various backends for keyrings (stores of username and password), however this is not sufficient to support Kerberos:

• Kerberos is uncommonly used with username and password to access services, instead using username and password to retrieve a token that is then used for further authentication. This token could somehow be retrieved by a different backend as you suggest but it will not work the same in terms of HTTP requests as a username and password still requiring changes in pip to make it work.
• Acquiring a service under a Kerberos ticket is a multi-authentication (including service to client authentication), multi-round-trip process that is not implemented in pip, its vendored dependencies or keyring.

Fundamentally, Kerberos is not a username-and-password system (though it uses them at one stage) and thus not a HTTP basic authentication protocol we can support without changing our Auth classes.

I might be misunderstanding, but I suggest reading the outline of the Kerberos protocol on Wikipedia before thinking it is 'just another username-and-password or username-and-token system.'

[suic86]

@dstufft and @pradyunsg: Is there any chance that this is going to be merged soon? Thanks

[gowdy]

This would be very useful for me too.

[ailurid]

I would also find this helpful

[BrownTruck]

Hello!

I am an automated bot and I have noticed that this pull request is not currently able to be merged. If you are able to either merge the master branch into this pull request or rebase this pull request against master then it will be eligible for code review and hopefully merging!

[pradyunsg]

Is there some reason other that pip._vendor.requests_kerberos might result in ImportErrors, outside of them being missing?

If not, please import these unconditionally at the top of the file, since pip does vendoring to ensure that vendored packages are always available. Basically, in not-broken installations of pip, this variable will always be True, which makes it redundant.

[cryvate]

"If py/winkerberos is not installed, the behaviour is the same as before": requests_kerberos on its own will not make kerberos work, one has to have either py/winkerberos installed (which we cannot vendor because they are compiled/platform dependent?). requests_kerberos in that case will fail with an ImportError.

[pradyunsg]

There's a checked in a venv directory here. That's going to have to be removed from the history of this PR, for it to be eligible for merging.

@cryvate please lemme know if you want any sort of help with that. :)

Ah, and the relevant change was commited and pushed in the same minute. :)

[pradyunsg]

I spent some time thinking about this.

I don't think that there's anyone on the pip team, who has expertise in Kerberos. In this situation, it's not reasonable to believe that we would be maintaining this support for Kerberos. I, personally, did not know it existed till this PR came along.

However, there's clearly a user base for this feature. So, here's my proposal:

• Put the kerberos support behind a --i-want-kerberos-support flag (please suggest better names).
  • Due to how pip's configuration handling works, enabling this on a system, is a matter of running pip config set global.i-want-kerberos-support true (or manually modifying the configuration file).
  • I don't want this to affect users who don't care about this in any way, unless the user explicitly opts-in.
• Clearly document that kerberos support is not directly maintained by pip's maintainers and is "community maintained".
  • i.e. kerberos support would be like Python 2 support today.

If other @pypa/pip-committers and @pypa/pip-helpers are OK with this, let's go down this route.

---

I'll also say this: If requests_kerberos gets significantly bigger, pip will de-vendor it and make it a conditional dependency (like we do for PyOpenSSL). At ~400 lines, right now it's not that big so I'm OK with vendoring it. If it's bigger, it's extra bytes for all users of pip, and AFAICT only a fairly small fraction of our users benefit from this support. Thus, requiring a bit of additional work on the end users who do need this support, is a trade-off I'm willing to make. (how big pip gets does affect how much bandwidth PyPI uses, so it is a concern as well here)

Related: In case it becomes a conditional dependency, and a user passes --i-want-kerberos-support, pip would complain that install requests_kerberos and abort.

[cryvate]

(full quote of previous post)

That all sounds reasonable, I will rework the PR for this.

[pradyunsg]

That all sounds reasonable, I will rework the PR for this.

Great! Thanks! ^>^

[BrownTruck]

Hello!

I am an automated bot and I have noticed that this pull request is not currently able to be merged. If you are able to either merge the master branch into this pull request or rebase this pull request against master then it will be eligible for code review and hopefully merging!

[pradyunsg]

Gentle nudge to see if there's interest in moving this forward.

If no one expresses interest to take this forward in the coming week, I'll close this PR when I come around to it.

I'll note that my comment above still stands even if I close this PR.

[chrahunt]

I'll close this given @pradyunsg's comment above. If there's any interest in picking this back up (this applies to anyone interested), please feel free to open a new PR.