Ask users not to raise issues to add random/system certs

[andrewleech]

It seems every second issue raise here is "please add some random or system certificate store to the package".

It might be useful to add a basic issue template to the github project asking users not to post requests like this with a couple of links and/or standard responses.

https://help.github.com/articles/creating-an-issue-template-for-your-repository/

[Lukasa]

Agreed, I’d happily merge an issue template that did that.

[jflemer-ndp]

Or maybe, this module is philosophically misguided? Isn't it reasonable and expected that there are organizational CAs that are not part of the biased Certifi concept of trust? Shouldn't python code be expected to honor a system level CA trust repo? Why is it logical that if I update the system trust, that most mainstream thing (gee, like pip) would respect that? Firefox honors my system CA trust.

I can see Certifi as a great fallback if the system doesn't have a CA trust repo already, or perhaps when running in a "paranoid" mode (i.e. the user does not trust the host administrator).

For others who might come across this, here is some help (on RHEL/CentOS) to make pip and Requests honor the system root CA trust, including any locally administered organizational CA certs:

export REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt

[Lukasa]

Or maybe, this module is philosophically misguided?

This module is a single file containing the Mozilla cert database. It’s not philosophically anything.

Isn't it reasonable and expected that there are organizational CAs that are not part of the biased Certifi concept of trust?

Leaving aside the weasel word biased, yes of course it’s reasonable that there are organisational CAs.

Shouldn't python code be expected to honor a system level CA trust repo?

Yes.

James, you’ve made comments on two issues on two repos and sent a personal email to me in the space of about an hour. This is clearly bothering you, and something seems to be making you think it’s my fault. So let’s be clear.

I don’t think this repo should exist! I don’t think Requests should have to use it. I agree with you.

But it turns out that merely wanting something to be true doesn’t make it so. If you want Requests to trust the system trust store, you can simply open a PR to make it do so. But if you try to actually do that you’ll encounter the issues I mentioned in psf/requests#2966, namely that the APIs you need are not currently exposed into Python, so you have to bind them yourself. This brings you into the realm of C extensions and CFFI, which make installing pip borderline impossible.

This is not an unsolveable problem. The way to solve it is to bring this interface into Python! No-one has done this yet because the last person who attempted was me (PEP 543), I bit off way more than I could chew, and then moved on out of the Python ecosystem for work reasons. But there is nothing preventing a motivated person, such as yourself, from proposing a PEP that adds the interface we need and then writing the code to do it. It just has to get done!

[jflemer-ndp]

Isn't it reasonable and expected that there are organizational CAs that are not part of the biased Certifi concept of trust?

Leaving aside the weasel word biased, yes of course it’s reasonable that there are organisational CAs.

Sorry about using "biased". I am not suggesting negative bias, just bias in that it has it's list and no way to alter. And it's actually not immediately clear that this is just a repackaging of the Mozilla database.

This is clearly bothering you, and something seems to be making you think it’s my fault.

No, I do not think it is your fault (or anyone's).

I don’t think this repo should exist! I don’t think Requests should have to use it. I agree with you.

I think there are some legitimate uses of Certifi, just not as the primary! :-)

But it turns out that merely wanting something to be true doesn’t make it so. If you want Requests to trust the system trust store, you can simply open a PR to make it do so. But if you try to actually do that you’ll encounter the issues I mentioned in psf/requests#2966, namely that the APIs you need are not currently exposed into Python, so you have to bind them yourself. This brings you into the realm of C extensions and CFFI, which make installing pip borderline impossible.

Ah! Thank you for that. The complication of using a C api when bootstrapping PIP install does make sense! That is because it would make the pip install have architecture/platform dependent code versus pure python, right? So this is another great fallback case for Certifi to fill.

This is not an unsolveable problem. The way to solve it is to bring this interface into Python! No-one has done this yet because the last person who attempted was me (PEP 543), I bit off way more than I could chew, and then moved on out of the Python ecosystem for work reasons. But there is nothing preventing a motivated person, such as yourself, from proposing a PEP that adds the interface we need and then writing the code to do it. It just has to get done!

I see, PEP 543 is rather large. I don't think I'll be able to tackle that directly.

I'll propose some README/doc updates to Certifi to help people stumbling into this to get a bit more context.

[mfhepp]

After spending almost a day fixing an SSL problem, I think we should distinguish two cases here:

1. Adding root certificates not included in the Mozilla bundle.
2. Adding missing intermediate certificates.

Understandably, no. 1 is not recommended in the general case.
But no. 2 is a different animal: Unfortunately, it is a common flaw in SSL server configurations to provide an incomplete chain of certificates with missing intermediate certificates. Because this configuration flaw is so common, most but not all modern browsers implement a technique called "AIA Fetching" to fix this on the fly (see e.g. https://www.thesslstore.com/blog/aia-fetching/).

Python's SSL support does not support AIA Fetching and hence depends on a complete chain of certificates from the server; otherwise it throws an exception, like so

SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1124)')))

There is a discussion about whether AIA Fetching should be added to Python, e.g. in this thread: https://bugs.python.org/issue18617#msg293894.

My impression is that this will remain an open issue for the foreseeable future.

Now, whenever a Python user runs into an SSLCertVerificationError, he or she may benefit from guidance on how this can be solved. Is the certifi bundle outdated? How can the missing intermediate certificate be added?

Ideally, certifi would implement AIA Fetching. But at least telling users where to search for a solution would help a lot.

This is why I took the time and documented my experiences for fixing this: https://stackoverflow.com/a/66111417/516699

[sigmavirus24]

Ideally, certifi would implement AIA Fetching

You're asking certifi to do more than provide a pem file and somehow inject itself into the ssl module which seems absolutely out of scope and like way more work than the people involved in this project have time to do.

Further, telling people to overwrite a file on their system has so many potential problems, not the least of which someone writing a script that downloads a malicious bundle or what happens if they don't have long-lived architecture (or even control over where certifi is installed and thus can't replace the file).

[mfhepp]

I am just asking that people who run into an SSL problem caused by server configuration errors that are (1) frequent (I read of 10-30 % of sites), and (2) for which browsers have a work-around in the form of AIA fetching, will at least be directed to the root cause of such problems with missing intermediate certificates. My workaround is likely not the most elegant approach to the problem.

Please consider telling people prominently that:

1. this type of problems is most likely due to missing intermediate certificates and in essence a configuration issue on the origin server;
2. that modern browsers will not complain, because they use AIA Fetching as a mitigation strategy;
3. that Python and Certifi do not use AIA Fetching and are hence raising errors where up-to-date browsers will not;
4. that the only way to solve this is to either fix the server (ideal, yet often impractical) or to add a local copy of the intermediate certificate to the chain of certificates used by requests and urllib.

I spent hours trying to understand this, and the first stop most people take is either the requests documentation or the certifi documentation, because the former states that it uses the certificates from certifi.

So maybe just add a paragraph to the README.md?

People will love you, promised ;-)

[roelandschoukens]

The way I see it, a lot of Python packages rely on a libraries like urllib2, which in turn just use the certificates in this package. So if you have an SSL server with some internal client certificate, Python bindings for various REST and similar APIs will not be able to connect to it. So this package has become, perhaps unintentionally, the arbiter of which root certificates are acceptable in Python land.

[sigmavirus24]

No it isn't. It's a re-packaging of what Mozilla curates. If you were to read this thread, you would understand that. Also, no library makes it impossible to use different trust bundles. So your feeble attempt at an argument holds no value.

[roelandschoukens]

I agree that this package does exactly one job and that it does it well. It is more a problem with packages that use certifi. So the answer to people asking to add more certificates is perhaps to raise an issue somewhere else, for example, requests which has an open bug report for this: psf/requests#5316.