gh-79846: Make `ssl.create_default_context()` ignore invalid certificates #91740

pukkandan · 2022-04-20T12:46:58Z

Currently, when loading certificates from the Windows certificate store, error in any one certificate causes ssl.create_default_context() to crash. This causes issues in systems that have certificates that are not quite to-spec. A primary culprit for this is "MUPCA Root", which (despite being is technically invalid) is essential for citizens of Serbia

See the conversations under the linked issues for more details

I believe it makes sense for create_default_context to ignore any invalid certificates in the system store. An existing comment in the related code seems to agree with me on this:

cpython/Lib/ssl.py

Lines 772 to 774 in 8497514

    
           # no explicit cafile, capath or cadata but the verify mode is 
        
           # CERT_OPTIONAL or CERT_REQUIRED. Let's try to load default system 
        
           # root CA certificates for the given purpose. This may fail silently.

This issue can be solved by loading each certificate one by one and ignoring any SSLErrors. I had outlined the idea for this patch in the above-mentioned issue, but never recieved any reply on whether this is acceptable. Hopefully, this PR receives better attention

Issue: Function ssl.create_default_context raises exception on Windows 10 when called with ssl.Purpose.SERVER_AUTH) attribute #79846

An error in one certificate should not cause the whole thing to crash Fixes python#79846, python#89475

ghost · 2022-04-20T12:47:01Z

All commit authors signed the Contributor License Agreement.

bedevere-bot · 2022-04-20T12:47:01Z

Every change to Python requires a NEWS entry.

Please, add it using the blurb_it Web app or the blurb command-line tool.

serhiy-storchaka · 2024-02-23T15:34:10Z

The issues was closed and I cannot reproduce the original problem.

ideasman42 · 2024-07-28T10:43:19Z

This issue still seems to be a problem.
While I don't use MS-Windows, multiple Blender users look to have run into this with Blender 4.2, using Python 3.11.
See: https://projects.blender.org/blender/blender/issues/124731#

serhiy-storchaka · 2024-07-29T19:14:56Z

Lib/ssl.py

-                    if trust is True or purpose.oid in trust:
-                        certs.extend(cert)
+                try:
+                    self.load_verify_locations(cadata=cert)


This loads the certificate unconditionally, unlike the current code.

Workaround: `[ASN1] nested asn1 error` error when making HTTPS connections on systems with certificates that OpenSSL cannot parse are installed. This is a general issue with Python, resolve by applying a proposed fix [0] to the extensions Python process at run-time. (this doesn't impact Blender's Python run-time). The down side is HTTPS connections will only work for extensions on systems with this problem so this needs to be resolved by Python long term. While any changes to Python's SSL checks is worth avoiding, this simply skips SSL certificates in the windows store that OpenSSL can't parse instead of failing all SSL connections. See related issues: - python/cpython#79846 - openssl/openssl#25023 [0]: python/cpython#91740 Ref !124943.

zooba · 2024-08-06T17:01:39Z

Is calling load_verify_locations multiple times okay? It doesn't replace the old loaded certs in any OpenSSL versions, right?

Workaround: `[ASN1] nested asn1 error` error when making HTTPS connections on systems with certificates that OpenSSL cannot parse are installed. This is a general issue with Python, resolve by applying a proposed fix [0] to the extensions Python process at run-time. (this doesn't impact Blender's Python run-time). The down side is HTTPS connections will only work for extensions on systems with this problem so this needs to be resolved by Python long term. While any changes to Python's SSL checks is worth avoiding, this simply skips SSL certificates in the windows store that OpenSSL can't parse instead of failing all SSL connections. See related issues: - python/cpython#79846 - openssl/openssl#25023 [0]: python/cpython#91740 Ref !124943.

serhiy-storchaka · 2024-08-06T17:31:38Z

There are tests that call load_verify_locations() multiple times. For example, test_load_verify_cadata checks that calling load_verify_locations() multiple times with different certificates increases some counter, but calling it with already loaded certificate does not. Also, calling it with concatenated certificates has the same effect on this counter as calling it separately for every certificate.

I have no plausible scenario in which it would replace the old loaded certificates, but can detect duplicates in statistics. Looking at the code of cert_store_stats(), it requests statistic from the library, not from Python-level cache.

zooba · 2024-08-06T18:08:30Z

If it's tested, then that's fine. This change LGTM

miss-islington-app · 2024-08-07T08:31:14Z

Thanks @pukkandan for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

miss-islington-app · 2024-08-07T08:31:14Z

Thanks @pukkandan for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

…ificates (pythonGH-91740) An error in one certificate should not cause the whole thing to fail. (cherry picked from commit 9e551f9) Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

bedevere-app · 2024-08-07T08:31:25Z

GH-122768 is a backport of this pull request to the 3.13 branch.

bedevere-app · 2024-08-07T08:31:29Z

GH-122769 is a backport of this pull request to the 3.12 branch.

…tificates (GH-91740) (#122769) An error in one certificate should not cause the whole thing to fail. (cherry picked from commit 9e551f9) Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

…ificates (pythonGH-91740) An error in one certificate should not cause the whole thing to fail. Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

…tificates (GH-91740) (#122768) gh-79846: Make ssl.create_default_context() ignore invalid certificates (GH-91740) An error in one certificate should not cause the whole thing to fail. (cherry picked from commit 9e551f9) Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

[ssl] Load CA certificates one by one

d2770a2

An error in one certificate should not cause the whole thing to crash Fixes python#79846, python#89475

bedevere-bot added the awaiting review label Apr 20, 2022

pukkandan changed the title ~~gh-79846: Load CA certificates one by one in ssl.create_default_context()~~ gh-79846: Make ssl.create_default_context() ignore invalid certificates Apr 20, 2022

Add NEWS entry

3411826

pukkandan mentioned this pull request Apr 20, 2022

SSLError when windows cert store contains bad certificates yt-dlp/yt-dlp#1060

Closed

Merge branch 'main' into bugs/79846

1275d5d

serhiy-storchaka requested a review from tiran February 21, 2024 21:45

Merge branch 'main' into bugs/79846

45dc30f

bedevere-app bot mentioned this pull request Feb 21, 2024

Function ssl.create_default_context raises exception on Windows 10 when called with ssl.Purpose.SERVER_AUTH) attribute #79846

Closed

serhiy-storchaka closed this Feb 23, 2024

levicki mentioned this pull request Jul 28, 2024

SSLError(58, '[ASN1] nested asn1 error (_ssl.c:4174)') and SSLError(0, 'not enough data: cadata does not contain a certificate (_ssl.c:4159)') #104135

Closed

serhiy-storchaka reopened this Jul 29, 2024

Merge branch 'main' into bugs/79846

94c8ec1

serhiy-storchaka reviewed Jul 29, 2024

View reviewed changes

Load only certificates that have correct encoding and purpose.

9d31e3a

serhiy-storchaka requested review from sethmlarson and a team July 30, 2024 09:47

serhiy-storchaka merged commit 9e551f9 into python:main Aug 7, 2024
33 of 34 checks passed

bedevere-app bot removed the awaiting review label Aug 7, 2024

serhiy-storchaka added needs backport to 3.12 bug and security fixes needs backport to 3.13 bugs and security fixes labels Aug 7, 2024

bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Aug 7, 2024

bedevere-app bot removed the needs backport to 3.12 bug and security fixes label Aug 7, 2024

DiablosOffens mentioned this pull request Feb 7, 2025

Reject blank passwords; use SSLContext for all urllib.request.urlopen requests simonrob/email-oauth2-proxy#321

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-79846: Make `ssl.create_default_context()` ignore invalid certificates #91740

gh-79846: Make `ssl.create_default_context()` ignore invalid certificates #91740

pukkandan commented Apr 20, 2022 •

edited by bedevere-bot

Loading

ghost commented Apr 20, 2022 •

edited by ghost

Loading

bedevere-bot commented Apr 20, 2022

serhiy-storchaka commented Feb 23, 2024

ideasman42 commented Jul 28, 2024

serhiy-storchaka Jul 29, 2024

zooba commented Aug 6, 2024

serhiy-storchaka commented Aug 6, 2024

zooba commented Aug 6, 2024

miss-islington-app bot commented Aug 7, 2024

miss-islington-app bot commented Aug 7, 2024

bedevere-app bot commented Aug 7, 2024

bedevere-app bot commented Aug 7, 2024

	# no explicit cafile, capath or cadata but the verify mode is
	# CERT_OPTIONAL or CERT_REQUIRED. Let's try to load default system
	# root CA certificates for the given purpose. This may fail silently.

gh-79846: Make ssl.create_default_context() ignore invalid certificates #91740

gh-79846: Make ssl.create_default_context() ignore invalid certificates #91740

Conversation

pukkandan commented Apr 20, 2022 • edited by bedevere-bot Loading

ghost commented Apr 20, 2022 • edited by ghost Loading

bedevere-bot commented Apr 20, 2022

serhiy-storchaka commented Feb 23, 2024

ideasman42 commented Jul 28, 2024

serhiy-storchaka Jul 29, 2024

Choose a reason for hiding this comment

zooba commented Aug 6, 2024

serhiy-storchaka commented Aug 6, 2024

zooba commented Aug 6, 2024

miss-islington-app bot commented Aug 7, 2024

miss-islington-app bot commented Aug 7, 2024

bedevere-app bot commented Aug 7, 2024

bedevere-app bot commented Aug 7, 2024

gh-79846: Make `ssl.create_default_context()` ignore invalid certificates #91740

gh-79846: Make `ssl.create_default_context()` ignore invalid certificates #91740

pukkandan commented Apr 20, 2022 •

edited by bedevere-bot

Loading

ghost commented Apr 20, 2022 •

edited by ghost

Loading