SSL releated deprecation for 3.6 #72209
Comments
|
I like to deprecate some SSL related parts of Python:
|
tiran
added
stdlib
|
10/10, yes. I will happily provide code review for this. |
I'm not sure about this one: |
|
Another deprecation: I like to deprecate all arguments from SSLSocket.__init__() and require users to go through SSLContext.wrap_socket(). It's going to make the implementation much simpler. The argument list is just crazy: class SSLSocket(socket):
def __init__(self, sock=None, keyfile=None, certfile=None,
server_side=False, cert_reqs=CERT_NONE,
ssl_version=PROTOCOL_TLS, ca_certs=None,
do_handshake_on_connect=True,
family=AF_INET, type=SOCK_STREAM, proto=0, fileno=None,
suppress_ragged_eofs=True, npn_protocols=None, ciphers=None,
server_hostname=None,
_context=None): |
|
I like the idea of using SSLContext as the obvious and only choice to |
|
memo to me: check if SSLContext.wrap_socket() can deal with a fileno as sock argument. |
|
First draft of a patch: https://github.com/tiran/cpython/commits/feature/ssl_deprecation |
|
urllib.request.urlopen() should be affected too right? |
|
Yes, urllib.request.urlopen needs an update too. It takes those certfile and keyfile and usage of those could be deprecated in favor of context. |
|
I have deprecated cafile, capath and cadefault for urlopen(). The function didn't pop up on my radar because I was looking for certfile and cert_file, not cafile. I also added deprecations to the documentation of SSLSocket.read and write. |
|
How does that tie in with SSLObject.read() and write(). I have never used this class, but the documentation refers back to SSLSocket. It seems there are no recv() etc methods to use as an alternative. So maybe deprecate read() and write() on SSLSocket only, and leave SSLObject intact? |
|
+1 for directing all programmatic configuration through SSLContext However, implicitly verifying certificates for protocols other than HTTPS needs to be contingent on a properly designed approach to configuration that leaves informed users in full control of the behaviour of their systems - while I'm fully supportive of secure-by-default behaviour to protect unaware users, it's also the case that most other protocols haven't had the forcing function of web browser behaviour encouraging them to improve their certificate handling, and even that's still in a tragically bad state once you get away from the public web. The file based scheme in PEP-493, https://www.python.org/dev/peps/pep-0493/#backporting-pep-476-to-earlier-python-versions, was deliberately written to be potentially suitable for expansion to other protocols, but actually using it for that purpose would require the definition of a new feature PEP targeting 3.7 (which may then potentially be pitched for backporting to earlier versions as a subsequent proposal). |
|
In the mean time I have reconsidered my position. How about we *document* that a future version of Python will very all TLS/SSL connections by default. Users have to explicitly pass an unverified context if they still want the old behavior. |
|
+1 for a common note in all affected modules along the lines of "An appropriately configured SSLContext should be provided for any use cases that involve accepting self-signed certificates, privately signed certificates, or any other kind of certificate that won't validate against the default system certificate trust store. It is expected that a future version of Python will switch to explicitly verifying SSL certificates for all SSL/TLS protected connections by default (due to the widespread use of self-signed and privately signed certificates for other protocols, full verification is currently the default only for HTTPS)." Regarding ssl.wrap_socket(), would it be feasible to provide a migration path to a situation where that's just a thin wrapper around ssl.get_default_context().wrap_socket()? Comparing the parameter lists: >>> module_params - method_params
{'ciphers', 'keyfile', 'ca_certs', 'ssl_version', 'cert_reqs', 'certfile'}
>>> method_params - module_params
{'server_hostname'}That means the real problems are the ciphers, keyfile, ca_certs, ssl_version, cert_reqs and certfile parameters and the internal use of SSLContext() rather than get_default_context(), rather than the essential idea of providing a shorthand spelling for "wrap a socket with the default SSL/TLS settings". |
|
New changeset aed3c541b2f1 by Christian Heimes in branch 'default': |
|
I have pushed all deprecation except ssl.wrap_socket(). Nick raised some concerns. I like to discourage people to use it because it hurts performance and is no longer best practice. How about we mark the function as legacy function and move it to a less prominent place in the documentation? |
|
I asked in more detail about this on the list, but my main question is why can't wrap_socket() be fixed by doing a rip-and-replace on its internals (e.g. by using a model similar to the one in random, where there's an implicit global Random instance that gets invoked if you use the module level API instead of creating your own instance), rather than having to tell users to change *their* code. Like Random, I'd like to see SSLContext as a lower level implementation detail that's there for when people need it, but can be largely ignored if they just want the default behaviours (i.e. system trust store with python-dev specified SSL/TLS settings) |
|
An implicit global SSL Context? It kinda sounds a bit gross. |
|
Thinking about that more, it's a bit harder than the Random module as well. The only state the random module has to worry about is the seed and internal state of the RNG. However, many of the arguments to ssl.wrap_socket change the SSLContext options for things like what ciphers are active, what trust stores, etc. So we couldn't have a single SSLContext at the global level without removing those options from wrap_socket. Otherwise we'd need some sort of dict of SSLContexts that keyed off of the options passed to wrap_socket. |
|
That sounds like the "re" module would be a better exemplar for an SSL module convenience API design than "random" then - that has a similar model of needing an LRU cache for the compiled patterns for performance reasons, while still making working with the compiled form in your own code optional (which means you don't need to find a place to store it to gain the performance benefits of pattern reuse). It would need to be a hidden cache, though - since SSLContext objects are mutable, it wouldn't be a good idea to expose any implicitly shared ones. |
|
The performance benefit is not worth the risk. For 10 httplib requests to pypi.python.org, a shared SSLContext is about 5% faster than a new context for each request. Session resumption improves the simple test case by another 20%. |
|
Leaving the option of context caching entirely to the caller would definitely make things simpler - my main interest is just in avoiding a hard compatibility break for folks that aren't doing anything particularly wrong, by which I mean specifically cases where a wrap_socket() implementation like this one would continue to work for them: def wrap_socket(sock, *args, *kwds):
return ssl.get_default_context().wrap_socket(sock, *args, **kwds) |
|
New test failure when using -Werror: ====================================================================== Traceback (most recent call last):
File "/media/disk/home/proj/python/cpython/Lib/test/test_httplib.py", line 1646, in test_local_bad_hostname
check_hostname=True)
File "/media/disk/home/proj/python/cpython/Lib/http/client.py", line 1373, in __init__
DeprecationWarning, 2)
DeprecationWarning: key_file, cert_file and check_hostname are deprecated, use a custom context instead. |
|
New changeset 2e541e994927 by Christian Heimes in branch 'default': |
|
Thanks for the report. "./python -Werror -m test -uall test_httplib" is now passing for me. |
|
test_imaplib is failed too. ====================================================================== Traceback (most recent call last):
File "/home/serhiy/py/cpython-debug/Lib/test/test_imaplib.py", line 641, in test_logincapa_with_client_certfile
_server = self.imap_class(self.host, self.port, certfile=CERTFILE)
File "/home/serhiy/py/cpython-debug/Lib/imaplib.py", line 1273, in __init__
"custom ssl_context instead", DeprecationWarning, 2)
DeprecationWarning: keyfile and certfile are deprecated, use acustom ssl_context instead |
|
New changeset 57e88d1159fc by Christian Heimes in branch 'default': |
|
Shouldn't this issue be closed? Since SSL context arguments are not supported in 2.7, the deprecated arguments can't be removed until EOL of 2.7. |
|
Hello everyone, and thank you as usual for all your hard work keeping the python ecosystem going. I saw that the start of this thread said it was going to
but this hasn't been done, at least not for imaplib. imaplib is still calling the undocumented "ssl._create_stdlib_context": Line 1320 in 2b496e7 which is actually "ssl._create_unverified_context": Line 842 in 2b496e7 which is indeed unverified: despite defaulting to PROTOCOL_TLS_CLIENT, which "enables CERT_REQUIRED and check_hostname by default.", it overrides that by setting check_hostname=False: Line 811 in 2b496e7 To demonstrate, check out this tester script: $ cat a.py
import os, imaplibwith imaplib.IMAP4_SSL(os.environ.get('HOSTNAME')) as S:
print(S.login(os.environ.get('USERNAME'), os.environ.get('PASSWORD')))
$ HOSTNAME=46.23.90.174 USERNAME=test1 PASSWORD=test1test1 python3 a.py
('OK', [b'Logged in'])I don't have a cert for 46.23.90.174 (no one will give out certs for IPs!), so this is wrong! In order to actually enable verification you need to know the incantation. It's not that long but it is subtle and frighteningly easy to get wrong. Here it is: $ cat a.py
import os, ssl, imaplibctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
ctx.load_default_certs()
with imaplib.IMAP4_SSL(os.environ.get('HOSTNAME'), ssl_context=ctx) as S:
print(S.login(os.environ.get('USERNAME'), os.environ.get('PASSWORD')))
$ HOSTNAME=46.23.90.174 USERNAME=test1 PASSWORD=test1test1 python3 a.py
Traceback (most recent call last):
File "a.py", line 6, in <module>
with imaplib.IMAP4_SSL(os.environ.get('HOSTNAME'), ssl_context=ctx) as S:
File "/usr/lib/python3.6/imaplib.py", line 1288, in __init__
IMAP4.__init__(self, host, port)
File "/usr/lib/python3.6/imaplib.py", line 198, in __init__
self.open(host, port)
File "/usr/lib/python3.6/imaplib.py", line 1301, in open
IMAP4.open(self, host, port)
File "/usr/lib/python3.6/imaplib.py", line 299, in open
self.sock = self._create_socket()
File "/usr/lib/python3.6/imaplib.py", line 1293, in _create_socket
server_hostname=self.host)
File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "/usr/lib/python3.6/ssl.py", line 817, in __init__
self.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 1077, in do_handshake
self._sslobj.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 694, in do_handshake
match_hostname(self.getpeercert(), self.server_hostname)
File "/usr/lib/python3.6/ssl.py", line 327, in match_hostname
% (hostname, ', '.join(map(repr, dnsnames))))
ssl.CertificateError: hostname '46.23.90.174' doesn't match either of 'comms.kousu.ca', 'comms3.kousu.ca'I can see from this thread there were some concerns about breaking people's self-signed certs back in 2016. But it's five years later now and letsencrypt is super common now, and most servers and clients are enforcing TLS, especially when credentials are involved. Could this be revisited? Thanks for any attention you have gifted to this :) |
|
The part with "make ftplib, imaplib, nntplib, pop3lib, smtplib etc. validate certs by default" was not implemented. These modules still default to unverified connections. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: