Issues in _urllib2_fork.py under Python 3.12 and newer due to changes in HTTPSConnection

[ericwallace904]

I'm getting this error at the point I try to run a mechanize.Browser.open():
TypeError: HTTPSConnection.init() got an unexpected keyword argument 'key_file'

Error reporting I'm seeing around other packages/modules pointed me to changes with HTTPSConnection:
https://docs.python.org/3.12/library/http.client.html#http.client.HTTPSConnection

I was seeing other issues related to HTTPS under mechanize 0.4.10, but also seeing it under mechanize 0.4.8 which previously worked for my code under an earlier version of Python.

edit: apparently those parameters will deprecated in Python 3.6:
python/cpython#72209

[kovidgoyal]

Mechanize hasnt used key_file in forever since python 3.5. If you are
getting an error you need to post the full traceback and what version of
mechanize you are actually using to get the traceback.

[ericwallace904]

This is with mechanize 0.4.8

Traceback (most recent call last):
File "program.py", line 118, in
br.open(br_request)
File "env\Lib\site-packages\mechanize_mechanize.py", line 257, in open
return self._mech_open(url_or_request, data, timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "env\Lib\site-packages\mechanize_mechanize.py", line 287, in _mech_open
response = UserAgentBase.open(self, request, data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "env\Lib\site-packages\mechanize_opener.py", line 193, in open
response = urlopen(self, req, data)
^^^^^^^^^^^^^^^^^^^^^^^^
File "env\Lib\site-packages\mechanize_urllib2_fork.py", line 425, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "env\Lib\site-packages\mechanize_urllib2_fork.py", line 414, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "env\Lib\site-packages\mechanize_urllib2_fork.py", line 1283, in https_open
return self.do_open(conn_factory, req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "env\Lib\site-packages\mechanize_urllib2_fork.py", line 1196, in do_open
h = http_class(host_port, timeout=req.timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: HTTPSConnection.init() got an unexpected keyword argument 'key_file'

[kovidgoyal]

Does not reproduce for me with current mechanize. Update.

python -c 'import sys; from mechanize import Browser; print(sys.version); print(len(Browser().open("https://bing.com").read().decode()))'
3.12.7 (main, Oct 1 2024, 11:15:50) [GCC 14.2.1 20240910]
175584

[ericwallace904]

This is what I get with mechanize 0.4.10, which is what led me to trying 0.4.8 since it previously worked for me (under older Python):

Traceback (most recent call last):
File "env\Lib\site-packages\mechanize_urllib2_fork.py", line 1242, in do_open
h.request(str(req.get_method()), str(req.get_selector()), req.data,
File "Python\Python312\Lib\http\client.py", line 1336, in request
self._send_request(method, url, body, headers, encode_chunked)
File "Python\Python312\Lib\http\client.py", line 1382, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "Python\Python312\Lib\http\client.py", line 1331, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "Python\Python312\Lib\http\client.py", line 1091, in _send_output
self.send(msg)
File "Python\Python312\Lib\http\client.py", line 1035, in send
self.connect()
File "Python\Python312\Lib\http\client.py", line 1477, in connect
self.sock = self._context.wrap_socket(self.sock,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "Python\Python312\Lib\ssl.py", line 455, in wrap_socket
return self.sslsocket_class._create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "Python\Python312\Lib\ssl.py", line 1041, in _create
self.do_handshake()
File "Python\Python312\Lib\ssl.py", line 1319, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "program.py", line 118, in
br.open(br_request)
File "env\Lib\site-packages\mechanize_mechanize.py", line 257, in open
return self._mech_open(url_or_request, data, timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "env\Lib\site-packages\mechanize_mechanize.py", line 287, in _mech_open
response = UserAgentBase.open(self, request, data)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "env\Lib\site-packages\mechanize_opener.py", line 193, in open
response = urlopen(self, req, data)
^^^^^^^^^^^^^^^^^^^^^^^^
File "env\Lib\site-packages\mechanize_urllib2_fork.py", line 431, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "env\Lib\site-packages\mechanize_urllib2_fork.py", line 420, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "env\Lib\site-packages\mechanize_urllib2_fork.py", line 1296, in https_open
return self.do_open(conn_factory, req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "env\Lib\site-packages\mechanize_urllib2_fork.py", line 1246, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)>

[kovidgoyal]

This has nothing to do with mechanize, its python's ssl module failing
to verify the certificate. On Windows that's typically because an
intermediate certificate is not in the OS certificate store, which you
can usually fix by visiting the website in question with edge. Or just
install something like cacerts and set the SSL_CERT_FILE env var to get
python to use those certs instead of windows ones. Google is your
friend.