SSL certificate error with pip 21.0.1

[rajesh0therascal]

pip version

21.0.1

Python version

3.7

OS

Windows 10

Additional information

After upgrading pip to latest version i.e. 21.0.1, I am unable to install any other packages.

Description

I am able to install packages with previous versions (19.0.3) of pip. Problem is with only 21.0.1.

Expected behavior

No response

How to Reproduce

Step 1 :

python -m pip install --upgrade pip 

Step 2:

 pip install matplotlib 

Output

WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1076)'))': /simple/matplotlib/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1076)'))': /simple/matplotlib/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1076)'))': /simple/matplotlib/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1076)'))': /simple/matplotlib/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1076)'))': /simple/matplotlib/
Could not fetch URL https://pypi.org/simple/matplotlib/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/matplotlib/ (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1076)'))) - skipping
ERROR: Could not find a version that satisfies the requirement matplotlib
ERROR: No matching distribution found for matplotlib
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1076)'))) - skipping

Code of Conduct

• I agree to follow the PSF Code of Conduct #10101

[bastiaan85]

Do you have a internet proxy configured at the Config Panel -> Internet Options? If so disable it and try again. If it then times out, add the proxy on the command line using --proxy http://address:port.

[rajesh0therascal]

@bastiaan85 Not helping me. Similar output. I am able to install with previous versions. Once i upgrade pip to 21.0.1, packages stop installing.

[bastiaan85]

What does it print out when you launch a PowerShell and try the three commands mentioned in #9216 (comment) ?

[rajesh0therascal]

@bastiaan85 It prints out my proxy server and proxy port number.

[bastiaan85]

If it's printing it in the third command, then it means you didn't disable the proxy properly. If it prints it only in the first and/or second command, then verify that the url set there starts with 'http' and not 'https'.

[rajesh0therascal]

@bastiaan85 If I disable proxy, i wont be able to connect to internet. My PC is behind proxy and all other applications will stop working.

[bastiaan85]

I understand, but I don't know of a another way to be able to let pip download something via a http-only proxy. As the bug here is that urllib3 will assume the proxy it found in the system settings (third command) is a https-proxy and will not fall back to http when its request fails (that's why you see the SSL errors in pip). So until this is fixed in urllib3 the workaround is to disable the system proxy, set the proxy in the pip command line using --proxy http://address:port to let pip download its things, then enable the system proxy again.

[anujsuchchal]

I was fighting with my organization's networking team from last one month to remove this error and finally I got rid off it by using --proxy http://address:port
Thanks a lot!

[CrazyBoyFeng]

I downgrade pip to 20.2.3 and it works fine.

[muellert]

I have the same problem on Linux with Python 3.9.1, and running pip install (20.2.3) does not fix it. I am behind an SSL proxy which I can't circumvent, so I can only re-install Python and then everything I installed since (except for a newer pip).

[bastiaan85]

I have the same problem on Linux with Python 3.9.1, and running pip install (20.2.3) does not fix it. I am behind an SSL proxy which I can't circumvent, so I can only re-install Python and then everything I installed since (except for a newer pip).

That can't be the same issue. The mentioned issue is Windows users behind a HTTP proxy, as there the 'system proxy' is set using the Internet Options panel, which then gets picked up by urllib3, assumed to be a HTTPS proxy, so it uses a SSL client, which then fails on the proxy (being a dumb HTTP endpoint) not responding with a proper SSL handshake. This is represented by the SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number message. This 'assumption bug' didn't occur before 21.0. So judging by both your OS as the fact that downgrading didn't help means you are experiencing another connectivity issue.

This could be related to either a schema mismatch (http:// in the environment variables while https:// should be used) or that you are in fact not behind a SSL proxy at all and thus do need to use http:// as the proxy url schema. I would recommend running tcpdump or wireshark to capture the traffic during a working example (eg using your browser to visit a site) then compare the traffic when running pip.

[muellert]

My proxy setup is correct, and furtnermore, out of my control (corporate networking here). I've successfully installed a number of packages with the original version of pip. But after upgrading, I got the SSL protocol mismatch problem, and after downgrading, my connections just time out. I would like to debug further, but don't know how.

[bastiaan85]

I understand, but the point I'm trying to make is that you are replying to a thread that discusses an opposite issue. Protocol mismatch usually means you are using a HTTP client on a HTTPS endpoint, while the thread is about urrllib3 erroneously using HTTPS for a HTTP system proxy.

However the 'protocol mismatch' does confirm that you are using an SSL proxy, so that points toward my first suggestion above.

a schema mismatch (http:// in the environment variables while https:// should be used)

Use echo $http_proxy and echo $https_proxy to check that they use the correct https:// url to your proxy. And as I mentioned, use Wireshark to double check the traffic flows in both pip's situation and in working situation with for example a browser.

[muellert]

Thanks, problem solved. My bad...

[junqfisica]

I have found the root cause of the problem and pip can fix its end.

I understand that urllib is getting the wrong scheme for https. A workaround would be to set https_proxy= http://your_proxy. However, this solution is not ideal. Many users behind proxy don't have full access to their system, also if you use IDEs like Pycharm this will not work for its installer. The SSL error also occurs when using Ansible to deploy to a Linux server.

Saying that pip has a way to pass proxy running: pip install package --proxy http://your_proxy, which by the way is how Ansible or Pycharm handle proxy to pip. And that is where the problem lies, after some debug I have realise that pip is not passing it to urllib request, this problem was unnoticed until now because, before urllib update, it didn't check ssl_certificate so it didn't matter if your scheme was either http or https.

I'm using pip 21.0.1 to debug and a venv.

To reproduce:

• Set a fake proxy using system var. i.e: set https_proxy: http://127.0.0.1:80 or export https_proxy: http://127.0.0.1:80 for Linux
• Now run pip passing a different proxy, i.e: pip install numpy --proxy http://127.0.0.2:80 or pip.main(['install', '--proxy=http://127.0.0.2:80', 'numpy']) for esier debug.

I would expect pip to use the proxy I'm passing to it and override any system proxy, but that is not the case. To see that add a breakpoint at the request method from PipSession class in pi\_internal\network\session.py. Here you can notice that PipSession has a parameter self.proxies that match the proxy I have passed to pip with --proxy. Then, this makes a call to the parent request
super().request(method, url, *args, **kwargs) which is Session class from pi\_vendor\requests\session.py. One of its kwargs is proxies, add a breakpoint there and you will notice that the proxies you get are from the system, whatever you set as set https_proxy: http://127.0.0.1:80.

So basically pip is not passing through the proxies set by the user when invoking pip, and that is the root cause of all the following errors with SSL.

This is the original request method of PipSession:

def request(self, method, url, *args, **kwargs):
        # Allow setting a default timeout on a session
        kwargs.setdefault("timeout", self.timeout)
        
        # Dispatch the actual request
        return super().request(method, url, *args, **kwargs)

An easy fix I suggest is to add the following line at the request method in the file pi_internal\network\session.py:

def request(self, method, url, *args, **kwargs):
     # Allow setting a default timeout on a session
     kwargs.setdefault("timeout", self.timeout)
     kwargs.setdefault("proxies", self.proxies)  # fix problem with proxies.

    # Dispatch the actual request
    return super().request(method, url, *args, **kwargs)

This way the proxies are carried out correctly to the request, if no proxy is set an empty dict is passed.

So now if you run pip install package --proxy http://proxy_ip everything works as expected. Because I'm using the right scheme for my proxy.

I hope this will help to remove this bug in the next version of pip asap.

[pfmoore]

I have found the root cause of the problem and pip can fix its end.

Thank you, that sounds like a good analysis. If someone is interested in raising a PR for this (preferably including a test that fails with current pip, so that we can avoid any regressions in future), we can look at getting this fix applied.

[CrazyBoyFeng]

I have found the root cause of the problem and pip can fix its end.

So there are two reasons for this error:

1. urllib misparses Windows registry proxy settings. This bug is being processing.
2. pip --proxy does not work while the system proxy is set.
   According to --cert and global.cert not working #9614 , this bug is not only related to system proxies, but also other system/environment settings.
   The real reason may be in merge_setting():

   pip/src/pip/_vendor/requests/sessions.py

   Lines 722 to 725 in 7b7469b

   	proxies = merge_setting(proxies, self.proxies)
   	stream = merge_setting(stream, self.stream)
   	verify = merge_setting(verify, self.verify)
   	cert = merge_setting(cert, self.cert)

   
   

   pip/src/pip/_vendor/requests/sessions.py

   Line 50 in 7b7469b

   def merge_setting(request_setting, session_setting, dict_class=OrderedDict):

   
   I am now trying to check this function.

[junqfisica]

@CrazyBoyFeng First, thank you to take this problem into consideration.

I don't think the problem with pip --proxy is related to the proxies = merge_setting(proxies, self.proxies) . You may be right there about the problem with the URL scheme.

However, it is pip's child class (PipSession(requests.Session)) job to pass the proxy set via pip --proxy to requests.Session.request(...).

this is a pice of code inside requests.Session.request():

 def request(self, method, url,
            params=None, data=None, headers=None, cookies=None, files=None,
            auth=None, timeout=None, allow_redirects=True, proxies=None,
            hooks=None, stream=None, verify=None, cert=None, json=None):

        proxies = proxies or {}

        settings = self.merge_environment_settings(
            prep.url, proxies, stream, verify, cert
        )

You can see that proxies will be an empty dict if nothing is passed to it. Then self.merge_environment_settings will try to get proxies from the system causing the problem with SSLError for https scheme.

However, if pip calls his method passing the proxy, everything works fine. As I have mentioned in the post above. The method merge_setting(proxies, self.proxies) only gets a wrong proxy if no proxy is passed to the request method. The constructor of requests.Session makes self.proxies = {}, so you must give proxies to its request method.

This is a tricky situation because usually there is no reason for the --proxy, set by the user, to be different from the one set in the system, regardless of the scheme. The point is, by my understanding so far, the parameter pip --proxy is useless, which is a big problem because many tools rely on it to pass the proxy.

It's also worth mentioning that this problem is not only related to Windows, I also have a problem using pip >= 20.3 in a Linux server via Ansible.

If you fix the problem with requests.Session, probably the SSL error will disappear, but the issue with pip --proxy will still be there causing future problems. As an end-user I expect pip to carry out whatever I pass to --proxy to the request. So I still think the core of this problem is in PipSession().request(...)

[CrazyBoyFeng]

I don't think the problem with pip --proxy is related to the proxies = merge_setting(proxies, self.proxies) . You may be right there about the problem with the URL scheme.

Yes, you are right. I have found I waste hours on merge_settings() yet.

Do you plan to make a PR patch?

[junqfisica]

@CrazyBoyFeng

Would be nice if you could do the PR patch or someone else. Currently, I'm using working hours for that...so :)

[QianYC]

Ubuntu1604, same issue

[Faholan]

This issue also happens even when you have configured the proxy through pip config : I'm under Windows 10, pip 21.0.1.

I have configured the proxy through pip config --global, with the http:// scheme. There is also a system proxy configured.

Downgrading to pip 20.2.3 fixes the issue though : the http proxy is used, and the installation is successfull.

[muellert]

1. `urllib` misparses Windows registry proxy settings. This [bug](https://bugs.python.org/issue42627) is being processed..

Thank you everyone for digging into this and actually finding a solution, but just for the record, I would like to note that my problem is completely unrelated to Windows, as I have been working on Linux only.

[bastiaan85]

That's why there were two reasons mentioned, not just one.

[lukasgebhard]

On Windows 10, the following two steps allowed me to install pip packages in a conda environment.

1. Set environment variables for proxy:

set HTTP_PROXY=http://<url>:<port>
set HTTPS_PROXY=https://<url>:<port>

2. Use Python 3.8 and pip 20.2.2:

conda install pip=20.2.2=py38_0

[0-6-1-7]

set HTTPS_PROXY=https://:

Thank you for this workaround!
In my case
Win 10 + Python 3.8.3 - 3.9.2 + corporate proxy + lazy admin
I set
HTTPS_PROXY=http://<url>:<port>
Yes, no 's' in proxy protocol prefix!
There is no need to set system-wide env. var, just start each pip session with this command.

[CrazyBoyFeng]

1. urllib misparses Windows registry proxy settings. This bug is being processing.
2. pip --proxy does not work while the system proxy is set.

--proxy parameter bug is a bug from requests library, and it is being processing at psf/requests#5735

[muellert]

Further debugging shows that, for some very strange reason, I can't load the SSL module in Python 3.9.5, but can load it in Python versions 3.6, 3.7 and 3.8, compiled with the same settings on the same machine, and in the same shell.

I'm not sure whether this is the same bug as before, or a different bug? I am hesitant to "test", since I don't know how to revert to an older version of pip - it seems to only support going forward, not backward.

[uranusjr]

The ssl module is built in, so if it changes in 3.9, you should report this to CPython, pip maintainers can't do anything for you.

[GoPapaSmurf]

I had this error message as well, with python-3.9.6-amd64 on Win10. I downgraded to python-3.8.3-amd64, rebooted, and this SSL error disappeared. I can now use pip again.

[CrazyBoyFeng]

Some news from psf/requests#5735:
The maintainers of requests believe that this commit would break backwards compatibility, so they are not considering changing it for now.
They also propose that pip.session.proxy can currently be handled in a similar way to pip.session.timeout, i.e. setting a proxy parameter for each request.

[junqfisica]

@CrazyBoyFeng That's correct.
That's why I have made the PR #10680 to pip, which handles pip.session.proxies the same way as pip.session.timeout. However, the PR is still waiting for checks/approvals from the maintainers.
Do you have any idea of how long it will take for the PR to be processed?

[CrazyBoyFeng]

@junqfisica
I'm not sure when we'll wait for a final solution either, I've even gotten used to the alternative of using environment variables over the year.
I'm sorry that now we have to wait for the decision of pip's maintainers.

[nicolaspbl]

In my case (Windows 10, corporate proxy, pip 21.2.3, python 3.10.0) I solved the issue after reading, attempting and mixing a lot things.

1. Environment variables

You need to set the 2 variables with appropriate values. NB https_proxy var doesn't contain "https://" but "http://"

set http_proxy=http://username:password@proxy-name:port
set https_proxy=http://username:password@proxy-name:port

If you stop at this point, and try for example : pip install pandas you get errors such as :

WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)'))': /simple/pandas/
Could not fetch URL https://pypi.org/simple/pandas/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pandas/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)'))) - skipping

2. Certificate

After having launched certmgr.msc I have exported in X.509 base 64 encoded format, the ROOT CA of my company I found in "Trusted Root Certification Authorities", into a temporary text file.

I pasted the temporary file content into cacert.pem located in Installation_Dir\Lib\site-packages\pip\_vendor\certifi

Then, I successfully could install modules with pip install module_name

Et voilà, c'est tout simple !

Nicolas.

[pradyunsg]

I imagine this issue has users who are trying to use pip within an organisation, behind a proxy or something else. Please see #8200 (comment) if that's your situation.

Some of the cases affected by proxy issue should also have been fixed by #10680 (which will the next pip release). There's a Windows-specific Python standard library bug (https://bugs.python.org/issue42627) that doesn't really need to be tracked on our end.

Closing this, since I don't think there isn't really anything actionable left here, that would involve making a dedicated code change in this project.