pip 9.0.1: SLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)

[az-z]

• Pip version: 9.0.1
• Python version: 2.7.13
• Operating system: Fedora 25

Description:

expect a normal operation, not an exception.
a note about my environment - i'm runing behind cntlm and the corp proxy.
the pip works only if i provide "--trusted-site" command line parameter.
the valid certificate has been placed into /etc/pki/ca-trust/, but since python is NOT checking that location:

python -c "import ssl; print(ssl.get_default_verify_paths())"
DefaultVerifyPaths(cafile='/etc/pki/tls/cert.pem', capath='/etc/pki/tls/certs', openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/etc/pki/tls/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/etc/pki/tls/certs')

I linked it to /etc/pki/tls/certs/:

[root@netdev ~]# ln -s /etc/pki/ca-trust/Proxy_215_PEM.cer /etc/pki/tls/certs/
[root@netdev ~]# ls -l /etc/pki/tls/certs/
total 12
lrwxrwxrwx 1 root root   49 Apr  4 17:37 ca-bundle.crt -> /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
lrwxrwxrwx 1 root root   55 Apr  4 17:37 ca-bundle.trust.crt -> /etc/pki/ca-trust/extracted/openssl/ca-bundle.trust.crt
-rwxr-xr-x 1 root root  610 Feb  6 11:04 make-dummy-cert
-rw-r--r-- 1 root root 2516 Feb  6 11:04 Makefile
lrwxrwxrwx 1 root root   35 May  1 12:33 Proxy_215_PEM.cer -> /etc/pki/ca-trust/Proxy_215_PEM.cer
-rwxr-xr-x 1 root root  829 Feb  6 11:04 renew-dummy-cert

yet, still i'm facing the same error.

Why is this an issue:

any software that relies on pip would fail in the situation where it must provide command line parameter. for example atom's apm.

What I've run:

pip -v search pip

python -V
Python 2.7.13
[az@netdev ~]$ pip -V
pip 9.0.1 from /usr/lib/python2.7/site-packages (python 2.7)
[az@netdev ~]$ pip -v search pip 
Starting new HTTPS connection (1): pypi.python.org
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/site-packages/pip/commands/search.py", line 45, in run
    pypi_hits = self.search(query, options)
  File "/usr/lib/python2.7/site-packages/pip/commands/search.py", line 62, in search
    hits = pypi.search({'name': query, 'summary': query}, 'or')
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1243, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib64/python2.7/xmlrpclib.py", line 1602, in __request
    verbose=self.__verbose
  File "/usr/lib/python2.7/site-packages/pip/download.py", line 775, in request
    headers=headers, stream=True)
  File "/usr/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 522, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/lib/python2.7/site-packages/pip/download.py", line 386, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 596, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/pip/_vendor/cachecontrol/adapter.py", line 47, in send
    resp = super(CacheControlAdapter, self).send(request, **kw)
  File "/usr/lib/python2.7/site-packages/pip/_vendor/requests/adapters.py", line 497, in send
    raise SSLError(e, request=request)
**SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)**

at the same time:

[az@netdev ~]$ pip -v search pip --trusted-host pypi.python.org
Starting new HTTPS connection (1): pypi.python.org
"POST /pypi HTTP/1.1" 200 None
test-install-1 (1.0.0)                               - this is to test the pip install
abel-airflow (1.7.1.3.post3)                         - Programmatically author, schedule and monitor data pipelines
pip-accel (0.43)                                     - Accelerator for pip, the Python package manager

<skipped>

[noamelf]

We had this error too with:

$ pip -V
pip 9.0.1 from /opt/virtenv/lib/python3.6/site-packages (python 3.6)

[marekyggdrasil]

Same here.

System

      System Version: OS X 10.11.3 (15D21)
      Kernel Version: Darwin 15.3.0

OpenSSL

$ openssl version -a
OpenSSL 1.0.2l  25 May 2017
built on: reproducible build, date unspecified
platform: darwin64-x86_64-cc
options:  bn(64,64) rc4(ptr,int) des(idx,cisc,16,int) idea(int) blowfish(idx) 
compiler: cc -I. -I.. -I../include  -fPIC -fno-common -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -arch x86_64 -O3 -DL_ENDIAN -Wall -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM
OPENSSLDIR: "/Users/marek/miniconda2/ssl"

pip

$ pip -V
pip 9.0.1 from /Users/human/virtualenvs/mypy2.7/lib/python2.7/site-packages (python 2.7)

[dbellavista]

Same here. I'm trying to use pip behind a corporate proxy.

Ubuntu 16.04
pip 9.0.1

I have correctly setup the proxy:

• /usr/lib/ssl/certs/ca-certificates.crt correctly include my CA
• /usr/lib/ssl/cert.pem is a symlink to /usr/lib/ssl/certs/ca-certificates.crt
• curl works correcly

python -c "import ssl; print(ssl.get_default_verify_paths())"

DefaultVerifyPaths(cafile='/usr/lib/ssl/cert.pem', capath='/usr/lib/ssl/certs', openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/usr/lib/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/usr/lib/ssl/certs')

pip search SimpleHTTPServer

Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/pip-9.0.1-py2.7.egg/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/local/lib/python2.7/dist-packages/pip-9.0.1-py2.7.egg/pip/commands/search.py", line 45, in run
    pypi_hits = self.search(query, options)
  File "/usr/local/lib/python2.7/dist-packages/pip-9.0.1-py2.7.egg/pip/commands/search.py", line 62, in search
    hits = pypi.search({'name': query, 'summary': query}, 'or')
  File "/usr/lib/python2.7/xmlrpclib.py", line 1243, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib/python2.7/xmlrpclib.py", line 1602, in __request
    verbose=self.__verbose
  File "/usr/local/lib/python2.7/dist-packages/pip-9.0.1-py2.7.egg/pip/download.py", line 775, in request
    headers=headers, stream=True)
  File "/usr/local/lib/python2.7/dist-packages/pip-9.0.1-py2.7.egg/pip/_vendor/requests/sessions.py", line 522, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pip-9.0.1-py2.7.egg/pip/download.py", line 386, in request
    return super(PipSession, self).request(method, url, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pip-9.0.1-py2.7.egg/pip/_vendor/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pip-9.0.1-py2.7.egg/pip/_vendor/requests/sessions.py", line 596, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pip-9.0.1-py2.7.egg/pip/_vendor/cachecontrol/adapter.py", line 47, in send
    resp = super(CacheControlAdapter, self).send(request, **kw)
  File "/usr/local/lib/python2.7/dist-packages/pip-9.0.1-py2.7.egg/pip/_vendor/requests/adapters.py", line 497, in send
    raise SSLError(e, request=request)
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)

• pip --cert /usr/lib/ssl/cert.pem search SimpleHTTPServer works
• as well as REQUESTS_CA_BUNDLE="/usr/lib/ssl/cert.pem" pip search SimpleHTTPServer

However since I'm using an automatic provisioning system I don't want to edit the pip command to always run with --cert or setting up specific environments just for supporting pip. And for security reasons, using PIP_TRUSTED_HOST is not an options.

Is there a way to automatically setup the certfile?

[0x00evil]

Got the same problem when install pyspark with pip3.
pip3 install pyspark

OS: MacOS 10.11
pip: 9.0.1

[butterl]

any solution here？

[dpdornseifer]

There is an issue regarding TLS name verification in pip 9.0.1. You can update it via

$ curl https://bootstrap.pypa.io/get-pip.py >> get-pip.py
$ python get-pip.py

If you are using virtualenv, you can use virtualenv 15.2.0 (https://pypi.org/project/virtualenv/) there the pip version has been fixed.

[pradyunsg]

On older versions of pip, you might be facing these SSL issues (likely due to the lack of support for TLS 1.2 on the system).

If you trust the hosts for PyPI or upgrade to a newer pip using:

pip install --upgrade --user pip --trusted-host=pypi.python.org --trusted-host=pypi.org --trusted-host=files.pythonhosted.org

If the above doesn't work, let me know. :)

[25b3nk]

Still could not use pip3 to install or upgrade.
System: Ubuntu 16.04
Python-3.6.3
pip-9.0.1

[pradyunsg]

/cc @ewdurbin @dstufft for inputs here.

[henrytom1703]

Hello,
I have also this issues.

C:\Users\Abc>curl https://bootstrap.pypa.io/get-pip.py | python
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1604k  100 1604k    0     0  1604k      0  0:00:01  0:00:01 --:--:-- 1532k
Collecting pip
c:\users\Abc~1\appdata\local\temp\tmpd1uzof\pip.zip\pip\_vendor\urllib3\util\ssl_.py:369: SNIMissingWarning: An HTTPS request has been made, but the SNI (Server Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
c:\users\Abc~1\appdata\local\temp\tmpd1uzof\pip.zip\pip\_vendor\urllib3\util\ssl_.py:160: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '_ssl.c:507: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version'),)': /simple/pip/
c:\users\Abc~1\appdata\local\temp\tmpd1uzof\pip.zip\pip\_vendor\urllib3\util\ssl_.py:160: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '_ssl.c:507: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version'),)': /simple/pip/
c:\users\Abc~1\appdata\local\temp\tmpd1uzof\pip.zip\pip\_vendor\urllib3\util\ssl_.py:160: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '_ssl.c:507: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version'),)': /simple/pip/
c:\users\Abc~1\appdata\local\temp\tmpd1uzof\pip.zip\pip\_vendor\urllib3\util\ssl_.py:160: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '_ssl.c:507: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version'),)': /simple/pip/
c:\users\Abc~1\appdata\local\temp\tmpd1uzof\pip.zip\pip\_vendor\urllib3\util\ssl_.py:160: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
Operation cancelled by user
c:\users\Abc~1\appdata\local\temp\tmpd1uzof\pip.zip\pip\_vendor\urllib3\util\ssl_.py:160: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError(SSLError(1, '_ssl.c:507: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version'),)) - skipping

pip 18.0 from c:\python27\lib\site-packages\pip (python 2.7)

[wkruse]

Same here

pip 19.0.3 from /usr/local/lib/python3.7/site-packages/pip (python 3.7)

python -c "import ssl; print(ssl.get_default_verify_paths())"
DefaultVerifyPaths(cafile='/etc/ssl/certs/ca-bundle.crt', capath='/etc/ssl/certs', openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/etc/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/etc/ssl/certs')

pip install --no-cache-dir awscli
  Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)'))': /simple/awscli/

pip --cert /etc/ssl/certs/ca-bundle.crt install --no-cache-dir awscli works.

Seems like SSL_CERT_FILE is ignored by pip.

[ghost]

i'm getting the same error

 Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /simple/numpy/
  Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /simple/numpy/
  Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /simple/numpy/
  Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /simple/numpy/
  Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /simple/numpy/
  Could not fetch URL https://pypi.org/simple/numpy/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/numpy/ (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)) - skipping
  Could not find a version that satisfies the requirement numpy (from versions: )
No matching distribution found for numpy

[ghost]

I'm facing the same issue from morning.

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = false
name = "pypi"

[packages]
pandas = "*"

[dev-packages]

[requires]
python_version = "3.6"

Installing dependencies from Pipfile…
An error occurred while installing pandas --trusted-host pypi.python.org! Will try again.
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/1 — 00:00:08
Installing initially–failed dependencies…
Collecting pandas ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/1 — 00:00:00
  Could not fetch URL https://pypi.python.org/simple/pandas/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.python.org', port=443): Max retries exceeded with url: /simple/pandas/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)) - skipping

I'hv tried all options suggested but doesn't work, I spent entire day debugging it but of no use.

Mac 10.13.6
py 3.6

[Ranjanpiyush]

i am also facing the same issue since the last 15 hours. Tried using the trusted-source that works for pip but when connecting to Salesforce through python using simple_salesforce i am getting the same SSL Error. any help will be highly appreciated.
Python version 3.6.6
OS Windows 8

[ghost]

I removed verify_ssl = True and it's working for now.
However, this issue made a mess of my python environment. I did many things as mentioned in variosu SO posts and don't even remember what all I did. Final result is that my python env is broken and I'hv installed py 3.7 fresh in a separate venv and using it for the time being.

Really frustrating experience

[Ranjanpiyush]

i reinstalled python and all the relevant packages but still getting the same error. Is it possible this might be caused by the firewall settings of my organization?

[cdauth]

I came across this error while operating in a chroot environment. I solved it by running on the host system:

mount --rbind /run /chroot/run
mount --rbind /dev /chroot/dev
mount --rbind /proc /chroot/proc
mount --rbind /sys /chroot/sys

[KhanterWinters]

I am getting this while trying to install a Package:

PS C:\> python -m pip install -U --user Red-DiscordBot
Collecting Red-DiscordBot
  WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SS
LError(SSLCertVerificationError(1, "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: format error in certific
ate's notBefore field (_ssl.c:1056)"))': /simple/red-discordbot/
  WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SS
LError(SSLCertVerificationError(1, "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: format error in certific
ate's notBefore field (_ssl.c:1056)"))': /simple/red-discordbot/
  WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SS
LError(SSLCertVerificationError(1, "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: format error in certific
ate's notBefore field (_ssl.c:1056)"))': /simple/red-discordbot/
  WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SS
LError(SSLCertVerificationError(1, "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: format error in certific
ate's notBefore field (_ssl.c:1056)"))': /simple/red-discordbot/
  WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SS
LError(SSLCertVerificationError(1, "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: format error in certific
ate's notBefore field (_ssl.c:1056)"))': /simple/red-discordbot/
  Could not fetch URL https://pypi.org/simple/red-discordbot/: There was a problem confirming the ssl certificate: HTTPS
ConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/red-discordbot/ (Caused by SSLError(SS
LCertVerificationError(1, "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: format error in certificate's not
Before field (_ssl.c:1056)"))) - skipping
  ERROR: Could not find a version that satisfies the requirement Red-DiscordBot (from versions: none)
ERROR: No matching distribution found for Red-DiscordBot

[chrahunt]

First off, sorry for any frustration this has caused. I've been there, and it sucks. There are a lot of moving parts to debug and understand in issues like these, and it doesn't help that some of the information on how pip behaves here isn't documented.

The current pip behavior is described in #6720 (comment). To summarize, here is the precedence for CA certificates (highest to lowest, where a higher one completely overrides the ones below):

1. path provided to --cert
2. Any cert setting in a pip configuration file (overridden as described in the docs)
3. REQUESTS_CA_BUNDLE environment variable
4. CURL_CA_BUNDLE environment variable
5. output of pip._vendor.certifi.where(), by default this is the bundled root CA certs from certifi but it may be overridden by your distribution

pip does not use:

1. ssl.get_default_verify_paths()
2. SSL_CERT_FILE

If you are facing issues like those described above, then try the following:

1. If you know that your repository uses a certificate that will need a custom CA certificate to verify, then provide that CA certificate to pip using the --cert option.
2. If it works, then set the cert option in your user-specific configuration file to point to that file.
3. If you are an administrator and you don't want every user to have to set this, then set it globally using the global pip configuration file relevant to your OS.

I hope there is enough information here to help most of the problems mentioned.

We want to make sure everyone gets help, so let's proceed like this: I will close this issue. Anyone still having problems, please create a separate issue so we can dig into your specific situation. Try to provide as much information as possible, at least what is mentioned in the Bug Report template. With that we should be able to help you more effectively.