Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use configured CA certificate when downloading packages #3349

Conversation

intgr
Copy link
Contributor

@intgr intgr commented Nov 10, 2020

Pull Request Check List

Resolves: possibly #3110 #2593 #1012 (I haven't looked into the nuances)

  • Added tests for changed code.
  • Updated documentation for changed code.

It seems that #1325 tried to add support to supply custom CA certificates for custom 3rd party repositories (via poetry config certificates.$repo.cert).

But the implementation was incomplete: poetry lock properly uses the correct CA certs for custom repository access, but poetry install still fails with a cert error because the package download code path does not use the configured CA certificates.

This issue surfaced for us when upgrading Poetry from 1.1.3 to 1.1.4, the minor release had change #3251 that gave our custom repository priority (it was ignored previously); the root cause was challenging to track down.

This PR is a hack to make artifact downloads use configured CA file too. I wanted to submit this first to get feedback whether I'm on the right track.

Error that occurs without this patch:

% poetry install
Installing dependencies from lock file

Package operations: 1 install, 0 updates, 0 removals

  • Installing django (3.1.3): Pending...
Retrying HTTP request in 0.5 seconds.
Retrying HTTP request in 1.0 seconds.
Retrying HTTP request in 1.5 seconds.
Retrying HTTP request in 2.0 seconds.
  • Installing django (3.1.3): Failed

  SSLError

  HTTPSConnectionPool(host='***', port=443): Max retries exceeded with url: /api/pypi/pypi/packages/packages/7f/17/16267e782a30ea2ce08a9a452c1db285afb0ff226cfe3753f484d3d65662/Django-3.1.3-py3-none-any.whl (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1122)')))

  at /usr/local/Cellar/poetry/1.1.4/libexec/vendor/lib/python3.9/site-packages/requests/adapters.py:514 in send
      510│                 raise ProxyError(e, request=request)
      511│ 
      512│             if isinstance(e.reason, _SSLError):
      513│                 # This branch is for urllib3 v1.22 and later.
    → 514│                 raise SSLError(e, request=request)
      515│ 
      516│             raise ConnectionError(e, request=request)
      517│ 
      518│         except ClosedPoolError as e:

@intgr intgr changed the title HACK: Check repository certificate when downloading packages HACK: Check configured certificate when downloading packages Nov 10, 2020
@intgr intgr changed the title HACK: Check configured certificate when downloading packages HACK: Use configured CA certificate when downloading packages Nov 10, 2020
return repository

def _get_links(self, package): # type: (Package) -> List[Link]
repository = self._get_repository(package)

links = repository.find_links_for_package(package)
Copy link
Contributor Author

@intgr intgr Nov 10, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the most appropriate way to pass along the repository attribute to Executor._download_link? Should I add a repository attribute to the Link class? 🤔

link.url,
stream=True,
io=self._sections.get(id(operation), self._io),
verify=getattr(repository, 'cert'),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only LegacyRepository has the cert attribute, should I have an isinstance() check or is getattr() good enough?

@intgr
Copy link
Contributor Author

intgr commented Jan 8, 2021

Ping @finswimmer @stephsamson any feedback on whether I'm on the right track with this?

@Spectre5
Copy link

I get this same error, it would be great to get this fixed! That said, here is my work-around that is working, but it just seems unnecessary since this is the same pem file that I already provided to poetry via poetry config certificates.$repo.cert command. A simple fix might be to just put this cert value into os.environ at the start of the poetry?

CURL_CA_BUNDLE="path/to/cerl/ca-bundle.pem" poetry install –vvv

@intgr intgr changed the title HACK: Use configured CA certificate when downloading packages Use configured CA certificate when downloading packages Apr 26, 2021
@neersighted neersighted self-assigned this Nov 11, 2021
@neersighted
Copy link
Member

This is obsolete as of the merge of #5719.

@neersighted neersighted closed this Jun 4, 2022
@gazpachoking
Copy link

I don't believe this is obsolete. I'm still having the same problem in 1.2b3. If I specify a custom certificate for a private repository (or turn off certificate verification completely for that repository) everything goes fine util it tries to download the wheel. At that point I get a certificate verify failed error. It seems that the certificate.REPO.cert configuration setting applies when communicating with the api for a given repo, but not when downloading the wheels from that repo.

@neersighted
Copy link
Member

This PR still conflicts/won't resolve that -- please open an issue with so we can take a look.

Copy link

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants