Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pex multiplatform resolution is broken for non-host-available platforms #694

Closed
jsirois opened this issue Mar 29, 2019 · 1 comment · Fixed by #788
Closed

pex multiplatform resolution is broken for non-host-available platforms #694

jsirois opened this issue Mar 29, 2019 · 1 comment · Fixed by #788
Assignees
@jsirois
Labels
bug resolver

Comments

@jsirois
Copy link
Member

jsirois commented Mar 29, 2019

In particular Platform.get_supported delegates much of it's work to pep425tags / glibc which both consult a local interpreter to answer questions like:

  1. Does the interpreter support manylinux?
  2. Does the interpreter have abi3 support?

Concretely, if a user runs pex on OSX and supplies a --platform=linux-x86_64-cp-37-m, there is no way for pex to determine the answers to those questions for the python 3.7 the pex will eventually run on. The problem is worse for non-extended platforms; eg: --platform=linux-x86_64. In those cases we don't know if the eventual linux runtime interpreter will be d, m, mu, dmu, etc... and currently accept wheels for any of these!

We need ways to do one or all of:

  1. Let the user specify the answers to the 1st two questions - perhaps with new pex API/CLI options.
  2. Fail if building a pex encounters situations where these unanswered questions are important - namely pex resolves platform-specific wheels.
@jsirois
Copy link
Member Author

jsirois commented Mar 29, 2019

Fixing #539 brough this issue to light. #658 also touches on this area of broken multiplatform resolution.

@jsirois jsirois mentioned this issue Oct 27, 2019
Merged
@jsirois jsirois mentioned this issue Nov 14, 2019
Merged
@jsirois jsirois self-assigned this Nov 14, 2019
jsirois added a commit that referenced this issue Nov 14, 2019
@jsirois
Use pip for resolving and building distributions. (#788)
7509a89 
This changes Pex to vendor pip and defer all resolution and building to it.

As part of this change some ambiguous and / or broken features were changed and APIs adjusted:
+ The egg distribution format is no longer supported.
+ The deprecated `--interpreter-cache-dir` CLI option was removed.
+ The `--cache-ttl` CLI option and `cache_ttl` resolver API argument were removed.
+ The resolver API replaced `fetchers` with a list of `indexes` and a list of `find_links` repos.
+ The resolver API removed (http) `context` which is now automatically handled.
+ The resolver API removed `precedence` which is now pip default precedence - wheels when available and not ruled out via the `--no-wheel` CLI option or `use_wheel=False` API argument.
+ The `--platform` CLI option and `platform` resolver API argument now must be full platform strings that include platform, implementation, version and abi; e.g.: `--platform=macosx-10.13-x86_64-cp-36-m`.
+ The `--manylinux` CLI option and `use_manylinux` resolver API argument were removed. Instead, to resolve manylinux wheels for a foreign platform, specify the manylinux platform to use with an explicit `--platform` CLI flag or `platform` resolver API argument; e.g.: `--platform=manylinux2010-x86_64-cp-36-m`.

Fixes #781

Additionally:
Fixes #771
Fixes #763
Fixes #761
Fixes #735 
Fixes #694 
Fixes #660
Fixes #658 
Fixes #642
Fixes #641
Fixes #628
Fixes #620
Fixes #614
Fixes #611
Fixes #608
Fixes #439
Fixes #415
Fixes #387
Fixes #315
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jsirois jsirois

Labels
bug resolver
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Use pip for resolving and building distributions. jsirois/pex
1 participant
@jsirois