Add --python-path to change interpreter search paths when building a PEX
#1077
Conversation
| "path to an interpreter, or specify a binary accessible on $PATH. This option " | ||
| "can be passed multiple times to create a multi-interpreter compatible pex. " | ||
| "Default: Use current interpreter.", | ||
| help=( |
There was a problem hiding this comment.
Reworded. I wanted to add a little insight into how Pex only sometimes searches for interpreters.
| 'e.g. "CPython>=2.7,<3" (A CPython interpreter with version >=2.7 AND version <3) ' | ||
| 'or "PyPy" (A pypy interpreter of any version). This argument may be repeated multiple ' | ||
| "times to OR the constraints.", | ||
| help=( |
There was a problem hiding this comment.
Only reformatted.
| ) | ||
|
|
||
| group.add_option( | ||
| "--rcfile", | ||
| dest="rc_file", | ||
| default=None, | ||
| help="An additional path to a pexrc file to read during configuration parsing. " | ||
| "Used primarily for testing.", | ||
| help=( |
There was a problem hiding this comment.
Reworded to emphasize this appends, rather than replaces. And to mention the env var.
|
Oops...my GitHub UI was messed up and showed this was green, when CI checks were still running. Let's hope this does go green..I'll fix forward otherwise. |
|
Thanks for digging into this issue. This is definitely the way to go. |
|
Yay! Should we indeed remove |
|
I'm not sure. The whole PEX_PYTHON, PEX_PYTHON_PATH, .pexrc business was a Twitter attempt in good faith that is currently a mess. There is no urgency I can see at any rate. |
|
The only urgency imo is that |
|
I think no-one except Twitter and Pants use PEX_PYTHON_PATH and IIUC Pants will now be dropping use - at least at buildtime. Twitter uses it in .pexrc files and so they (should) know what they're in for, especially since they wrote the feature. |
This is correct. Pants will only use |
…interpreter_search_paths` (#10965) ### Problem `PEX_PYTHON_PATH` is not used building a PEX, unless RC files are permitted, which we must ban. This means that `[python-setup].interpreter_search_paths` had zero impact when building a PEX, and Pex always looked at `$PATH` (controlled by `[pex].executable_search_paths`) ### Solution Use the new `--python-path` build-time flag introduced in pex-tool/pex#1077. Also fix the argv for our rule to select an interpreter, which was putting Pex flags in the `--` passthrough args section, so they were being ignored. ### Result `[python-setup].interpreter_search_paths` correctly controls the interpreter search paths both when building a PEX and running a PEX. [ci skip-rust]
Closes #1012.
Currently, PEX defaults to searching the $PATH when
--interpreter-constraintand/or--resolve-local-platformsare used. (Otherwise, it uses the current interpreter or--python).You can specify
PEX_PYTHON_PATH, but this has several issues:PEX_IGNORE_RCFILESis not set to true. Pants must set this option, meaning Pants cannot leverage this mechanism.PEX_PYTHON_PATHimpacts interpreter selection for Pex installed from pip, but not thepexrelease file #1075Instead of further using
PEX_PYTHON_PATH, we add a proper--python-pathflag. In a followup, we can remove thePEX_PYTHON_PATHmechanism.