Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't unconditionally prepend "" to a venv PEX's sys.path #1983

Closed
thejcannon opened this issue Nov 17, 2022 · 2 comments · Fixed by #1984
Closed

Don't unconditionally prepend "" to a venv PEX's sys.path #1983

thejcannon opened this issue Nov 17, 2022 · 2 comments · Fixed by #1984
Labels

Comments

@thejcannon
Copy link
Contributor

#1832 (comment) captures the location of the bug.

sys.path behavior is documented https://docs.python.org/3/library/sys.html#sys.path which shows the prepending to sys.path is dependent on how python is invoked.

And to further prove the point:

$ cat foo.py
print(__import__("sys").path)

$ python -m venv venv
...

$ venv/bin/python foo.py
['/home/josh/work/pants', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/josh/work/pants/venv/lib/python3.8/site-packages']

$ venv/bin/python -m foo
['/home/josh/work/pants', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/josh/work/pants/venv/lib/python3.8/site-packages']

$ MYCODE=$(cat foo.py) &&  venv/bin/python -c "$MYCODE"
['', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/josh/work/pants/venv/lib/python3.8/site-packages']
@benjyw
Copy link
Collaborator

benjyw commented Nov 17, 2022

Related: pantsbuild/pants#17531

@jsirois
Copy link
Member

jsirois commented Nov 18, 2022

@thejcannon, your example shell session is useful and is also true all the way back to Python 2.7 - almost:

$ cat foo.py
import sys; print("\n".join(sys.path))
$ python2.7 foo.py
/home/jsirois/dev/pantsbuild/jsirois-pex
/usr/lib/python2.7
/usr/lib/python2.7/plat-x86_64-linux-gnu
/usr/lib/python2.7/lib-tk
/usr/lib/python2.7/lib-old
/usr/lib/python2.7/lib-dynload
/usr/local/lib/python2.7/dist-packages
/usr/lib/python2.7/dist-packages
$ python2.7 -m foo

/usr/lib/python2.7
/usr/lib/python2.7/plat-x86_64-linux-gnu
/usr/lib/python2.7/lib-tk
/usr/lib/python2.7/lib-old
/usr/lib/python2.7/lib-dynload
/usr/local/lib/python2.7/dist-packages
/usr/lib/python2.7/dist-packages
$ python2.7 -c "$(cat foo.py)"

/usr/lib/python2.7
/usr/lib/python2.7/plat-x86_64-linux-gnu
/usr/lib/python2.7/lib-tk
/usr/lib/python2.7/lib-old
/usr/lib/python2.7/lib-dynload
/usr/local/lib/python2.7/dist-packages
/usr/lib/python2.7/dist-packages

In other words, all three of those modes insert CWD at the head of sys.path with sometimes CWD spelled as an absolute path and sometimes as ""; so when PEX acts as an interpreter (via either forcing with PEX_INTERPRETER=1 or else when it has no entry point defined), it should do the same. Currently zipapp mode does this correctly just here:
https://github.com/pantsbuild/pex/blob/67fc7e05315fc347bcf269712814f160c6035efe/pex/pex.py#L627-L631

But venv mode does it unconditionally as you highlight here:
https://github.com/pantsbuild/pex/blob/67fc7e05315fc347bcf269712814f160c6035efe/pex/venv/pex.py#L443-L444

@jsirois jsirois added the bug label Nov 18, 2022
thejcannon added a commit that referenced this issue Nov 21, 2022
…ter mode (#1984)

Fixes #1983.

Note that this PR doesn't revert the code to use the prior code (`sys.path.extend(os.environ.get("PEX_EXTRA_SYS_PATH", "").split(os.pathsep))`) as that might still add `""` when `PEX_EXTRA_SYS_PATH` is not in the environment.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants