Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python: do not add main brewed Python site-package to virtualenv #8880

Merged
merged 1 commit into from
Oct 8, 2020

Conversation

iMichka
Copy link
Member

@iMichka iMichka commented Oct 8, 2020

If added, this makes the virtualenv read the main site-package from brewed Python,
and especially makes it read our sitecustomize.py file, which will
modify the sys.executable path.

See the full discussion at:
#8873

I also took the opportunity to not include test deps, as these will
be not be installed, so the .pth file should not contains references
to site-packages from test deps.

Previous packages on Linux did already contain the wrong lines in the pth file,
for example:
cat /home/linuxbrew/.linuxbrew/Cellar/aws-google-auth/0.0.36_1/libexec/lib/python3.8/site-packages/homebrew_deps.pth
import site; site.addsitedir('/home/linuxbrew/.linuxbrew/opt/python@3.8/lib/python3.8/site-packages')
import site; site.addsitedir('/home/linuxbrew/.linuxbrew/opt/libxml2/lib/python3.8/site-packages')

This might have caused subtle bugs for some packages but not for others.

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same change?
  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your changes? Here's an example.
  • Have you successfully run brew style with your changes locally?
  • Have you successfully run brew tests with your changes locally?
  • Have you successfully run brew man locally and committed any changes?

@iMichka iMichka self-assigned this Oct 8, 2020
If added, this makes the virtualenv read the main site-package from brewed Python,
and especially makes it read our sitecustomize.py file, which will
modify the sys.executable path.

See the full discussion at:
Homebrew#8873

I also took the opportunity to not include test deps, as these will
be not be installed, so the .pth file should not contains references
to site-packages from test deps.

Previous packages on Linux did already contain the wrong lines in the pth file,
for example:
cat /home/linuxbrew/.linuxbrew/Cellar/aws-google-auth/0.0.36_1/libexec/lib/python3.8/site-packages/homebrew_deps.pth
import site; site.addsitedir('/home/linuxbrew/.linuxbrew/opt/python@3.8/lib/python3.8/site-packages')
import site; site.addsitedir('/home/linuxbrew/.linuxbrew/opt/libxml2/lib/python3.8/site-packages')

This might have caused subtle bugs for some packages but not for others.
@iMichka
Copy link
Member Author

iMichka commented Oct 8, 2020

On Mac, dep_site_packages points to: /usr/local/opt/python@3.9/lib/python3.9/site-packages (for the python 3.9 dependency), which does not exist.

On Linux, dep_site_packages points to: /home/linuxbrew/.linuxbrew/opt/python@3.9/lib/python3.9/site-packages (for the python 3.9 dependency), which does exist.

This is why there was a .pth for Linux and not for Mac. The filtering was done through the next unless dep_site_packages.exist?.

Nevertheless, I do believe this patch is a good thing and another level of security for our virtual env.

@iMichka
Copy link
Member Author

iMichka commented Oct 8, 2020

dep_site_packages should have been /usr/local/opt/python@3.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages on Mac, and not /usr/local/opt/python@3.9/lib/python3.9/site-packages. If that would have been the case, the bug would have also been there on Mac.

Actually there were 2 bugs here. Once this is merged, I might check the dep_site_packages variable on Mac but that's not urgent right now.

@iMichka iMichka requested a review from fxcoudert October 8, 2020 17:27
@iMichka iMichka merged commit 0c7a718 into Homebrew:master Oct 8, 2020
@iMichka iMichka deleted the brew-pth branch October 8, 2020 20:23
@BrewTestBot BrewTestBot added the outdated PR was locked due to age label Dec 3, 2020
@Homebrew Homebrew locked as resolved and limited conversation to collaborators Dec 3, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
outdated PR was locked due to age
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants