wxpython: system python doesn't pick up .pth in /usr/local/lib

[tdsmith]

import wx doesn't work for system Python users after brew installing wxpython.

• Homebrew advises users of system Python to add /usr/local/lib/python2.7/site-packages to ~/Library/Python/2.7/lib/python/site-packages/homebrew.pth so that brewed packages end up in sys.path. This is almost the same as setting PYTHONPATH.[1]
• wxpython installs /usr/local/lib/python2.7/site-packages/wx.pth, which adds /usr/local/lib/python2.7/site-packages/wx-3.0-osx_cocoa to the search path. import wx doesn't work unless this .pth file is processed.
• .pth files are not processed except in special paths. Paths added with .pth files or with PYTHONPATH are not special enough. Therefore, import wx doesn't work out-of-the box with system Python with the suggestions that Homebrew provides.
• Paths can be made special by calling site.addsitedir(new_special_path).
• Code in ~/Library/Python/2.7/lib/python/site-packages/usercustomize.py is executed at the beginning of every Python session; the intended purpose is to munge paths and call functions from site.*.

Some options:

1. wxpython could install wx directly into site-packages without the wx-3.0-osx_cocoa path element Other packages require this behavior, and overriding it would be at odds with the wxpython versioning system.
2. wxpython could install a wx symlink into site-packages pointing to wx-3.0-osx_cocoa/wx This is at odds with the spirit of the wxpython versioning system.
3. Homebrew could add a caveat to this formula if brewed Python isn't chosen asking users to run:
   echo "import site; site.addsitedir('#{HOMEBREW_PREFIX}/lib/python2.7/site-packages')" >> ~/Library/Python/2.7/lib/python/site-packages/usercustomize.py
4. Homebrew could globally change its advice about .pth files to site.addsitedir as above (since the homebrew.pth file is unnecessary after the addsitedir call)

4 would work and in some sense would be the most durable but I don't know if other packages try to install .pth files.

Previously mentioned: #27112 (comment)

[1] PYTHONPATH is different from .pth files in that PYTHONPATH is prepended to sys.path and paths discovered in .pth files are appended. This is, just incidentally, bad for homebrew-python, where we would like to avoid importing system numpy, and need to come first in sys.path. addsitedir also appends, so we would still need to either a) set PYTHONPATH, or b) forcibly insert homebrew site-packages early in sys.path inside (user|site)customize.py.

[tdsmith]

Just noticed this in the site package docs:

Lines starting with import (followed by space or tab) are executed.

So, for option 3, Homebrew could still recommend an echo into homebrew.pth instead of creating a usercustomize.py.

[dakcarto]

@tdsmith Just ran into this when needing to have a formula import wx during its install.

I agree with the need to have better support for .pth files, and think your following solution is a good starting point:

echo "import site; site.addsitedir('#{HOMEBREW_PREFIX}/lib/python2.7/site-packages')" >> ~/Library/Python/2.7/lib/python/site-packages/homebrew.pth

Still, more needs done for supporting 'keg-only' packages. So, the above #{HOMEBREW_PREFIX}/lib would need replaced with #{Formula["myformula"].opt_lib} for such formulae.

For building within a formula, it is better to adjust PYTHONPATH, as that will be picked up by the package's build process, e.g. via system "python". This is how I got import wx to work within the build process: I compared sys.path before and after issuing site.addsitedir to generate the appropriate path(s) to prepend to PYTHONPATH:

def install
  ...
  wx_paths = formula_site_packages "wxpython"
  ENV.prepend("PYTHONPATH", wx_paths, File::PATH_SEPARATOR) if wx_paths
  ...
end

def formula_site_packages f
  `python -c "import os, sys, site; sp1 = list(sys.path); site.addsitedir('#{Formula[f].opt_lib}/python2.7/site-packages'); print(os.pathsep.join([x for x in sys.path if x not in sp1]))"`.strip
end

That works regardless of whether using OS X or Homebrew's Python, though a bit of overkill for Homebrew's. Maybe a site_packages function can be added to the Formula class to help with this.

Lastly, the current python caveats is broken, in that the user is recommended to overwrite, instead of append to, the homebrew.pth file.

s += "  echo '#{site_packages}' > #{file}" # should be >>

So that needs fixed as well, but then IMO the import site;... methodology, with appending, should be used instead anyway.

[1] PYTHONPATH is different from .pth files in that PYTHONPATH is prepended to sys.path and paths discovered in .pth files are appended. This is, just incidentally, bad for homebrew-python, where we would like to avoid importing system numpy, and need to come first in sys.path. addsitedir also appends, so we would still need to either a) set PYTHONPATH, or b) forcibly insert homebrew site-packages early in sys.path inside (user|site)customize.py.

This was brought up in PR #28597 and looks like it may have been forgotten (even by myself).

/cc @samueljohn Any ideas here?

[MikeMcQuaid]

@dakcarto Please file a separate issue/PR for things that are broken. This whole situation is a mess, unfortunately, and I'd love us to find a final solution for this.

[chrisnorman7]

Hi,
I am just trying to use wxPython again after a frew install of my mac, and it's not working at all now. I'm wondering what steps I need to take after:

(installing XCode and dev tools, agreeing to the license etc)...
brew install wxmac

Everything seemed to work, although it did say something about I might have problems referencing it (which seems to be relevant), or linking other packages to it.

I tried using the echo command above, but still nothing from Python. I can't import wx, or wxversion.

What are my options? The Cocoa version of wx from the wxpython site has a broken .pkg in it's dmg, so I can't use that one, and I don't want to use Carbon, since it doesn't provide the accessibility I rely on.

Cheers.

[tdsmith]

@chrisnorman7: Ignore Homebrew's messages at your peril. :) Although they wouldn't have helped here, yet! You can see the caveats that were printed after installation with brew info wxpython. This approach should have worked, though. You brew installed wxpython as well, right? Did you replace ${HOMEBREW_PREFIX} with (e.g.) /usr/local in the echo command you ran? Try that first. Can you provide the verbatim command you ran (or the content of all files in ~/Library/Python/2.7/lib/python/site-packages), the output of brew config, the output of python -c "import sys; print sys.path", and cat /usr/local/lib/python2.7/site-packages/wx.pth?

[chrisnorman7]

No worries,
I fixed it by doing:
brew install python
(that installed wxmac too)
brew install python

Now I can wx away.

Cheers,

On 17/09/2014 15:49, tim smith wrote:

@chrisnorman7 https://github.com/chrisnorman7: Ignore Homebrew's
messages at your peril. :) You can see the caveats that were printed
after installation with |brew info wxpython|. This approach should
have worked, though. You brew installed wxpython as well, right? Did
you replace ${HOMEBREW_PREFIX} with (e.g.) /usr/local in the echo
command you ran? Try that first. Can you provide the verbatim command
you ran (or the content of all files in
~/Library/Python/2.7/lib/python/site-packages), the output of |brew
config|, the output of |python -c "import sys; print sys.path"|, and
|cat /usr/local/lib/python2.7/site-packages/wx.pth|?

—
Reply to this email directly or view it on GitHub
#30844 (comment).