-
-
Notifications
You must be signed in to change notification settings - Fork 11.3k
cmake find_package(PythonLibs) broken with brewed python on 10.9 #25118
Comments
I see it found What are the dependencies of your formula? I notice it's not picking up the Homebrewed Python framework, even though it should be in the search path. |
Yes unfortunately:
I'm not developing a formula. I am building a CMake project using find_package(PythonLibs). It's building, but brewed python is not happy when /usr/lib/libpython2.7.dylib is importet (segfault 11). PYTHON_FRAMEWORK_INCLUDES is set inside FindPythonLibs.cmake, but only under certain conditions. I don't actually use it directly. I'm unsure if this is related to the wrong lib and include paths, or if it is a seperate issue. Python_FRAMEWORKS is set inside CMakeFindFrameworks.cmake, but it only searches a hardcoded list of locations and as mentioned, |
I'm not very familiar with cmake, but is it possible to override |
That doesn't change anything. Looking inside FindPythonLibs.cmake I don't think its meant to be used that way. I actually think that FindPythonLibs.cmake determines the lib and include paths for python idependent of the frameworks (variables Python_FRAMEWORKS and PYTHON_FRAMEWORK_INCLUDES). I'm not sure though. |
@NikolausDemmel We ran into the same issue with the CMake build for QGIS. While an upstream fix is ideal, as a workaround I added a CMake option QGIS If you use that module and find any issues with or fixes for it, please post to the QGIS issue tracker. |
@dakcarto Thanks for that info and the linked cmake file. I agree that using the found python interpreter from find_package(PythonInterp) and then using Edit: |
Should we raise this upsteam with CMake to see how we can change the broken FindPythonLibs.cmake? |
Yes. |
Has a ticket been opened on the cmake bug tracker for this ? |
@iMichka: No, it hasn't. After researching a bunch of existing mailinglist discussions and bug tracker issues about related problems and digging into the code it is not clear to me what the solution should be. I did a local hack along the lines of what @dakcarto suggested and then dropped the ball on the issue, sorry. I should try to at least dig up the list of links to existing issues and list discussions... |
If not, you should open a ticket on cmake's bug tracker so that they know about the issue. Perhaps upstream will fix it (I know there can be some delay for this, they still have a lot of bugs open) |
This stuff won't be fixed if it's not raised upstream. |
I am experiencing a similar issue, but building a different package (libfreenect) that I want to link against the brewed Python I just installed. I was surprised to see the suggestion that this was an upstream problem. It very well may be, but before I read through this thread I was trying to determine if the formula for brewed python was intentionally not linking the libpython.dylib and header directory or if that was a mistake. My very first expectation was that I would find the symbolic links:
Other parts of the brewed Python seemed to have been linked, like the bin and share directories, but then I considered that maybe developers were meant to pass these libraries and include directories as parameters or possibly environment variables. I seem to remember the post install caveats for readline, gdbm, and/or sqlite mentioning this. Personally, I intended to always link against the brewed applications I have installed instead of the system provided versions. Maybe that is naive. I am new to the OSX platform, so maybe one of the more experienced Mac developers would like to chime in with the best practices they follow. Thank you, @NikolausDemmel, for posting the simple CMakeLists.txt example. It was instructive as I worked through my issue. I wanted to ask, or maybe point out, a few things that came up for me.
I wasn't and having your example, as well as, the reference by @dakcarto helped me out; so thank you again. I added the lines:
to the CMakeLists.txt example above (placing them immediately before the foreach-loop iterates over versions) and experimented with various options. I looked at the disutils technique suggested too. I haven't gotten to the bottom of all the issues, but it was enough to convince me that passing cmake some parameters was going to be the easiest way to achieve my goal.
which generated
I am going down this path. Hopefully, this helps you too. distutilsAt a glance it seemed that the python environment path (sys.path) might be problematic for the suggested approach (using distutils to determine the proper location of the library files). cmake optionsThere were a few options that the find_package and find_library functions took that might have helped sort out which library (system vs. brewed) should be preferred, but I did not try them out. brew --linkappsI didn't look into this option either; not really sure if it would have worked because I have never even done it. Maybe one of the Homebrew gurus can chime in here. Would it be possible, or advisable, to try and overshadow the system packages by overtaking them in their own directory space? I think I read or heard that Homebrew is strict about not going outside of the prefix directory it is configured to use- I think that is a good philosophy, so I never tried this feature. |
To clarify, aren't there really two CMake issues here?
|
In general using the |
This is a bug in cmake, or possibly OS X's Python configuration. If you Essentially, cmake can only see libs for 2.7.5, no matter what. |
True. However 2.7 is a common case, in particular it is the case if no version is specified. It turns out the only case I encountered in practice in cmake projects is not specifying a version. That works with system python, but not with brewed. It is actually more intrecate. I'm running some more tests right now. It turns out if we don't call the above test in a loop, but rather seperate invocations of cmake for each version, the result is slightly different. Then, it correctly finds I fully agree that the solution is a fix for CMake. |
@MikeMcQuaid , can you comment on the point brought up by @GitSullied above? Was it a delibertate choice to not include symlinks |
It's a deliberate choice by Python upstream. I don't like us hacking around what upstream is doing because other packages are broken. |
Finally got round to reporting this upstream: http://www.itk.org/Bug/view.php?id=14809, http://www.itk.org/Bug/view.php?id=14810 |
I am running into the same issue building KDL in OS X 10.9. CMake detects a Python 2.7.6. interpreter, but finds the system Python 2.7.5 library. Has anyone found a temporary workaround while waiting for an upstream fix? |
I too am trying to build ROS by running my brew is updated, doctored and ready to brew. |
Look how we do it in the VTK formula: https://github.com/Homebrew/homebrew-science/blob/master/vtk.rb
|
@skotturi : You can try |
I rewrote the module to be compliant with the find module's interface (input/output variables) and it should do the right thing in more cases. It's not thoroughly tested, but you can check it out here: NikolausDemmel/CMake#2 and/or install via |
Thanks @NikolausDemmel for the help. Unfortunately,
I poked around in |
Ah dang it. I got skrewed by homebrew download caching when testing this. There might be other issues since I apparently didnt test it properly. I will fix it when I get home. The "foo/bar/formulaname" syntax is for Taps. Look up brew tap in the man page or docs. It means you are installing cmake from github.com/NikolausDemmel/homebrew-devel |
You should find the file with that expected sha1 in |
@NikolausDemmel I could successfully install your new cmake after I then tried to build ROS catkin workspace again and got the same error, here's the gist: |
It looks like @NikolausDemmel's CMake tap is working now. Thanks a lot for the help! I now get past the configure step, but compiling KDL fails due to a massive number of errors (see this Gist: https://gist.github.com/mkoval/66b6af8f064ea2b5bc39). These appear to be legitimate C++ errors, so I'm not sure exactly whats going on. [EDIT] This is a known issue with building KDL in clang. See this answer on ROS answers for a workaround. I think the problem @skotturi is having is caused by:
You probably have the default cmake tapped. You should manually remove cmake, then tap the patched version:
|
@mkoval: Good to know you got past the python issues (for now). Let's not highjack this homebrew issue for ROS problems. But to get started, check out http://answers.ros.org/question/94771/building-ros-on-osx-109-solution/?answer=101176#post-id-101176. Feeld free to post any remaining issues on answers.ros.org (and maybe link them from here). @skotturi: The warning is fine and it looks like it did install the fromula from my Tap. Please edit your comment and move all the output to pastebin.com or gist.github.com, this thread is long enough as it is. And then indicate the lines that you think are informative. Just to make sure, please remove the build_isolated directory after installing the patched cmake version. |
I ran But it works! I'm now getting this error when trying to process |
This is also an issue for 10.10 beta 1. |
cmake's FindPythonLibs appears to correctly find a brewed Python if ~/Library/Frameworks contains a symbolic link to the homebrew Python framework.
Should homebrew maintain this link? |
@dbrnz Homebrew doesn't modify files in the user's home directory. |
@dbrnz: Moreover, creating such a link is a workaround only for the current python version (2.7.). The module will still break if you look for other versions. |
thks @mistydemeo |
It seems we're agreed that this is an upstream issue so I'm closing this out. |
I had some problems with the Shogun toolbox giving me segmentation fault when using import. Adding -DPYTHON_LIBRARY=#{%x(python-config --prefix).chomp}/lib/libpython2.7.dylib seems to fix this. Thankx! |
@dbrnz's solution worked for me. |
Hi,
this is possibly an upstream cmake issue, but in affects in particular users of a brewed python.
I'm on Mavericks and have brewed python 2.7.6 and brewed cmake 2.8.12.1 installed.
brew doctor
shows no errors. (https://gist.github.com/NikolausDemmel/7893695)While find_package(PythonInterp) does find the correct binary (e.g. brewed for 2.7, system for version 2.6 and 2.5), find_package(PythonLibs) always finds librarie and include paths related to the system 2.7 python (even when requesting version 2.6 or 2.5). Needless to say that this causes great grief and segmentation faults when the linked libs don't match the executing interpreter. (In my case it breaks a bunch of python libs in ROS http://wiki.ros.org/hydro/Installation/OSX/Homebrew/Source). The only configuration where this would be correct is when using the system python 2.7 (which incidentally might also be the most used configuration, maybe?).
I believe this is an issue of the cmake modules FindPythonLibs.cmake, but possibly also CMakeFindFrameworks.cmake and the way that the brewed python installs itsself (the list of locations that CMakeFindFrameworks.cmake searches does not include /usr/local/Frameworks, which is where the brewed python installs itself as a framework). The framework might be a lesser issue in my scenario at least, but it seems to invite trouble. I'm thus opening this issue here instead of upstream cmake.
The following test CMakeLists.txt and output shows the faulty behaviour illustrated above. Do other people see the same?
This is the output of
mkdir build && cmake ..
:Possibly related, 2-year-old and never resolved issue is #10393.
The text was updated successfully, but these errors were encountered: