Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Segfault on MacOS caused by LibGMT.log_to_file #164

Closed
chandrex opened this issue Apr 20, 2018 · 16 comments
Closed

Segfault on MacOS caused by LibGMT.log_to_file #164

chandrex opened this issue Apr 20, 2018 · 16 comments
Labels
bug Something isn't working

Comments

@chandrex
Copy link

chandrex commented Apr 20, 2018

screen shot 2018-04-20 at 16 54 44

Hi, I've been trying to install gmt-python for the past day and a half with no avail. I am on macOS 10.13.4, Python 3.6.5 (installed with Anaconda 3). I've been following all of the installation instruction, and everything seems fine except when I get to the very end, with the gmt.test().

I am in the conda environment (using source activate gmt-python), and trying this in the Python interpreter. This is what happens:

(gmt-python) Chandras-MacBook-Pro:~ chandra$ python
Python 3.6.5 | packaged by conda-forge | (default, Apr 6 2018, 13:44:09)
[GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import gmt
gmt.test()

returns

============================= test session starts ==============================
platform darwin -- Python 3.6.5, pytest-3.5.0, py-1.5.3, pluggy-0.6.0 -- /anaconda3/envs/gmt-python/bin/python
cachedir: .pytest_cache
Matplotlib: 2.2.2
Freetype: 2.8.1
rootdir: /Users/chandra, inifile:
plugins: mpl-0.9
collected 126 items
gmt/base_plotting.py::gmt.base_plotting.BasePlotting._preprocess <- ../../anaconda3/envs/gmt-python/lib/python3.6/site-packages/gmt/base_plotting.py PASSED [ 0%]
gmt/figure.py::gmt.figure.Figure <- ../../anaconda3/envs/gmt-python/lib/python3.6/site-packages/gmt/figure.py FAILED [ 1%]
gmt/clib/core.py::gmt.clib.core.LibGMT <- ../../anaconda3/envs/gmt-python/lib/python3.6/site-packages/gmt/clib/core.py PASSED [ 2%]
gmt/clib/core.py::gmt.clib.core.LibGMT._check_dtype_and_dim <- ../../anaconda3/envs/gmt-python/lib/python3.6/site-packages/gmt/clib/core.py PASSED [ 3%]
gmt/clib/core.py::gmt.clib.core.LibGMT.extract_region <- ../../anaconda3/envs/gmt-python/lib/python3.6/site-packages/gmt/clib/core.py gmt-python-session [ERROR]: Long input record (236 bytes) was truncated to first 126 bytes!
FAILED [ 3%]
gmt/clib/core.py::gmt.clib.core.LibGMT.grid_to_vfile <- ../../anaconda3/envs/gmt-python/lib/python3.6/site-packages/gmt/clib/core.py FAILED [ 4%]
gmt/clib/core.py::gmt.clib.core.LibGMT.log_to_file <- ../../anaconda3/envs/gmt-python/lib/python3.6/site-packages/gmt/clib/core.py Segmentation fault: 11

I feel like this might be because of a bash profile issue. I am not sure how to set up a .bashrc profile properly on the Mac (sorry!), but my .bash_profile looks like this (note the 'added by...' is commented out):

added by Anaconda3 5.1.0 installer
export PATH="/anaconda3/bin:$PATH"
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/anaconda3/envs/gmt-python/lib

Any help would be much appreciated! 😄
ps. let me know if any clarifications are needed, sorry first time writing an issue in!
pps. attached a screenshot of terminal, just to be super clear!

@leouieda
Copy link
Member

@chandrex thank you reporting this and sorry for all the trouble! I really appreciate the amount of effort you put in to the issue. That really helps! Welcome to the world of open-source! I hope you'll like it and consider becoming involved 😀

OK, this is a bad crash. It normally shouldn't happen but it's really tricky dealing with C code from Python. One possibility is that Python is finding the wrong version of GMT. Do you have GMT installed from another source (homebrew, fink, etc)?

Could you try removing the LD_LIBRARY_PATH line? That's not usually necessary unless you can't get it to work otherwise. Sorry about this, it should be clearer in the docs and I opened #165 to remind me to deal with this.

If that doesn't work, then can you please post the output of conda list? That way I can try to see if everything is being installed in the correct version.

Again, thanks for detailed issue!

@chandrex
Copy link
Author

Hiya,

Yes I have GMT installed already (5.4.2), which I installed with homebrew. It won't work with or without the LD_LIBRARY_PATH (I tried!), but I will remove it anyhow.

Here is my conda list

screen shot 2018-04-23 at 10 07 34

screen shot 2018-04-23 at 10 07 46

screen shot 2018-04-23 at 10 07 59

screen shot 2018-04-23 at 10 08 21

Thanks for your help 😁

@leouieda
Copy link
Member

OK, here are a few things you can try:

  1. I noticed that your pandas and xarray were installed by pip. Which indicates that they were pulled in when you installed gmt-python. Also, your numpy is a release candidate. We noticed these problems before so we updated the install instructions to match. Could you dele the current environment (conda env remove -n gmt-python) and follow these steps?
  2. If that doesn't work, then try setting LD_LIBRARY_PATH using export LD_LIBRARY_PATH=$HOME/anaconda3/envs/gmt-python/lib:$LD_LIBRARY_PATH . This gives anaconda a higher priority.

I'm working on a better way that we can diagnose these problems (#167).

Again, thanks for reporting this!

@leouieda
Copy link
Member

@chandrex just to let you know that your issue contributed several changes to the project: #168 #173 #175 #176 and #177

#177 introduces a new way to set the path to the GMT shared library. Once, that's incorporated (it will say "Merged" on the link) please try deleting your conda environment and following the new instructions that will appear in http://www.gmtpython.xyz/install.html

Again, thank you for helping us make things more robust!

@leouieda
Copy link
Member

@chandrex please let me know if the new changes work for you. If not we can at least better diagnose the problem.

@chandrex
Copy link
Author

Hi @leouieda, alas I have tried everything you suggested and I am still getting the same problem at the final step when I do the gmt.test(), with the failures and segmentation fault.
Steps I took apart from your suggestions:
• Set up a proper .bashrc (my mac did not have one previously)
• I thought it could have been a problem with where anaconda was installed, so uninstalled/reinstalled in where $HOME is (it was not there previously)
Maybe I should uninstall my GMT 5 version and see if I can get it working without it? What do you think?

@leouieda
Copy link
Member

:( sorry to hear. You shouldn't have uninstall GMT5. Could you please post the new output of gmt.test()? This should let us diagnose the problem now.

@chandrex
Copy link
Author

chandrex commented May 1, 2018

Hi @leouieda here is the screenshot:

image

And in writing:

import gmt
gmt.test()

=========================== Currently loaded libgmt ============================
binary dir: /Users/chandra/anaconda3/envs/gmt-python/bin
cores: 4
grid layout: rows
image layout:
library path: /Users/chandra/anaconda3/envs/gmt-python/lib/libgmt.dylib
padding: 2
plugin dir: /Users/chandra/anaconda3/envs/gmt-python/lib/gmt/plugins
share dir: /Users/chandra/anaconda3/envs/gmt-python/share/gmt
version: 6.0.0
============================= test session starts ==============================
platform darwin -- Python 3.6.5, pytest-3.5.0, py-1.5.3, pluggy-0.6.0 -- /Users/chandra/anaconda3/envs/gmt-python/bin/python
cachedir: .pytest_cache
Matplotlib: 2.2.2
Freetype: 2.8.1
rootdir: /Users/chandra, inifile:
plugins: mpl-0.9
collected 132 items

anaconda3/envs/gmt-python/lib/python3.6/site-packages/gmt/base_plotting.py::gmt.base_plotting.BasePlotting._preprocess PASSED [ 0%]
anaconda3/envs/gmt-python/lib/python3.6/site-packages/gmt/figure.py::gmt.figure.Figure FAILED [ 1%]
anaconda3/envs/gmt-python/lib/python3.6/site-packages/gmt/clib/core.py::gmt.clib.core.LibGMT PASSED [ 2%]
anaconda3/envs/gmt-python/lib/python3.6/site-packages/gmt/clib/core.py::gmt.clib.core.LibGMT._check_dtype_and_dim PASSED [ 3%]
anaconda3/envs/gmt-python/lib/python3.6/site-packages/gmt/clib/core.py::gmt.clib.core.LibGMT.extract_region gmt-python-session [ERROR]: Long input record (236 bytes) was truncated to first 126 bytes!
FAILED [ 3%]
anaconda3/envs/gmt-python/lib/python3.6/site-packages/gmt/clib/core.py::gmt.clib.core.LibGMT.grid_to_vfile PASSED [ 4%]
anaconda3/envs/gmt-python/lib/python3.6/site-packages/gmt/clib/core.py::gmt.clib.core.LibGMT.log_to_file Segmentation fault: 11

@mikehagerty
Copy link

Hi.
I just followed your wiki download/install instructions on mac os 10.10.5 and I get the exact same error:

gmt.test()
...
gmt/clib/core.py::gmt.clib.core.LibGMT.get_libgmt_func PASSED [ 4%]
gmt/clib/core.py::gmt.clib.core.LibGMT.grid_to_vfile PASSED [ 5%]
gmt/clib/core.py::gmt.clib.core.LibGMT.log_to_file Segmentation fault: 11

My dependencies (e.g., xarray, pandas) were all installed with conda.
Everything looks good (no complaints) except for the seg fault.

Please let me know if anyone makes any progress on solving this.

Cheers!
-Mike

@leouieda
Copy link
Member

Hi @mikehagerty and @chandrex this is a very strange failure. I'll try to get my hands on a Mac to investigate.

@mikehagerty out of curiosity, do you also have GMT5 installed?

leouieda added a commit that referenced this issue May 11, 2018
There seems to be a segfault problem (#164) that doesn't happen on
Linux. I'm re-enabling the Mac build to check if it happens on Travis as
well.
@leouieda
Copy link
Member

OK, I managed to replicate the issue using Travis CI on #180. It's not a problem with your installation or with the pre-installed GMT5. I'll investigate further to try to get to the bottom of this.

Thanks for bringing this to my attention! It's hard to track these issues when I'm not using a Mac.

@leouieda leouieda changed the title Issues installing - failures and segmentation fault Segfault on MacOS during LibGMT.log_to_file May 11, 2018
@leouieda leouieda added the bug Something isn't working label May 11, 2018
@mikehagerty
Copy link

mikehagerty commented May 21, 2018 via email

@leouieda
Copy link
Member

@mikehagerty sorry, haven't had much time to look into to yet.

It seems like you don't have pytest-mpl installed. Can you check, please? A lot of the tests compare generated plots using the matplotlib pytest plugin (hurray for duck typing!).

@leouieda leouieda changed the title Segfault on MacOS during LibGMT.log_to_file Segfault on MacOS during LibGMT.extract_region May 31, 2018
@mikehagerty
Copy link

mikehagerty commented Jun 1, 2018 via email

@leouieda
Copy link
Member

leouieda commented Jun 1, 2018

@mikehagerty thanks for the info!

I suspect that it's the log file actually but it's a bit tough to debug when TravisCI is not cooperating. I haven't found a good fix for that yet so I might just disable it for now. This is used to get the error messages printed by GMT. It's helpful but not vital for now.

leouieda added a commit that referenced this issue Jun 14, 2018
There seems to be a segfault problem (#164) that doesn't happen on
Linux. I'm re-enabling the Mac build to check if it happens on Travis as
well.
@leouieda leouieda changed the title Segfault on MacOS during LibGMT.extract_region Segfault on MacOS caused by LibGMT.log_to_file Jun 16, 2018
@leouieda
Copy link
Member

Good news everyone @chandrex @mikehagerty! 🎉 🍰 🎈

I finally managed to fix this issue in #188! It was a long struggle and I still have no idea why the fault was happening. But I replaced the code that was causing the problem with something better and the fault went away. Things should be working smoothly on Mac now.

Could you please give it a try and report back? Thank you for all the information you all provided and sorry for long delay. 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants