pdsh 2.32 segfault on OS X 10.11

[ilovezfs]

I'm getting a segfault with the 2.32 release on OS X 10.11:

iMac-TMP:~ joe$ sudo lldb /usr/local/Cellar/pdsh/2.32/bin/pdsh 
Password:
(lldb) target create "/usr/local/Cellar/pdsh/2.32/bin/pdsh"
Current executable set to '/usr/local/Cellar/pdsh/2.32/bin/pdsh' (x86_64).
(lldb) r -V
Process 76749 launched: '/usr/local/Cellar/pdsh/2.32/bin/pdsh' (x86_64)
pdsh@iMac-TMP: Unable to determine ownership of pdsh binary: Bad address
Process 76749 stopped
* thread #1: tid = 0x8af88a, 0x000000010001bb92 pdsh`_next_tok(sep=",", str=0x00007fff5fbffac0) + 34 at split.c:51, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xa00000020)
    frame #0: 0x000000010001bb92 pdsh`_next_tok(sep=",", str=0x00007fff5fbffac0) + 34 at split.c:51
   48  	    int level = 0;
   49  	
   50  	    /* push str past any leading separators */
-> 51  	    while (**str != '\0' && strchr(sep, **str) != NULL)
   52  	        (*str)++;
   53  	
   54  	    if (**str == '\0')
(lldb) bt
* thread #1: tid = 0x8af88a, 0x000000010001bb92 pdsh`_next_tok(sep=",", str=0x00007fff5fbffac0) + 34 at split.c:51, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xa00000020)
  * frame #0: 0x000000010001bb92 pdsh`_next_tok(sep=",", str=0x00007fff5fbffac0) + 34 at split.c:51
    frame #1: 0x000000010001bafb pdsh`list_split(sep=",", str="") + 75 at split.c:96
    frame #2: 0x0000000100005e57 pdsh`_mod_initialize_modules_by_name(names="", m=0x0000000100800600) + 55 at mod.c:348
    frame #3: 0x0000000100005a5e pdsh`mod_load_modules(dir="/usr/local/Cellar/pdsh/2.32/lib/pdsh", opt=0x00007fff5fbffb80) + 78 at mod.c:367
    frame #4: 0x0000000100001b3c pdsh`main(argc=2, argv=0x00007fff5fbffc98) + 236 at main.c:112
    frame #5: 0x00007fff8eea35ad libdyld.dylib`start + 1
    frame #6: 0x00007fff8eea35ad libdyld.dylib`start + 1
(lldb) 

[grondo]

Thanks, what was the pdsh cmdline that caused the segfault?

[ilovezfs]

pdsh -V

[ilovezfs]

see the (lldb) r -V above 🙂

[grondo]

Oh, got it, I didn't put it together that you were running under a debugger 😅

In order to update to latest autotools, I had dropped support for libltdl in favor of straight dlopen(3) -- I'm guessing this has something to do with the new OSX failure. I don't know anything about loading DSOs on OSX -- apparently dlopen is supported but I'm not sure if there is some extra magic required... I'll have to find a Mac on which to test.

[ilovezfs]

@grondo feel free to abuse our CI if you want.

url "https://github.com/grondo/pdsh.git", :revision => "943bf12f622a1165058ee8240044980b4594646a"
version "2.32.0.1"

to test an arbitrary commit as if it's a real version.

[grondo]

Cool! Thanks, can you paste quick instructions?

Travis-CI also has OSX images for testing and I should have added that to the automated CI earlier. The only problem is it seems to take many minutes for an OSX image to be available to run the tests.

[ilovezfs]

@grondo here's an example where I was testing an s3fs fix at upstream's request: https://github.com/Homebrew/homebrew-core/pull/14006/files

http://docs.brew.sh/How-To-Open-a-Homebrew-Pull-Request.html

[grondo]

Thank you!

[grondo]

Wow, using Travis as an OSX development environment was painful.

I think I did find the source of this crash though (I hope). The problem was that different source files in pdsh were seeing different definitions of the bool type with different sizes, causing bad offsets into structs, most notably the pdsh options struct opt_t.

See

https://github.com/grondo/pdsh/tree/osx-fixes

This branch additionally has some warnings from clang on OSX fixed, and adds an OSX build to Travis-CI for pdsh.

There is one remaining problem, however. If I add genders to the Travis build for OSX then crashes start again whenever the genders module is used. I was not successful debugging this iteratively via Travis, so if anyone gets a good backtrace, that would help.