Mac - linking error (dyld: Symbol not found: __cg_jpeg_resync_to_restart) in beta but not stable

[compressed]

This came up in sgrif/pq-sys#1. It seems when trying to run/test anything using beta or later this crate fails to run (not build) with the below error.

Note: I don't have DYLD_LIBRARY_PATH set as an env var anywhere. It seems to be picking up the wrong libJPEG.dylib in beta (/usr/local/lib) instead of in the OSX library location (/System/Library/Frameworks/ImageIO.framework/Resources/...). In part, I believe this to be due do the fact that the pg-sql crate is adding native=/usr/local/lib via -L because that's where the postgresql libraries from homebrew end up being installed. Curiously though, this linking problem does not occur on stable, leading me to believe it is resolving to the proper libJPEG.dylib instead of the one in /usr/local/lib.

Running OSX 10.11.5.

Also, not sure if related or not, but I tried the workaround specified in #34754, codegen-units=1, but it did not eliminate the error.

Beta (or nightly):

~/D/pq-sys git:master ❯❯❯ rustup override set beta
info: using existing install for 'beta-x86_64-apple-darwin'
info: override toolchain for '/Users/chris/Documents/pq-sys' set to 'beta-x86_64-apple-darwin'

  beta-x86_64-apple-darwin unchanged - rustc 1.11.0-beta.1 (8dc253bcf 2016-07-05)

~/D/pq-sys git:master ❯❯❯ cargo test --verbose
   Compiling pq-sys v0.2.1 (file:///Users/chris/Documents/pq-sys)
   Compiling libc v0.2.14
     Running `rustc build.rs --crate-name build_script_build --crate-type bin -g --out-dir /Users/chris/Documents/pq-sys/target/debug/build/pq-sys-683de9b8454d5f4b --emit=dep-info,link -L dependency=/Users/chris/Documents/pq-sys/target/debug -L dependency=/Users/chris/Documents/pq-sys/target/debug/deps`
     Running `rustc /Users/chris/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.14/src/lib.rs --crate-name libc --crate-type lib -g --cfg feature=\"use_std\" --cfg feature=\"default\" -C metadata=1f3392fe1afd1313 -C extra-filename=-1f3392fe1afd1313 --out-dir /Users/chris/Documents/pq-sys/target/debug/deps --emit=dep-info,link -L dependency=/Users/chris/Documents/pq-sys/target/debug/deps -L dependency=/Users/chris/Documents/pq-sys/target/debug/deps --cap-lints allow`
     Running `/Users/chris/Documents/pq-sys/target/debug/build/pq-sys-683de9b8454d5f4b/build-script-build`
     Running `rustc src/lib.rs --crate-name pq_sys -g --test -C metadata=5b5cdff68186daa5 -C extra-filename=-5b5cdff68186daa5 --out-dir /Users/chris/Documents/pq-sys/target/debug --emit=dep-info,link -L dependency=/Users/chris/Documents/pq-sys/target/debug -L dependency=/Users/chris/Documents/pq-sys/target/debug/deps --extern libc=/Users/chris/Documents/pq-sys/target/debug/deps/liblibc-1f3392fe1afd1313.rlib -L native=/usr/local/lib`
     Running `rustc src/lib.rs --crate-name pq_sys --crate-type lib -g --out-dir /Users/chris/Documents/pq-sys/target/debug --emit=dep-info,link -L dependency=/Users/chris/Documents/pq-sys/target/debug -L dependency=/Users/chris/Documents/pq-sys/target/debug/deps --extern libc=/Users/chris/Documents/pq-sys/target/debug/deps/liblibc-1f3392fe1afd1313.rlib -L native=/usr/local/lib`
     Running `/Users/chris/Documents/pq-sys/target/debug/pq_sys-5b5cdff68186daa5`
dyld: Symbol not found: __cg_jpeg_resync_to_restart
  Referenced from: /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
  Expected in: /usr/local/lib/libJPEG.dylib
 in /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
error: Process didn't exit successfully: `/Users/chris/Documents/pq-sys/target/debug/pq_sys-5b5cdff68186daa5` (signal: 5, SIGTRAP: trace/breakpoint trap)

Caused by:
  Process didn't exit successfully: `/Users/chris/Documents/pq-sys/target/debug/pq_sys-5b5cdff68186daa5` (signal: 5, SIGTRAP: trace/breakpoint trap)
~/D/pq-sys git:master ❯❯❯

Stable:

~/D/pq-sys git:master ❯❯❯ rustup override set stable
info: using existing install for 'stable-x86_64-apple-darwin'
info: override toolchain for '/Users/chris/Documents/pq-sys' set to 'stable-x86_64-apple-darwin'

  stable-x86_64-apple-darwin unchanged - rustc 1.10.0 (cfcb716cf 2016-07-03)

~/D/pq-sys git:master ❯❯❯ cargo test --verbose
       Fresh libc v0.2.14
   Compiling pq-sys v0.2.1 (file:///Users/chris/Documents/pq-sys)
     Running `rustc build.rs --crate-name build_script_build --crate-type bin -g --out-dir /Users/chris/Documents/pq-sys/target/debug/build/pq-sys-683de9b8454d5f4b --emit=dep-info,link -L dependency=/Users/chris/Documents/pq-sys/target/debug -L dependency=/Users/chris/Documents/pq-sys/target/debug/deps`
     Running `/Users/chris/Documents/pq-sys/target/debug/build/pq-sys-683de9b8454d5f4b/build-script-build`
     Running `rustc src/lib.rs --crate-name pq_sys --crate-type lib -g --out-dir /Users/chris/Documents/pq-sys/target/debug --emit=dep-info,link -L dependency=/Users/chris/Documents/pq-sys/target/debug -L dependency=/Users/chris/Documents/pq-sys/target/debug/deps --extern libc=/Users/chris/Documents/pq-sys/target/debug/deps/liblibc-1f3392fe1afd1313.rlib -L native=/usr/local/lib`
     Running `rustc src/lib.rs --crate-name pq_sys --crate-type lib -g --test -C metadata=5b5cdff68186daa5 -C extra-filename=-5b5cdff68186daa5 --out-dir /Users/chris/Documents/pq-sys/target/debug --emit=dep-info,link -L dependency=/Users/chris/Documents/pq-sys/target/debug -L dependency=/Users/chris/Documents/pq-sys/target/debug/deps --extern libc=/Users/chris/Documents/pq-sys/target/debug/deps/liblibc-1f3392fe1afd1313.rlib -L native=/usr/local/lib`
     Running `/Users/chris/Documents/pq-sys/target/debug/pq_sys-5b5cdff68186daa5`

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured

   Doc-tests pq_sys
     Running `rustdoc --test /Users/chris/Documents/pq-sys/src/lib.rs --crate-name pq_sys -L dependency=/Users/chris/Documents/pq-sys/target/debug/deps -L dependency=/Users/chris/Documents/pq-sys/target/debug -L native=/usr/local/lib --extern libc=/Users/chris/Documents/pq-sys/target/debug/deps/liblibc-1f3392fe1afd1313.rlib --extern pq_sys=/Users/chris/Documents/pq-sys/target/debug/libpq_sys.rlib`

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured

I'm not sure of a smaller set to replicate. In order to get into this situation on a mac, you need to have done the following:

1. brew install postgres
2. brew install libjpeg. This will add /usr/local/lib/libjpeg.dylib which beta ends up linking to.

Also related otool output:

~/D/pq-sys git:master ❯❯❯ otool -L /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
/System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO:
    /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO (compatibility version 1.0.0, current version 1.0.0)
    /System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
    /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics (compatibility version 64.0.0, current version 600.0.0)
    /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices (compatibility version 1.0.0, current version 48.0.0)
    /System/Library/PrivateFrameworks/AppleVPA.framework/Versions/A/AppleVPA (compatibility version 1.0.0, current version 1.0.0)
    /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1259.0.0)
    /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1258.1.0)
    /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices (compatibility version 1.0.0, current version 728.10.0)
    /System/Library/PrivateFrameworks/AppleJPEG.framework/Versions/A/AppleJPEG (compatibility version 1.0.0, current version 1.0.0)
    /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libJPEG.dylib (compatibility version 1.0.0, current version 1.0.0)
    /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libTIFF.dylib (compatibility version 1.0.0, current version 1.0.0)
    /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libPng.dylib (compatibility version 1.0.0, current version 1.0.0)
    /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libGIF.dylib (compatibility version 1.0.0, current version 1.0.0)
    /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libJP2.dylib (compatibility version 1.0.0, current version 1.0.0)
    /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libRadiance.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
    /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.5)
    /System/Library/Frameworks/IOSurface.framework/Versions/A/IOSurface (compatibility version 1.0.0, current version 1.0.0)
    /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.1.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)

[nagisa]

Where the dependency on ImageIO comes from? Is it the pgsql library itself?

My initial impression is UB caused by a miscofigured system rather than an actual regression (i.e. its a miracle it worked before). Given the compilation succeeded, the linker got the right library/framework/etc and linked the binary up properly. After that, its all in the loader’s hands.

[compressed]

@nagisa thanks for the insight! Yes, I believe postgres requires libjpeg.

Your comment led me to an interesting finding: when running the built binary directly on my shell, it doesn't generate the linking error... Only when invoked with cargo run does this occur. Maybe an issue with cargo then?

Stable cargo:

~/D/pq-sys git:master ❯❯❯ cargo --version                                                                                                                   
cargo 0.11.0-nightly (259324c 2016-05-20)

Beta cargo:

~/D/pq-sys git:master ❯❯❯ cargo --version    
cargo 0.12.0-nightly (6b98d1f 2016-07-04)

~/D/pq-sys git:master ❯❯❯ rustup override set beta                                                                                                          ✱
info: using existing install for 'beta-x86_64-apple-darwin'
info: override toolchain for '/Users/chris/Documents/pq-sys' set to 'beta-x86_64-apple-darwin'

  beta-x86_64-apple-darwin unchanged - rustc 1.11.0-beta.1 (8dc253bcf 2016-07-05)

~/D/pq-sys git:master ❯❯❯ cargo test --verbose                                                                                                              ✱
   Compiling pq-sys v0.2.1 (file:///Users/chris/Documents/pq-sys)
   Compiling libc v0.2.14
     Running `rustc build.rs --crate-name build_script_build --crate-type bin -g --out-dir /Users/chris/Documents/pq-sys/target/debug/build/pq-sys-683de9b8454d5f4b --emit=dep-info,link -L dependency=/Users/chris/Documents/pq-sys/target/debug -L dependency=/Users/chris/Documents/pq-sys/target/debug/deps`
     Running `rustc /Users/chris/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.14/src/lib.rs --crate-name libc --crate-type lib -g --cfg feature=\"default\" --cfg feature=\"use_std\" -C metadata=1f3392fe1afd1313 -C extra-filename=-1f3392fe1afd1313 --out-dir /Users/chris/Documents/pq-sys/target/debug/deps --emit=dep-info,link -L dependency=/Users/chris/Documents/pq-sys/target/debug/deps -L dependency=/Users/chris/Documents/pq-sys/target/debug/deps --cap-lints allow`
     Running `/Users/chris/Documents/pq-sys/target/debug/build/pq-sys-683de9b8454d5f4b/build-script-build`
     Running `rustc src/lib.rs --crate-name pq_sys -C codegen-units=1 -g --test -C metadata=5b5cdff68186daa5 -C extra-filename=-5b5cdff68186daa5 --out-dir /Users/chris/Documents/pq-sys/target/debug --emit=dep-info,link -L dependency=/Users/chris/Documents/pq-sys/target/debug -L dependency=/Users/chris/Documents/pq-sys/target/debug/deps --extern libc=/Users/chris/Documents/pq-sys/target/debug/deps/liblibc-1f3392fe1afd1313.rlib -L native=/usr/local/lib`
     Running `rustc src/lib.rs --crate-name pq_sys --crate-type lib -g --out-dir /Users/chris/Documents/pq-sys/target/debug --emit=dep-info,link -L dependency=/Users/chris/Documents/pq-sys/target/debug -L dependency=/Users/chris/Documents/pq-sys/target/debug/deps --extern libc=/Users/chris/Documents/pq-sys/target/debug/deps/liblibc-1f3392fe1afd1313.rlib -L native=/usr/local/lib`
     Running `/Users/chris/Documents/pq-sys/target/debug/pq_sys-5b5cdff68186daa5`
dyld: Symbol not found: __cg_jpeg_resync_to_restart
  Referenced from: /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
  Expected in: /usr/local/lib/libJPEG.dylib
 in /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
error: Process didn't exit successfully: `/Users/chris/Documents/pq-sys/target/debug/pq_sys-5b5cdff68186daa5` (signal: 5, SIGTRAP: trace/breakpoint trap)

Caused by:
  Process didn't exit successfully: `/Users/chris/Documents/pq-sys/target/debug/pq_sys-5b5cdff68186daa5` (signal: 5, SIGTRAP: trace/breakpoint trap)

Now if I run the built binary manually it works:

~/D/pq-sys git:master ❯❯❯ /Users/chris/Documents/pq-sys/target/debug/pq_sys-5b5cdff68186daa5

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured

[compressed]

Got it - this is due to this cargo change rust-lang/cargo#2763, which explains why stable and beta have different behavior. Error starts with the 2016-06-07 nightly.

Edit: The reason this worked before is because DYLD_LIBRARY_PATH had native=/usr/local/lib added instead of /usr/local/lib, so /usr/local/lib/libjpeg.dylib could never have been found because /usr/local/lib wasn't added to DYLD_LIBRARY_PATH correctly. With the cargo bug fixed, now this issue has appeared :) We should be able to resolve this in pq-sys directly.

[Xe]

I ran into this today with a fresh install of rust nightly on a MacBook.

om:testdata cadey$ cargo install racer
    Updating registry `https://github.com/rust-lang/crates.io-index`
 Downloading racer v2.1.5
  Installing racer v2.1.5
 Downloading clap v2.32.0
 Downloading lazycell v1.0.0
 Downloading rustc-ap-syntax v237.0.0
 Downloading lazy_static v1.1.0
 Downloading rls-span v0.4.0
 Downloading log v0.4.5
 Downloading bitflags v1.0.4
 Downloading racer-cargo-metadata v0.1.0
 Downloading derive_more v0.11.0
 Downloading vec_map v0.8.1
 Downloading strsim v0.7.0
 Downloading textwrap v0.10.0
 Downloading ansi_term v0.11.0
 Downloading rustc-ap-rustc_errors v237.0.0
 Downloading rustc-ap-rustc_target v237.0.0
 Downloading rustc-ap-rustc_data_structures v237.0.0
 Downloading scoped-tls v0.1.2
 Downloading smallvec v0.6.5
 Downloading rustc-ap-syntax_pos v237.0.0
 Downloading rustc-ap-serialize v237.0.0
 Downloading termcolor v0.3.6
 Downloading rustc-ap-arena v237.0.0
 Downloading ena v0.9.3
 Downloading parking_lot_core v0.2.14
 Downloading rustc-rayon v0.1.1
 Downloading stable_deref_trait v1.1.1
 Downloading parking_lot v0.5.5
 Downloading rustc-hash v1.0.1
 Downloading rustc-rayon-core v0.1.1
 Downloading rustc-ap-rustc_cratesio_shim v237.0.0
 Downloading rand v0.4.3
 Downloading unreachable v1.0.0
 Downloading void v1.0.2
 Downloading either v1.5.0
 Downloading num_cpus v1.8.0
 Downloading crossbeam-deque v0.2.0
 Downloading crossbeam-epoch v0.3.1
 Downloading crossbeam-utils v0.2.2
 Downloading arrayvec v0.4.7
 Downloading memoffset v0.2.1
 Downloading nodrop v0.1.12
 Downloading scopeguard v0.3.3
 Downloading version_check v0.1.4
 Downloading owning_ref v0.3.3
 Downloading regex v1.0.5
 Downloading termcolor v1.0.3
 Downloading aho-corasick v0.6.8
 Downloading regex-syntax v0.6.2
 Downloading utf8-ranges v1.0.1
 Downloading memchr v2.0.2
 Downloading thread_local v0.3.6
 Downloading ucd-util v0.1.1
 Downloading racer-interner v0.1.0
 Downloading serde v1.0.76
 Downloading serde_json v1.0.27
 Downloading serde_derive v1.0.76
 Downloading quote v0.6.8
 Downloading proc-macro2 v0.4.18
 Downloading syn v0.14.9
 Downloading unicode-xid v0.1.0
 Downloading itoa v0.4.2
 Downloading ryu v0.2.6
 Downloading quote v0.5.2
 Downloading syn v0.13.11
 Downloading proc-macro2 v0.3.8
   Compiling version_check v0.1.4
   Compiling nodrop v0.1.12
   Compiling cfg-if v0.1.5
   Compiling memoffset v0.2.1
dyld: Symbol not found: __cg_jpeg_resync_to_restart
  Referenced from: /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
  Expected in: /usr/local/lib/libJPEG.dylib
 in /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
dyld: Symbol not found: __cg_jpeg_resync_to_restart
  Referenced from: /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
  Expected in: /usr/local/lib/libJPEG.dylib
 in /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
dyld: Symbol not found: __cg_jpeg_resync_to_restart
  Referenced from: /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
  Expected in: /usr/local/lib/libJPEG.dylib
 in /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
error: Could not compile `nodrop`.
warning: build failed, waiting for other jobs to finish...
error: Could not compile `memoffset`.
warning: build failed, waiting for other jobs to finish...
error: Could not compile `version_check`.
warning: build failed, waiting for other jobs to finish...
dyld: Symbol not found: __cg_jpeg_resync_to_restart
  Referenced from: /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
  Expected in: /usr/local/lib/libJPEG.dylib
 in /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
error: failed to compile `racer v2.1.5`, intermediate artifacts can be found at `/var/folders/90/m9qxx4xn30j6qb9_kmyz85gc0000gn/T/cargo-installHUG23T`

Caused by:
  Could not compile `cfg-if`.

To learn more, run the command again with --verbose.
om:testdata cadey$

[donbright]

i had this on cargo build of a 'hello world' project with no dependencies. i was able to "solve" this by uninstalling homebrew. MacOS 10.13.6