Using on OS X: 'Symbol not found: __cg_jpeg_resync_to_restart' #1

whodidthis · 2016-07-10T08:26:32Z

I tried using diesel but something gives the following error on a mac from just adding the pq-sys dependency:

> cat Cargo.toml
[package]
name = "testing"
version = "0.1.0"
[dependencies]
pq-sys = "0.2.1"

> cat src/lib.rs
extern crate pq_sys;

> cargo test --verbose
       Fresh libc v0.2.13
       Fresh pq-sys v0.2.1
       Fresh testing v0.1.0 (file:///Users/user/testing)
     Running `/Users/user/testing/target/debug/testing-0688888a7aeac232`
dyld: Symbol not found: __cg_jpeg_resync_to_restart
  Referenced from: /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
  Expected in: /usr/local/lib/libJPEG.dylib
 in /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
error: Process didn't exit successfully: `/Users/user/testing/target/debug/testing-0688888a7aeac232` (signal: 5, SIGTRAP: trace/breakpoint trap)

Caused by:
  Process didn't exit successfully: `/Users/user/testing/target/debug/testing-0688888a7aeac232` (signal: 5, SIGTRAP: trace/breakpoint trap)

The text was updated successfully, but these errors were encountered:

whodidthis · 2016-07-10T08:53:42Z

Also doing this weird stuff from here seems to fix the problem if that helps:

cd /usr/local/lib
mv libjpeg.dylib libjpeg.dylib.backup
ln -s /System/Library/Frameworks/ImageIO.framework/Resources/libJPEG.dylib libJPEG.dylib
mv libtiff.dylib libtiff.dylib.backup
ln -s /System/Library/Frameworks/ImageIO.framework/Resources/libTIFF.dylib libTIFF.dylib
mv libpng.dylib libpng.dylib.backup
ln -s /System/Library/Frameworks/ImageIO.framework/Resources/libPng.dylib libPNG.dylib

Edit: Not recommended as breaks things!

compressed · 2016-07-16T03:02:18Z

I've run into the same problem and traced it back to here as well...

Running:

DYLD_LIBRARY_PATH=/System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/ cargo run

Seems to make it compile. In my case, some packages I installed using homebrew have made these symlinks in /usr/local/lib. It seems though that somehow /usr/local/lib is added first to the DYLD_LIBRARY_PATH env var when compiling. I've checked my shell though and I don't have that environment variable set anywhere...

It seems since it includes /usr/local/lib first then cargo ends up linking to the wrong files (https://discussions.apple.com/thread/2084183)...

compressed · 2016-07-16T03:07:10Z

The other strange thing, if I compile with rust beta (1.11.0-beta.1), I get the linking error. If I compile with rust stable (1.10.0), it builds correctly.

compressed · 2016-07-17T20:31:56Z

So after going down the rabbit hole, here's what's going on:

There was a bug in cargo (Correctly record multiple native dirs per package rust-lang/cargo#2763) that was fixed on June 6th. This bug actually prevented this line from having any impact on mac because it wasn't formed correctly when insert into DYLD_LIBRARY_PATH env var; it had native=/usr/local/lib added instead of /usr/local/lib.
After June 6th, when the bug was fixed, the build line actually inserted /usr/local/lib into the DYLD_LIBRARY_PATH env var which messes up the mac loader. From what I've read, setting/modifying this env var on macs is generally not a good idea for these exact reasons.
Stable rust uses a cargo version prior to June 6th so no one using that version would see this problem. Beta uses a newer cargo version so it winds up failing because it's loading the wrong libjpeg.dylib.

So now.. how do we fix this? As @whodidthis mentioned, you could change the symlinks in /usr/local/lib, but that isn't really a good fix because now any packages that you've installed using homebrew with a different version of libjpeg won't work anymore.

You could install libpq in a non-standard search path, like I guess windows does, but that seems like a bad idea.

The other option that could work would be just to add some conditional compilation here. If windows is the only platform with libpq outside its search path, then we can have that build logic only run for windows.

How about this for build.rs?

#[cfg(windows)]
fn main() {
    use std::process::Command;

    for output in Command::new("pg_config").arg("--libdir").output() {
        if output.status.success() {
            for path in String::from_utf8(output.stdout) {
                println!("cargo:rustc-link-search=native={}", &path);
            }
        }
    }
}

#[cfg(not(windows))]
fn main() {}

compressed · 2016-07-18T11:41:40Z

Ping @sgrif

sgrif · 2016-07-18T13:23:19Z

Hrm. I've always assumed this was a Rust bug. brew uninstall libjpeg has fixed this for me in the past (and then you can reinstall afterwards IIRC)

compressed · 2016-07-18T14:03:31Z

Ah, I hadn't tried that. However, it doesn't seem to help. I needed to do a brew uninstall libjpeg libpng giflib libtiff and then pq-sys will run, but now all the homebrew packages I have that use them (e.g. vips) are broken. If I then do brew install libjpeg libpng giflib libtiff to get them back, then the homebrew packages will work, but now I'm back with pq-sys loading the wrong dependencies (from /usr/local/lib).

sgrif · 2016-07-18T14:10:39Z

What the hell is even requiring libjpeg in the first place? I'd be fine with changing the build script to make sure that we only specifically change the linking for libpq, but I'd prefer not to make the change you've proposed as it will break anyone who has libpq installed in a nonstandard location.

I'm also wondering if this is actually something that needs to get looked at upstream? Surely anybody who ends up doing cargo:rustc-link-search=native=/usr/local/lib will break mac users for this reason?

compressed · 2016-07-18T15:18:52Z

To be honest, I'm not exactly sure how libjpeg, etc. is coming into the picture. Homebrew isn't listing it as a dependency:

~/D/pq-sys git:master ❯❯❯ brew deps postgresql                                                    ⏎ ⬆ ✱
openssl
readline

I still get the error if I reduce lib.rs to just this:

#[link(name = "pq")]
extern "C" {}

To your second point, I'm a bit torn on it. On one hand, it's obviously useful to specify it for non-standard locations, but now when we go to run, we're saying we're going to specifically override the loader's search path to have a priority for /usr/local/lib. So it's technically doing what we've asked, but it just won't work in this scenario. But you're correct, adding /usr/local/lib to DYLD_LIBRARY_PATH is probably going to break for mac users with homebrew packages installed (bunch of issues related to this on stackoverflow, etc.).

I was originally thinking it could be possible to only specify that rust-link-search override if that location isn't in the platform's linker path already, but there doesn't seem to be a straight-forward way to do so (e.g. something like gcc -Xlinker -v).

sgrif · 2016-07-18T15:48:29Z

Is there a way for us to tell the linker to additionally look in a path but
continuing to give the system path priority? (That was what I thought -L
did)

On Mon, Jul 18, 2016, 11:18 AM compressed notifications@github.com wrote:

To be honest, I'm not exactly sure how libjpeg, etc. is coming into the
picture. Homebrew isn't listing it as a dependency:

~/D/pq-sys git:master ❯❯❯ brew deps postgresql ⏎ ⬆ ✱
openssl
readline

I still get the error if I reduce lib.rs to just this:

#[link(name = "pq")]
extern "C" {}

To your second point, I'm a bit torn on it. On one hand, it's obviously
useful to specify it for non-standard locations, but now when we go to run,
we're saying we're going to specifically override the loader's search path
to have a priority for /usr/local/lib. So it's technically doing what
we've asked, but it just won't work in this scenario. But you're correct,
adding /usr/local/lib to DYLD_LIBRARY_PATH is probably going to break for
mac users with homebrew packages installed (bunch of issues related to this
on stackoverflow, etc.).

I was originally thinking it could be possible to only specify that
rust-link-search override if that location isn't in the platform's linker
path already, but there doesn't seem to be a straight-forward way to do so
(e.g. something like gcc -Xlinker -v).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABdWK57R_Z1aaDFEzWLEhLH07x-vIsoXks5qW5lcgaJpZM4JIyln
.

compressed · 2016-07-19T10:32:29Z

I'm not sure there is a way to do that. We would probably need to add the system paths first.

I remembered today that openssl-sys requires environment vars to install correctly on mac, see: https://github.com/sfackler/rust-openssl and https://github.com/sfackler/rust-openssl/blob/master/openssl-sys/build.rs. We could follow a similar approach (e.g. PQ_LIB_DIR) that would work for all platforms. If you wanted to go further, we could use this pkg-config crate it seems to use to try to find the library; I haven't had any experience with that.

Interestingly, brew --prefix postgresql returns (surprise) yet another path (/usr/local/opt/postgresql, which is a symlink) but would also work, since it's not /usr/local/lib anymore (/usr/local/opt/postgresql/lib). Actually though, for mac I don't believe we would even need to set this path since it's already on the mac loader path.

What do you think of this approach?

compressed · 2016-07-25T16:56:46Z

I submitted a PR for the above idea. I think this seems to be the safest way to move forward. I didn't bother with pulling in pkg-config since it seemed to add more complexity than we probably need given the existing way is working for most users already.

sgrif · 2016-07-27T15:50:03Z

I'm still wondering what on earth was wanting to link against libjpeg

sgrif · 2016-08-12T17:16:11Z

So to follow up, I still never figured out exactly why libpq depends on libjpeg, but it's some very deep transitive dependency of a platform framework it uses on Mac. I've spent a good bit of time researching the issue, and now that I have a better understanding of the problem I decided this is common enough to try to handle.

So basically on Mac, if pg_config gives us a directory where libpq.dylib is a symlink, we will canonicalize that path and then add the resulting directory to the search path instead. So we'll get something like /usr/local/Cellar/postgresql/9.5.3/lib instead of /usr/local/lib. I've released 0.2.3 with this change.

compressed · 2016-09-02T15:31:17Z

Thanks for the update. I agree it is some deep transitive dependency as well.

Good idea for following the symlink! I've tried the new version and it works for me, without having to set the env var. Thanks!

Possibly related to the conversation in sgrif/pq-sys#1. Attempts to fix a `dyld: Symbol not found: __cg_jpeg_resync_to_restart` in travis.

Spandan-Madan · 2018-08-12T09:17:42Z

@whodidthis Cannot thank you enough. Been trying to debug this for over 3 hours, your solution finally worked. Thanks a bunch :)

MicheleGrizi7 · 2018-11-05T11:43:23Z

Also doing this weird stuff from here seems to fix the problem if that helps:

cd /usr/local/lib
mv libjpeg.dylib libjpeg.dylib.backup
ln -s /System/Library/Frameworks/ImageIO.framework/Resources/libJPEG.dylib libJPEG.dylib
mv libtiff.dylib libtiff.dylib.backup
ln -s /System/Library/Frameworks/ImageIO.framework/Resources/libTIFF.dylib libTIFF.dylib
mv libpng.dylib libpng.dylib.backup
ln -s /System/Library/Frameworks/ImageIO.framework/Resources/libPng.dylib libPNG.dylib

Edit: Not recommended as breaks things!

Thank you. I met the some issue and solved it by typing these command!

compressed mentioned this issue Jul 16, 2016

Mac - linking error (dyld: Symbol not found: __cg_jpeg_resync_to_restart) in beta but not stable rust-lang/rust#34869

Closed

compressed mentioned this issue Jul 25, 2016

fix: support PQ_LIB_DIR env variable #2

Merged

sgrif closed this as completed in #2 Jul 25, 2016

millerjs added a commit to millerjs/modelm that referenced this issue Sep 6, 2016

travis: uninstall lib{jpeg,png}

5d7f57a

Possibly related to the conversation in sgrif/pq-sys#1. Attempts to fix a `dyld: Symbol not found: __cg_jpeg_resync_to_restart` in travis.

millerjs added a commit to millerjs/modelm that referenced this issue Sep 6, 2016

travis: uninstall libpng

2fca608

Possibly related to the conversation in sgrif/pq-sys#1. Attempts to fix a `dyld: Symbol not found: __cg_jpeg_resync_to_restart` in travis.

bfops mentioned this issue Feb 2, 2017

Unable to start playform on macOS Sierra bfops/playform#189

Open

subnomo mentioned this issue Dec 9, 2018

Can't install on OSX 10.14.1 subnomo/tumblr-likes#2

Closed

kouroshD mentioned this issue Feb 11, 2019

Resolve problems to install of ROS in MacOS for the HDE visualization robotology/human-dynamics-estimation#93

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Using on OS X: 'Symbol not found: __cg_jpeg_resync_to_restart' #1

Using on OS X: 'Symbol not found: __cg_jpeg_resync_to_restart' #1

whodidthis commented Jul 10, 2016

whodidthis commented Jul 10, 2016 •

edited

Loading

compressed commented Jul 16, 2016

compressed commented Jul 16, 2016

compressed commented Jul 17, 2016

compressed commented Jul 18, 2016

sgrif commented Jul 18, 2016

compressed commented Jul 18, 2016

sgrif commented Jul 18, 2016

compressed commented Jul 18, 2016

sgrif commented Jul 18, 2016

compressed commented Jul 19, 2016

compressed commented Jul 25, 2016

sgrif commented Jul 27, 2016

sgrif commented Aug 12, 2016

compressed commented Sep 2, 2016

Spandan-Madan commented Aug 12, 2018

MicheleGrizi7 commented Nov 5, 2018

Using on OS X: 'Symbol not found: __cg_jpeg_resync_to_restart' #1

Using on OS X: 'Symbol not found: __cg_jpeg_resync_to_restart' #1

Comments

whodidthis commented Jul 10, 2016

whodidthis commented Jul 10, 2016 • edited Loading

compressed commented Jul 16, 2016

compressed commented Jul 16, 2016

compressed commented Jul 17, 2016

compressed commented Jul 18, 2016

sgrif commented Jul 18, 2016

compressed commented Jul 18, 2016

sgrif commented Jul 18, 2016

compressed commented Jul 18, 2016

sgrif commented Jul 18, 2016

compressed commented Jul 19, 2016

compressed commented Jul 25, 2016

sgrif commented Jul 27, 2016

sgrif commented Aug 12, 2016

compressed commented Sep 2, 2016

Spandan-Madan commented Aug 12, 2018

MicheleGrizi7 commented Nov 5, 2018

whodidthis commented Jul 10, 2016 •

edited

Loading