VideoIO tests fail to find zlib1.2.9 when loading ImageMagick.jl on Travis

[kmsquire]

It might be that I need to install something else in Travis, but I'm wondering if anyone has any hints here? It worked fine when testing against v0.6.3.

Fatal error:
ERROR: LoadError: LoadError: InitError: error compiling __init__: could not load library "/home/travis/.julia/packages/ImageMagick/W81WA/deps/usr/lib/libMagickWand-6.Q16.so"
/lib/x86_64-linux-gnu/libz.so.1: version `ZLIB_1.2.9' not found (required by /home/travis/.julia/packages/ImageMagick/W81WA/deps/usr/lib/libpng16.so.16)

See: See https://travis-ci.org/kmsquire/VideoIO.jl/jobs/418942394#L1206

[SimonDanisch]

libz.so.1 - that looks a lot like JuliaLang/julia#22828

[kmsquire]

@vtjnash, can you give suggestions on the proper fix here, or point to some docs on the matter? Thanks!

[vtjnash]

@vchuravy IIRC, you were working on changing our zlib version. Can you remind me, what was the end state of that?

[vchuravy]

We removed ZLib as a dependency from Julia/LLVM.

[kmsquire]

@vtjnash, @vchuravy, sorry I bothered you--this seems to be an issue how libpng is being built here.

@SimonDanisch, In the deps directory, you're building zlib v1.2.11, but, the version of libpng installed by ImageMagick was (per the error message) linked with zlib v1.2.9.

I think you need to update https://github.com/SimonDanisch/LibpngBuilder to build with the same version of zlib.

Presumably, then, it will use the version installed in the same directory under ImageMagick (/home/travis/.julia/packages/ImageMagick/W81WA/deps/usr/lib/)

[SimonDanisch]

Hm that's weird, since zlibbuilder has always just offered 1.2.11 as far as I can tell!

[kmsquire]

Could it be that libpngbuilder isn't using the output of zlibbuilder, but is instead using, e.g., a system zlib?

[SimonDanisch]

-- Found ZLIB: /workspace/destdir/lib/libz.so (found version "1.2.11")

that's the output of the configure of libpngbuilder

[SimonDanisch]

well, considering that travis tests run fine for imagemagick, could this be some problem introduced by ffmpeg in VideoIO?

[kmsquire]

-- Found ZLIB: /workspace/destdir/lib/libz.so (found version "1.2.11")
that's the output of the configure of libpngbuilder

Interesting. If run ldd on https://github.com/SimonDanisch/LibpngBuilder/releases/download/1.0.0/libpng.v1.6.31.i686-linux-gnu.tar.gz, I get the same error as in VideoIO.jl's logs:

# ldd libpng16.so.16.31.0
./libpng16.so.16.31.0: /usr/lib32/libz.so.1: version `ZLIB_1.2.9' not found (required by ./libpng16.so.16.31.0)
	linux-gate.so.1 (0xf7755000)
	libz.so.1 => /usr/lib32/libz.so.1 (0xf76f1000)
	libm.so.6 => /lib32/libm.so.6 (0xf769c000)
	libc.so.6 => /lib32/libc.so.6 (0xf74e5000)
	/lib/ld-linux.so.2 (0xf7756000)

[kmsquire]

And if I make https://github.com/bicycle1885/ZlibBuilder/releases/download/v1.0.2/Zlib.v1.2.11.i686-linux-gnu.tar.gz available, then the error goes away!

So, the problem is probably that the library search path isn't configured to find zlib in /home/travis/.julia/packages/ImageMagick/W81WA/deps/usr/lib/ when ImageMagick is installed under VideoIO.

[kmsquire]

... any thoughts on how to fix that?

[kmsquire]

(Moving this issue to BinaryBuilder)

[kmsquire]

After exploring a bit more, it seems that the problem is likely related to JuliaPackaging/Requires.jl#52 . Thanks for the feedback here, and sorry for the noise!

[kmsquire]

After exploring a bit more, it seems that the problem is likely related to JuliaPackaging/Requires.jl#52 .

Okay, that doesn't seem to be it. :-( Will explore some more.

[kmsquire]

Okay, I think I figured out the issue. VideoIO is not using BinaryBuilder, and is instead depending on detecting and using system-installed versions of ffmpeg or libav. Those libraries are linked against the system libz, so if they are loaded, then the library loader attempts to resolve any future references to libz against the system libz, causing the error above.

[SimonDanisch]

Hm, I actually have the same issue with ImageMagick on ubuntu only in the Makie tests... ImageMagick alone passes tests.

What's weird is, that it only happens if I actually run ]test Makie - if I just include the runtest, I don't get the problem ... So must be something about the code loading of Pkg + together with the new isolated environment that Pkg3 creates for running the tests (that's what Pkg3 does, right?).

@staticfloat, @StefanKarpinski @KristofferC, any idea what could lead to this?
Looks a bit like ImageMagick only gets partially build leaving out the dependency of e.g. zlib - which makes it try to pick it up from the path, or something like this?!

Visual Regression: Error During Test at /home/s/.julia/packages/Makie/YPDAk/test/runtests.jl:110
  Got exception outside of a @test
  InitError: error compiling __init__: could not load library "/home/s/.julia/packages/ImageMagick/d5KBL/deps/usr/lib/libMagickWand-6.Q16.so"
  /lib/x86_64-linux-gnu/libz.so.1: version `ZLIB_1.2.9' not found (required by /home/s/.julia/packages/ImageMagick/d5KBL/deps/usr/lib/libpng16.so.16)
  during initialization of module ImageMagick

[timholy]

This just cropped up in JuliaImages/ImageView.jl#156. Can we build zlib statically into ImageMagick?

[bjarthur]

the hack of importing ImageMagick before using another package which relies on it (in my case Images) works for me.

[timholy]

Should be fixed on Julia 1.3+