Turing on Windows requires a compiler

[ChrisRackauckas]

=========================================[ ERROR: Turing ]==========================================

LoadError: could not spawn `mingw32-make`: no such file or directory (ENOENT)
while loading C:\Users\Chris\.julia\v0.6\Turing\deps\build.jl, inexpression starting on line 2

====================================================================================================

Can this instead be done with BinaryProvider.jl and BinaryBuilder.jl?

[xukai92]

@yebai Can you have a look into this issue please?

[yebai]

@ChrisRackauckas I believe this can be solved by packages such as BinaryProvider.jl and BinaryBuilder.jl. However, I don't have direct experience with these packages. Would someone with more experience create a pull request?

cc'ing @antoine-soubret (#258) and @hessammehr (#435) and @AresFishy, #409

[hessammehr]

For what it's worth, I couldn't get very far with BinaryBuilder.jl on Windows (native and WSL). Will try again in a virtual machine and let you know.

[ChrisRackauckas]

I would recommend using an Ubuntu 16 virtual machine, and then setting up the Builder repo with Travis doing the auto builds.

[hessammehr]

Okay, thanks for the pointers. I have no experience with BinaryBuilder but will have a go at it in a minute.

[hessammehr]

So the issue here is that the build script depends on Julia itself (task.c uses julia.h and links against Julia libs), which BinaryBuilder.jl doesn't provide in its RootFS. And none of the examples that I looked at depended on Julia, so I'm clueless. For the Linux target one could wget the Julia binary tarball and extract it on the fly; not too sure about Windows.

[yebai]

For the Linux target one could wget the Julia binary tarball and extract it on the fly; not too sure about Windows.

@hessammehr You may find the Appveyor configuration file helpful. Basically, we download Julia and MinGW first, then Turing's build script is run:

https://github.com/yebai/Turing.jl/blob/master/appveyor.yml

I'm glad that someone gave it an attempt at this issue, I'll be happy to help along the process if this can be solved.

[hessammehr]

Thanks @yebai, I'll definitely check out your Appveyor config. Just wondering, though - how indispensable is task.c?

[hessammehr]

So I have BinaryBuilder working for all Linux targets here (still waiting on Travis). I'm still struggling a bit with Windows though since I haven't figured out how to extract the Julia installer .exe file. Anyone know of a working portable Linux version of 7zip?
@yebai If I understand it correctly, your Appveyor scripts simply runs the runs the Julia Windows installer, which isn't possible in the MinGW/MSYS/Linux environment of BinaryBuilder. Any thoughts?

[yebai]

Just wondering, though - how indispensable is task.c?

Well, it is not essential, but it reduces the time complexity for particle MCMC methods from O(N^2) to linear time. If you are interested, you can have a look at JuliaLang/julia#4085 and JuliaLang/julia#15078. Ideally, we should get the PR for task copying merged into Julia, but that has not happened yet.

@yebai If I understand it correctly, your Appveyor scripts simply runs the runs the Julia Windows installer, which isn't possible in the MinGW/MSYS/Linux environment of BinaryBuilder. Any thoughts?

That's correct - we just run the Julia installer on Appveyor. Perhaps you can have a look at the following 7zip page for ubuntu?

https://www.howtoforge.com/tutorial/how-to-install-and-use-7zip-file-archiver-on-ubuntu-linux/

[yebai]

@hessammehr I've manually converted the windows exe file to tar.gz format on my Mac. Please find links below:

http://mlg.eng.cam.ac.uk/hong/julia-0.6.2-win32.tar.gz
http://mlg.eng.cam.ac.uk/hong/julia-0.6.2-win64.tar.gz

This isn't a perfect solution, but perhaps useful for getting the main job here done quickly.

[hessammehr]

@yebai Thanks, I'll give it a shot with your tarball and let you know.

Re. 7zip - the problems is that BinaryBuilder's rootfs has no apt equivalent. That and I couldn't find a working statically-linked version of p7zip.

[hessammehr]

@yebai Looking pretty good (hessammehr/LibtaskBuilder2) - Windows build seems to be working just fine. The big holdup right now is 7zip to extract the Julia installer package on Windows (.exe) and macOS (.dmg). Perhaps I'll use BinaryBuilder to distribute 7z itself!

[yebai]

@hessammehr I just became aware of the following portable version of 7zip for Linux

https://sourceforge.net/projects/p7zip/files/p7zip/16.02/

There is a generic binary tarball for this project, which might be helpful.

[hessammehr]

Yes, I tried that and ... no luck :( Apparently, it's a 32-bit ELF so I get the following when I run it

$ p7zip_16.02/bin/7z
-bash: p7zip_16.02/bin/7z: No such file or directory
$ file p7zip_16.02/bin/7z
p7zip_16.02/bin/7z: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 2.0.30, stripped
$ ldd p7zip_16.02/bin/7z
        not a dynamic executable

Hmmm...

[hessammehr]

So we have build artefacts now!

[yebai]

@hessammehr Looks great! Here is the link to the tarball of OSX version of Julia

http://mlg.eng.cam.ac.uk/hong/julia-0.6.2-mac64.tar.gz

If you can create a PR for Turing using BinaryProvider, I will test it on Travis and Appveyor quickly.

[hessammehr]

I have binaries built for all platforms now and the BinaryBuilder-generated build.jl on my fork (hessammehr/Turing.jl), however:

On Windows:

Pkg.clone("https://github.com/hessammehr/Turing.jl")
Pkg.build("Turing")

Errors out with

BinaryProvider.LibraryProduct("C:\\blahblah\\.julia\\v0.6\\Turing\\deps\\usr\\bin", String["libtask"], :libtask, "Prefix(C:\\blahblah\\.julia\\v0.6\\Turing\\deps\\usr)") is not satisfied, cannot generate deps.jl!

@yebai @ChrisRackauckas Ever encountered this sort of error message?

[ChrisRackauckas]

Why are you using an absolute path for the binaries? That seems prone to failure if the location of the package moves. I thought they give you the build script?

[hessammehr]

Oh right, you're looking at the error message on that bottom line. I should have made that more clear. There are no hard-coded paths as far as I can see.

[yebai]

@hessammehr This is really helpful. Many thanks for the contribution!

cc'ing @xukai92 @emilemathieu @willtebbutt @trappmartin Turing now has binary build, and no more require MinGW on Windows!