Add Binaries installation method on linux

[panlanfeng]

I am not sure if there is any reason not to include Binaries method on linux but there are some packages do provide binary files for linux, like Yeppp!.

[tkelman]

Hm. I guess the only reason for not preferring these by default is that it's fairly involved to create binaries that properly work across many different Linux distributions. I wouldn't want to encourage people to compile a binary on Ubuntu, which isn't likely to work for a user on CentOS or Fedora. We could maybe do this at a lower priority than package manager or system paths? It's also possible to push to the default providers in your package's specific deps/build.jl script as a workaround.

[panlanfeng]

How should I push to the default providers?

[tkelman]

See https://github.com/JuliaOpt/NLopt.jl/blob/e8e736f20cd4b08e8fbfc15686f4a87ba5e1aae7/deps/build.jl#L52-L56 for an example - it's not exactly clean, but works if you're in a hurry. I guess there's no harm in this PR though.

[panlanfeng]

Thanks for your help! It works.
We probably should give a warning if using Binaries provider on Linux instead of disabling it.

[bjarthur]

+1 for including Binaries in the default providers for linux

...because i don't have sudo access on some of the machines i use julia on, and so yum is not an option. and other times the BuildProcess for certain packages doesn't make the specific version i need (e.g. imagemagick). there are even cases where the BuildProcess is (currently) broken (e.g. ffmpeg/VideoIO).

so it's handy to be able to build things yourself and put them in your home directory, and then tell the build script where they are, like this.

[tkelman]

Pointing to a local source build isn't really what the Binaries provider is for. If it's easy to do a local source build and use that, then you should add that as a fallback option to the package's build.jl script.

[bjarthur]

having build.jl fallback to a BuildProcess does not provide for the following use cases:

1. the user wants a specific version

2. the user has a binary already installed in a non-standard location they'd like to use (by either themselves or provided by their sysadmin)

3. the BuildProcess is broken and a work around is needed.

it would be quite helpful if Binaries was in linux defaults, and all build.jl scripts provided a mechanism whereby users could specify a custom location through ENV, like ImageMagick's does.

[tkelman]

I don't think it's appropriate for this to be enabled by default, given the number of specific steps you need to go through to actually get redistributable binaries on Linux. Whatever replaces BinDeps will be designed to make that straightforward, but currently we shouldn't be encouraging people to package up whatever they've built with their default Ubuntu system compilers in a way that BinDeps will attempt to use on other distros.

[tkelman]

In your situation, modifying search paths like LD_LIBRARY_PATH should allow the existing enabled-by-default providers to find your custom local versions.

[staticfloat]

The lack of this is breaking my ability to mock up a BinaryProvider provider for BinDeps. I think we should make this change.

[tkelman]

What does hooking up BinaryProvider to BinDeps give us that the existing Binaries provider here doesn't?

The build scaffolding to easily produce binaries that are portable across distros is the harder part of this. If we have that, we don't need BinDeps at all.

[staticfloat]

What does hooking up BinaryProvider to BinDeps give us that the existing Binaries provider here doesn't?

Well for one thing, the existing Binaries provider doesn't work at all on Linux, because this PR isn't merged.

The build scaffolding to easily produce binaries that are portable across distros is the harder part of this. If we have that, we don't need BinDeps at all.

While I certainly hope that will be true, there's a long way from here to there, and I can very easily imagine a situation where we have a package author that builds great binaries for their favorite platforms but either does not care to, or for some reason cannot build proper binaries for some other platform. Some other contributor finds out that Homebrew or WinRPM works well, and wants to add it in, and all of a sudden we need a way for everything to live happily within one build.jl. While I'd like to get us away from BinDeps completely, there's no way we're not going to have situations where we need to have some backwards compatibility, so since it's an easy thing to do, might as well just do it and avoid the pain later.

[tkelman]

that Homebrew or WinRPM works well

they don't though.

[tkelman]

If we're building a replacement, the first thing that replacement should involve is deleting all code that makes use of BinDeps in the packages that switch. If we encourage it to stay around, it will never go away.

If a platform isn't yet supported, the way to fix that will be for the contributor to help get binaries built for that platform.

BinDeps plus "new package for downloading binaries" doesn't provide any new functionality that BinDeps doesn't already have. And it makes the new package more complicated to support functionality that it doesn't need.

[staticfloat]

If we encourage it to stay around, it will never go away.

It will go away because people hate it. Seriously. When was the last time you heard someone say something good about BinDeps or how much they love wrestling with macros? What people want is something that works, and if we can build something that works more reliably than BinDeps, they will use it, despite the fact that BinDeps will always be there. Your argument doesn't make any sense, because we're not going to remove BinDeps from METADATA.jl; it will always be an option until it bitrots so badly that it's unusable.

If a platform isn't yet supported, the way to fix that will be for the contributor to help get binaries built for that platform.

Let's figure this one out once we have the scaffolding in place. Maybe it will be trivial to get binaries built for every platform easily, but from the packaging that I've done, I think it will be a major achievement if we can get cross-platform packaging easy enough that every package author can trivially support platforms that they have no access to.

BinDeps plus "new package for downloading binaries" doesn't provide any new functionality that BinDeps doesn't already have. And it makes the new package more complicated to support functionality that it doesn't need.

I think you may be overstating the complexity imposed by BinDeps compatibility. I've got it in 40 lines of code, that could probably be reduced if I didn't want to automatically install the files into a location relative to the client package.

In any case, we're pretty far off topic here; I'm just giving another reason why it would be nice to have the Binaries class of providers enabled on Linux. Regardless of the implementation details of what I intend to use the Binaries class of providers for, I think we can all agree that we should update and then merge this PR, yes?

[tkelman]

I still have the same opinion about this that I did in February.

[tkelman]

BinDeps compatibility may have been a small amount of code in that package, but the combination doesn't serve much purpose that one or the other on their own can't accomplish. There's nothing preventing people from using a mix of both in the same build script if they want to if they're both standalone, so why integrate them?

[ararslan]

I went to great lengths to package redistributable binaries for Linux in https://github.com/ararslan/fftw-builder and we finally figured out why FFTW.jl won't use them: exactly what this PR fixes. Can we please do this? Not being able to use the packaged libfftw binaries on Linux is INCREDIBLY frustrating. System package managers don't all have the same version of FFTW and not every Linux user has the necessary tools to compile it from source, so what else are we supposed to do in that case?

People packaging and redistributing things that won't work everywhere isn't our problem, it's theirs. If their users have issues installing, they'll file an issue against the package.

[ararslan]

I can't fix the conflicts here so I've set up a new PR, #326, that retains Lanfeng's authorship.