Allow dependencies to build binaries

[lilyball]

I have a desire for a cargo package to build a binary even when it's used as a dependency. This binary is expected to be used by the client package as part of the build (more specifically, in this case, it's meant to be used during installation of the client package, although I can imagine other packages wanting the binary to be available to the client package's custom build command).

[huonw]

I imagine this is highly related to rust-lang/rfcs#403.

[alexcrichton]

Given the contents of the RFC @huonw linked to, could you rephrase some of your use cases @kballard? In general the concept of installation is something that Cargo doesn't touch on at this time, and I'd otherwise be under the impression that the RFC handles your use cases, but I just want to make sure!

[lilyball]

I get that installation is something Cargo doesn't touch on at this time, but I can imagine other use-cases for building binaries for use by dependent libraries. For example, I might write a package that works like Thrift, bundling a library together with a command-line tool that's used by the dependent project to compile a specification file. The dependent library would need to be able to request that my library build its binary, and then use the binary in the build command.

I've started reading the RFC, but I have an appointment to go to. When I get back, I'll read it and comment as to whether I think it solves my request.

[lilyball]

The RFC looks promising in general, but it doesn't seem to solve the case where a package needs to use a binary that a dependency builds. I think we could tweak dependency definitions to support

[dependency.foo]
git = "https://github.com/user/repo"
include_bin = true

to have it build any specified binaries in addition to the library. The built binaries would go somewhere in target (either at the top level, or in a path specific to the dependency, but it has to be predictable by a build script), and would be compiled before any build command for the dependent package is executed.

I think this would work today, and I think it would work equally well with rust-lang/rfcs#403.

In my particular case, I don't actually need it to be made available before build, because I want to drive the think with a Makefile that runs cargo build and then executes the tool from the dependency as part of the install, but the example I gave before of a Thrift-like library would definitely need its binary to be made available to the build script of any dependent packages.

[alexcrichton]

With the build overhaul RFC, I'm not sure that expressing a need for binaries is really that necessary in dependencies. You'll have a clear set of "build dependencies" as they'll be listed in the build-dependencies section of the manifest, and you'll be running rust code as the build script in which case it's far easier to call other rust code than it is to spawn a process (both in terms of overhead and ergonomics).

If the build script were a shell script or a makefile, then we'd certainly want something like this, but because you can just do extern crate to pull in any dependency then I would say that if this is ever wanted then the library should expose a more ergonomic interface to execute its functionality.

@kballard, could you elaborate a little more on why you'd want a binary to be available for a rust build script as opposed to the library itself?

[lilyball]

@alexcrichton Well actually, I would like the capability of bundling non-Rust scripts for this same purpose. I really only started writing this executable as a Rust binary because there wasn't any way of providing a shell script to do the same thing.

Also, more generally, in my project I'm driving cargo with a Makefile, because I need to be able to provide make install. So the Makefile is what really needs to be able to execute the binary provided by the dependency (whether it's a binary written in Rust, or a shell script bundled in the repo).

[alexcrichton]

Why do you want to include non-rust build scripts? Moving forward cargo will be deprecating support for a non-rust build script and it will likely be removed in the future.

Perhaps if I gave an example it would make a little more sense? Do you have some more details about your specific use case so I could spell out what I'm thinking in a concrete sense?

[alexcrichton]

Now that #792 has landed I'm going to close this in favor of that. I'd like to get some experience with using build scripts before pushing forward with ensuring binaries are available in dependencies for downstream packages. We can certainly reevaluate if it's not sufficient though!