Mechanism to vendor binaries

[nipunn1313]

We would love to vendor binaries normally installed by cargo install X that we use as developer tools (eg. sccache, cargo-vendor, etc).

cargo install will correctly respect the vendor directory with the appropriate .cargo/config, which is great.

However, we're not sure how to get cargo vendor to locate all the right items to vendor.
Our best idea was to have a dummy Cargo.toml with "dependencies" on the crates of the binaries (eg cargo-vendor and sccache). However, this means that dev-dependencies of the binaries don't end up in the vendor directory (eg assert_cli is a dev-dependency of sccache), which leads to cargo install failing.

Furthermore, it would be great if we could vendor in such a way that it would work cross platform. Eg if we vendor on a mac machine, the vendor directory would contain all dependent crates so that it could compile on windows as well.

Is there a way to do this today? Thanks!

[alexcrichton]

Hm an intersting use case! I think this wouldn't be too hard to do actually. I suspect we could add a new option like cargo vendor --cratesio-crate foo which would vendor foo and all its dependencies. We could then similarly add like a --cratesio-req '^ 0.3.3' to allow specifying version constraints as well.

I think the implementation would look very similar to cargo install itself, basically just generating a resolution graph for a remotely specified crate.

[nipunn1313]

That sounds great. It would be nice to be able to vendor a workspace as well as any binaries we want to use within that workspace into one vendor directory (to save some dep redundancy).

The args you provided sound pretty reasonable as "additional reqs" on top of the Cargo.toml being vendored.

[nipunn1313]

For example, in our current setup, we need cargo-vendor and sccache installable as developer tools. Ideally this would be hermetic and not require download from the internet.

Our current workaround:
We clone a rev of cargo-vendor in one directory and rev of sccache in another directory.
We vendor each of those separately (cargo-vendor/vendor and sccache/vendor). This results in a lot of redundant vendored files.

We can't add the tools into one big workspace because workspaces share Cargo.lock. We would like to add the tools into one giant vendor.

crates-io mirroring rather than vendoring will ultimately solve this problem for us, but it would be elegant if cargo-vendor also provided a way.

[alexcrichton]

Oh yeah that wouldn't be too hard I think, we could do something like accept --cratesio-crate arguments as well as --sync arguments so you pass in a bunch of manifests and a bunch of crates, and the vendor dir contains everything necessary for all that on the command line.

[nipunn1313]

That sounds great!

[luser]

We can't add the tools into one big workspace because workspaces share Cargo.lock. We would like to add the tools into one giant vendor.

FYI Alex added support to cargo-vendor for precisely this use case. We weren't able to use a cargo workspace for Firefox until recently, so we had several lockfiles and passed them all to cargo vendor at once.

[alexcrichton]

This crate is now included natively in Cargo as of rust-lang/cargo#6869 and today's nightly Rust. I'm closing all issues in this repository before I archive the repository, but if you're still interested in seeing this issue fixed then please feel free to file an issue with Cargo itself!