Improve multi-crate project management

[alexcrichton]

Cargo's story around having a project (or a repo) with many crates isn't the best right now, there are a number of papercuts that add up over time:

• Each crate gets its own Cargo.lock, which means that each crate can be working with a different set of dependencies. Not only is it difficult to manage all these lockfiles but it's often not desired to have different sets of dependencies when working in the source tree.
• By default crates have distinct target folders, meaning crates are recompiled as you move around the project tree.
• Currently you can't run cargo test -p foo for a crate that has a dev-dependency Dev-dependencies are not linked when testing subpackages #860
• "test everything" isn't an easily accessible command

I'll likely add more over time!

[icorderi]

I keep running into needing some project level cargo for large projects.

I was thinking if something like this would help:

[project]
name = "theproject"
version = "0.1.0"
authors = ["people"]
readme = "README.md"
description = "..."

[crates]
onecrate = "0.1"
another = { git = "https://github.com/some/dude" }
relative = { path="./some/other" } # weird, but might happen

# feature crates
uhg = { optional = true }

[features]
nightly = ["uhg"]

Having the special [project] instead of [package] root, tells cargo to deal with this .toml differently.

1. There are no [lib] or [bin]/[[bin]] entries
2. cargo build would output target/debug/<crate>/...
3. cargo test would test each crate (we could also check for a root level folder test for cross crate tests)
4. cargo run with that .toml as-is would make no sense, but having an entry crate would be helpful for some kinds of projects. Such projects can add entry = onecrate to the project definition.
5. What does cargo publish mean? (a) Recursive (explicit, with publish=true on project, requires all crates to be publishable too), (b) individual crates (implicit+default, except when publish=false on subcrate), (c) not allowed
6. other subtleties...

We can also have a common crate cache, if a bunch of the project crates share the same dependencies there is no need compile them all over and over again.

@alexcrichton what are your thoughts on this? Could we get some traction to get "project" support on cargo?

cc: @jsgf

[alexcrichton]

Thanks for the comment @icorderi! I've been thinking about this recently and I hope to soon formulate what I'm thinking into an RFC. I suspect that this will basically manifest itself in somehow linking together Cargo.toml files into one common "parent".

To me most commands would continue to basically just work "as-is" today without many changes. Grouping projects together would primarily:

• Share one output directory (dependencies aren't recompiled)
• Share one Cargo.lock reducing divergence across directories and alleviating registry updates
• Fixing testing transitive path dependencies by storing more information in the lock file

I think that some operations that are out of scope in the short term are:

• cargo publish --all
• cargo test --all
• cargo bench --all
• etc

Basically operations that work over an entire tree probably want to be held off for now. Perhaps one day we can add such support but for now it may be premature and just bog down the design.

[icorderi]

@alexcrichton looking forward to that RFC.

[alexcrichton]

Ok, I've got a draft now, yay! @icorderi curious as to your thoughts on -- https://github.com/alexcrichton/rfcs/blob/cargo-workspace/text/0000-cargo-workspace.md?

[alexcrichton]

er, wrong RFC, fixed the link

[alexcrichton]

I've now opened an RFC for this issue.

[alexbool]

Any good news? :)

[alexcrichton]

Ah oops, I forgot to link the implementation PR! I have an implementation here: #2759

[alexbool]

Great!