Patch git dependency in a workspace or sub-directory

[ratijas]

Describe the problem you are trying to solve

Patch syntax allows replacing a dependency's git URL, but it doesn't seem allow to specify a directory in that repository. At least, it is not documented at https://doc.rust-lang.org/cargo/reference/overriding-dependencies.html. Essentially, it is useless when patched crate is a part of a cargo workspace.

Describe the solution you'd like

Two solutions:

More integrated with cargo/rust: specify workspace member's package name, so that cargo will parse manifest and resolve to the required directory automatically.

More general: manually specify sub-directory of a git repository, so that cargo would look for a crate's Cargo.toml manifest in that dir.

Notes

I think, both proposed solutions are fine and can be implemented concurrently and without interfering each other.

Related: #6126

[ratijas]

Because of this lack of functionality, I am currently forced to download the whole repo, store it somewhere, and override each dependency of a workspace in a patch section like this:

[patch.crates-io]
cpp        = { path = "../../rust-cpp/cpp" }
cpp_common = { path = "../../rust-cpp/cpp_common" }
cpp_macros = { path = "../../rust-cpp/cpp_macros" }
cpp_build  = { path = "../../rust-cpp/cpp_build" }

Instead, I'd like to be able to write something like this:

[patch.crates-io]
cpp        = { git = "https://github.com/mystor/rust-cpp", member = "cpp" }
cpp_common = { git = "https://github.com/mystor/rust-cpp", member = "cpp_common" }
cpp_macros = { git = "https://github.com/mystor/rust-cpp", member = "cpp_macros" }
cpp_build  = { git = "https://github.com/mystor/rust-cpp", member = "cpp_build" }

[ehuss]

Cargo should automatically find the package based on the name. That is, you should be able to write:

[patch.crates-io]
cpp        = { git = "https://github.com/mystor/rust-cpp" }
cpp_common = { git = "https://github.com/mystor/rust-cpp" }
cpp_macros = { git = "https://github.com/mystor/rust-cpp" }
cpp_build  = { git = "https://github.com/mystor/rust-cpp" }

Does that work for you?

[ratijas]

Works on Linux now. How strange. I'm almost sure I tried exactly that, and it didn't work yesterday on Windows. I'll check Windows again soon.

[ratijas]

Nevermind my stupidity. git patches work with workspaces on Windows too. Anyway, it probably worth mentioning in the documentation that workspaces are explicitly supported.

[ratijas]

However, path variant does not support workspaces. It requires a full path to the final directory with a specific crate.

No good:

qttypes          = { path = "../../qmetaobject-rs" }
qmetaobject      = { path = "../../qmetaobject-rs" }
qmetaobject_impl = { path = "../../qmetaobject-rs" }

Good:

qttypes          = { path = "../../qmetaobject-rs/qttypes" }
qmetaobject      = { path = "../../qmetaobject-rs/qmetaobject" }
qmetaobject_impl = { path = "../../qmetaobject-rs/qmetaobject_impl" }

[ehuss]

I think it is unlikely that we will change the path dependency to do a recursive search. Is there a problem with specifying the path where the package is located?

[ratijas]

Is being inconvenient and inconsistent with `git = ` counts as a problem?Besides, the whole thing is poorly documented (no examples for workspaces, no mentions of differences between git and path), and that definitely should be improved.

[weihanglo]

If anyone wants to give a hand with this, here are some pointers for the possible enhancements:

• Under Specifying dependencies from git repositories section, explain the auto-discovery functionality of git dependencies, such as

  Note that the package you depends on isn't required to be under the root of git dependency. Cargo is smart enough to traverse the repository and find the package matching the name specified in Cargo.toml. That said, Cargo will be confused if there are multiple package with the same name in a git repository.

  And if not overkill, probably provide a simple file tree graph to help understand that.

• Under Specifying path dependencies, we could add something like

  Unlike git dependencies, Cargo doesn't do any traversal for path dependencies. You need to specify the path to the root of the package you depends on.

I am not good at writing so feel free to sumbit a PR with better wordings than this!

[wabisabia]

If no one else is working on this, I'd be happy to make this change!

[wabisabia]

@weihanglo I've noticed that when duplicate crates are discovered in a git repository, Cargo actually selects one of them, and a "Skipping duplicate package ..." warning is issued for each skipped crate.

It seems that the eventually selected crate is the one that was discovered first, but I'm not certain what determines the discovery order:

cargo/src/cargo/ops/cargo_read_manifest.rs

Lines 181 to 197 in c9d8c28

	match all_packages.entry(pkg_id) {
	Entry::Vacant(v) => {
	v.insert(pkg);
	}
	Entry::Occupied(_) => {
	// We can assume a package with publish = false isn't intended to be seen
	// by users so we can hide the warning about those since the user is unlikely
	// to care about those cases.
	if pkg.publish().is_none() {
	let _ = config.shell().warn(format!(
	"skipping duplicate package `{}` found at `{}`",
	pkg.name(),
	path.display()
	));
	}
	}
	}

Would you have any guidance on whether/how this behaviour should be documented?

[weihanglo]

Hi, @wabisabia. Sorry for the late response. I don't know why it is off my radar.

We can skip the skipping behavior for now. It's not reliable and is considered as a bug, see #10752 (comment). If you come up with any better idea or question, please share with us!

[rimutaka]

If this is still relevant I'd like to submit a PR.

Pages in question:

• https://doc.rust-lang.org/nightly/cargo/reference/specifying-dependencies.html#specifying-dependencies-from-git-repositories
• https://doc.rust-lang.org/cargo/reference/overriding-dependencies.html