Cargo vendor doesn't include dependency's hidden files

[joshuachp]

Problem

Vendoring a dependency ignores hidden files present in the published crate and that are required to build it.

Steps

1. Create a library dependency or crate that include_str!(".hidden") (e.g. test-dep-cargo-hidden)
2. This crate need to be published on a registry, the crate will include the .hidden file
3. Create another crate that uses the published dependency
4. Use cargo vendor and in the vendor directory the .hidden file is missing
5. Building with the vendor will fail, while normal builds will succeed

Possible Solution(s)

Vendoring should probably include all the files of the published package.

I believe those line caused this discrepancy:

cargo/src/cargo/sources/path.rs

Lines 159 to 162 in 3b43aa8

	if no_include_option && git_repo.is_none() {
	// no include option and not git repo discovered (see rust-lang/cargo#7183).
	exclude_builder.add_line(None, ".*")?;
	}

This will cause the publish command to list all the files in the git repo of the dependency, while the vendor will use the sparse cache, which is not a git repository, to copy the files.

Notes

An example of this is having a dependency using sqlx offline files that are stored in the .sqlx/ directory.

Relevant documentation, issues and PRs I found:

• https://doc.rust-lang.org/cargo/reference/manifest.html#the-exclude-and-include-fields
• cargo package excludes hidden files that are explicitly included #7183
• include dotfiles in packages #7680

Version

cargo 1.77.0 (3fe68eabf 2024-02-29)
release: 1.77.0
commit-hash: 3fe68eabf93cbf3772bbcad09a9206c783e2de3f
commit-date: 2024-02-29
host: x86_64-unknown-linux-gnu
libgit2: 1.7.2 (sys:0.18.2 vendored)
libcurl: 8.5.0-DEV (sys:0.4.70+curl-8.5.0 vendored ssl:OpenSSL/1.1.1w)
ssl: OpenSSL 1.1.1w  11 Sep 2023
os: NixOS 24.5.0 [64-bit]

[charmitro]

Does cargo vendor include dotfiles (.file) when specified in the package.include field?

Given that dotfiles sometimes contain sensitive information, it would be prudent to include only those explicitly mentioned in the package.include field.

[joshuachp]

Yes it does, but I am talking specifically of a package already published on a registry. The published crate already includes the .files since they were added in the VCS and not in the exclude list, but the vendor doesn't.

[charmitro]

Yes it does, but I am talking specifically of a package already published on a registry. The published crate already includes the .files since they were added in the VCS and not in the exclude list, but the vendor doesn't.

Ah, now I get it. We should be including dotfiles just like publish does, except when they are present in the package.exclude field.