Vendoring from a git repository should include files as if they were published

[glandium]

When we vendor forked versions of crates from git repositories in Firefox, cargo vendor adds plenty of files that wouldn't be there if the crate had been published on crates.io, like CI files, .gitignore, etc. Even Cargo.toml is normalized, and this makes vendoring more noisy than necessary.

See e.g.
https://phabricator.services.mozilla.com/D18767
https://phabricator.services.mozilla.com/D20289

(originally filed as alexcrichton/cargo-vendor#203)

Summary Notes

• normalized-paths-difference-on-windows by weihanglo

Generated by triagebot, see help for how to add more

[stevenroose]

@glandium question: how are you vendoring a git dependency? I can't seem to find what to include in the .cargo/config file to overwrite the git source with the vendored files.. (Even though cargo-vendor actually downloads the dependencies into the folder.)

[glandium]

With [patch.crates-io] in Cargo.toml. See https://phabricator.services.mozilla.com/D18767#change-rFeP59HZ8M0a

[weihanglo]

@rustbot note normalized-paths-difference-on-windows

#14965 is another instance that when

• linking to files outside package root, e.g., license-file = "../LICENSE",
• workspace dependencies are involved,

the path normalizations are different on Windows and Unix.

This might be resolved by vendoring as if they were published. With the help of (-Zworkspace-package this could be easier to achieve. The other way to fix this is choosing one way for normalization, and fixes on it. However, we might need to patch up case by case.

Or we could do both:

• cargo package always normalization in a canonical way, like in Unix path format. This could also improve the story of publishing from different platforms.
• cargo vendor reuses some logic from prepare_archieve in cargo package for git dependencies, so it gets a list of files to copy as if they were from an archive.

[glandium]

#15736 is another instance of a change that triggered changes in vendored Cargo.toml for git repositories, under package.metadata. It changed the ordering of the package.metadata.* sections and the order of the fields under those sections. It /seems/ they used to be sorted and aren't anymore.

[glandium]

It /seems/ they used to be sorted and aren't anymore.

Well, considering #15736 switched from BTreeMap to IndexMap, that seems like the right observation.