Cargo dies when in worktree of bare repository

[maurer]

Problem
When building rustc from a downloaded tarball (in this case, 1.40.0) that is inside a worktree of a bare git repo, Cargo will notice the .git directory and attempt to list files and die while trying.

Steps

1. Download a Rust source tarball
2. Unpack it and check it into a git repository
3. Create a bare clone of that git repository
4. Use "git worktree" to create a checkout of the bare git repository.
5. Attempt to build rustc (e.g. ./x.py install)

(This obviously isn't minimized, but is the original environment in which I encountered the problem. I expect the problem would occur when trying to use a path source in any worktree.)

Possible Solution(s)

• Use list-tree based on the current worktree rather than list-files
• Fall back to non-git base path sources in the event of a bare repo

Notes
This is related to #4131 but actually caused our CI to start failing because the backend switched to bare repos from normal clones to save disk.

I created a separate issue because it is not restricted to cargo publish and is not just a workflow issue.

Output of cargo version: cargo 1.40.0-dev

[ehuss]

I can't seem to figure out how to reproduce this. Can you put together an exact series of commands to run to repro it? I tried:

cargo new foo
cd foo
# Create build.rs so it will list git files.
echo "fn main() {}" > build.rs
git add * .gitignore
git commit -m initial
cd ..
git clone --bare foo foo.bare
cd foo.bare
git worktree add master
cd master
cargo build
cargo package

I also tried something similar with the rustc-src tarball.

[jynnantonix]

I am hitting this as well. Why on earth does a build system care about the version control I'm using?!?!

[chachi]

This also seems to fail when doing a sparse checkout with recent git (used to cut down cloned repo size). You'll need something like git v2.27 to run the following.

Here's a repro with this repo itself:

git clone --sparse --filter=blob:none https://github.com/rust-lang/cargo/
cd cargo
git sparse-checkout add src crates
cargo +nightly check

I get the output

Caused by:
  failed to determine the most recently modified file in /home/jack/src/cargo

Caused by:
  failed to determine list of files in /home/jack/src/cargo

Caused by:
  did not expect repo at /home/jack/src/cargo/.git/ to be bare

Even though the repo is not bare, (git rev-parse --is-bare-repository prints false).

It seems libgit2 is not quite sparse checkout supporting yet (libgit2/libgit2#2263) but it'd be stellar if Cargo could just notice that the Cargo.toml file actually exists right there anyways.

[eminence]

I'm not able to reproduce the original worktree issue with the reported versions, but I did find that cargo 1.46 and 1.47 would give an error about "did not expect repo at XYZ to be bare" when run in a worktree. I also was able to get the same error with a sparse checkout, as reported by @chachi. I'm pretty sure the upgrade of git2 in #8778 is what made things work in 1.48.

I haven't found any issues with the latest cargo using worktrees or spase checkouts

Possible same root cause as #4131

[weihanglo]

Thanks @eminence for the investigation.

Given that this might be fixed already, I'd suggest adding tests for them (#7876 and #4131).

If you need git CLI to create such an environment, use #[cargo_test(requires_git)] to require the existence of git. You can check this test as a reference.

[LuuuXXX]

@rustbot claim