Huge performance regression when evaluating clean flake

[Kha]

Describe the bug

On a clean checkout of a sizeable repo (~20k commits), the time for the first evaluation of a commit has recently increased by more than 10x (and an unhelpful message from Git is shown):

$ nix build  # cache build

$ echo >> flake.nix; git commit -am foo
[fvis 296fea547d] foo
 1 file changed, 1 insertion(+)

$ nix shell github:NixOS/nix/90b2dd570cbd8313a8cf45b3cf66ddef2bb06e07 -c sh -c 'nix --version; time nix build'
nix (Nix) 2.4pre20210902_90b2dd5
2.10user 0.59system 0:02.60elapsed 103%CPU (0avgtext+0avgdata 287372maxresident)k
0inputs+402536outputs (0major+115786minor)pagefaults 0swaps

$ echo >> flake.nix; git commit -am foo
[fvis 07a735a0b9] foo
 1 file changed, 1 insertion(+)

$ nix shell github:NixOS/nix/6678e98411cd3bfe8143a4ba1b35d1451babce32 -c sh -c 'nix --version; time nix build'
nix (Nix) 2.4pre20210907_6678e98
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: 	git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: 	git branch -m <name>
94.89user 2.88system 0:35.60elapsed 274%CPU (0avgtext+0avgdata 1234872maxresident)k
264inputs+402224outputs (1major+488572minor)pagefaults 0swaps

This is due to #4922, which enables cloning submodules by default. But the submodules option is extremely inefficient, even if the repo does not even contain any submodules.

I know there has been some discussion on the PR itself after merging, but I felt that this regression deserved its own issue due to its severity.

Steps To Reproduce

git clone https://github.com/leanprover/lean4
cd lean4
nix flake prefetch

Expected behavior

A much faster store copy, as before

nix-env --version output

nix (Nix) 2.4pre20210907_6678e98

[Kha]

Also note this comment from the original PR introducing submodules:

This PR does not change any of the fetchGit behavior when submodules are not used (the default), so only people using this feature are paying these costs.

[edolstra]

Hm, maybe we should revert #4922 for now and revisit the submodules issue after 2.4 is released.

[Kha]

Yes, that would probably be for the best. Ironically I updated Nix specifically so I could add a (non-critical) submodule to this flake, so I'm interested in seeing this resolved properly eventually, but the current state is not really tenable.

[edolstra]

Closing this because #4922 was reverted.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/tweag-nix-dev-update-18/15300/1