Undesired git fetch of path flake inputs that point at git submodules

[mightyiam]

It seems that path inputs that happen to point at git submodules trigger git fetching. And I don't think that's desirable.

$ git clone https://github.com/mightyiam/infra.git repro
$ git checkout --detach 5a46333c95a9238f1e0b376acf196cb2b7b76f2e`

Notice a flake input home-manager that has a url "path:./forks/home-manager".

At that path is a git submodule.

From .gitmodules:

[submodule "forks/home-manager"]
	path = forks/home-manager
	url = ./.

What I expect Nix to do is treat that home-manager input as a local path only.

For example, if that path is empty (pre submodule init) I expect Nix to complain that there is no flake at that path.

Instead, Nix seems to find out that at that path is a git submodule and further find out the url for it and attempt to fetch from there 🤒

$ nix flake show
warning: could not read HEAD ref from repo at '/home/mightyiam/worktrees/repro', using 'master'
error:
       … while fetching the input 'git+file:///home/mightyiam/worktrees/repro?rev=5a46333c95a9238f1e0b376acf196cb2b7b76f2e&submodules=1'

       … while fetching the input 'git+https://github.com/mightyiam/infra.git/?rev=babb4fa136adac892d9b2a300573d914cc2b6baf&submodules=1'

       error: Cannot find Git revision 'babb4fa136adac892d9b2a300573d914cc2b6baf' in ref 'refs/heads/main' of repository 'https://github.com/mightyiam/infra.git/'! Please make sure that the rev exists on the ref you've specified or add allRefs = true; to fetchGit.

Should the attempt to fetch git+https://github.com/mightyiam/infra.git/?rev=babb4fa136adac892d9b2a300573d914cc2b6baf&submodules=1 be considered desired behavior?

Metadata

nix-env (Nix) 2.28.3
nix-env (Nix) 2.29.0

Checklist

• checked latest Nix manual (source)
• checked open bug issues and pull requests for possible duplicates

---

Add 👍 to issues you find important.

[mightyiam]

Removing the path: prefix from the URL of the home-manager input so that it is "./forks/home-manager" seems to avoid the git fetch and result in a more reasonable (albeit long) error:

warning: Git tree '/home/mightyiam/worktrees/infra' is dirty
fatal: '/home/mightyiam/worktrees/infra/forks/home-manager' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
warning: could not read HEAD ref from repo at 'file:///home/mightyiam/worktrees/infra/forks/home-manager', using 'master'
fatal: '/home/mightyiam/worktrees/infra/forks/home-manager' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
warning: could not update mtime for file '"/home/mightyiam/.cache/nix/gitv3/0w4hd6570ygfkhd4f02xv4vkvih5k0k7gwjqwrmywpkpyan5h3fw/refs/heads/master"': changing modification time of "/home/mightyiam/.cache/nix/gitv3/0w4hd6570ygfkhd4f02xv4vkvih5k0k7gwjqwrmywpkpyan5h3fw/refs/heads/master" (using `utimensat`): No such file or directory
fatal: Refusing to point HEAD outside of refs/
warning: could not update cached head 'master' for 'file:///home/mightyiam/worktrees/infra/forks/home-manager'
error:
       … while fetching the input 'git+file:///home/mightyiam/worktrees/infra?submodules=1'

       … while fetching the input 'git+file:///home/mightyiam/worktrees/infra/forks/home-manager?submodules=1'

       error: resolving Git reference 'master': revspec 'master' not found

Was I simply wrong about the path: prefix?

[mightyiam]

Removing the path: prefix from the URL of the home-manager input so that it is "./forks/home-manager" seems to avoid the git fetch and result in a more reasonable (albeit long) error:

Yes, but sorry. This does not seem to have anything to do with the url. It seems that the repo being in a dirty status avoids the git fetching and results in the more expected error.

That does imply that this has to do with evaluation cache?

[mightyiam]

When clean with -vv we get the surprising

$ nix eval .#dpms-all -vv
evaluating file '<nix/derivation-internal.nix>'
evaluating file '/home/mightyiam/worktrees/infra/flake.nix'
using revision 41152dd01ab0426e9be2ff19f3ec7fe68ee6599e of repo '/home/mightyiam/worktrees/infra'
fetching Git repository 'https://github.com/mightyiam/infra.git/'...
error:
       … while fetching the input 'git+file:///home/mightyiam/worktrees/infra?ref=refs/heads/playing-around&rev=41152dd01ab0426e9be2ff19f3ec7fe68ee6599e&submodules=1'

       … while fetching the input 'git+https://github.com/mightyiam/infra.git/?rev=babb4fa136adac892d9b2a300573d914cc2b6baf&submodules=1'

       error: Cannot find Git revision 'babb4fa136adac892d9b2a300573d914cc2b6baf' in ref 'refs/heads/main' of repository 'https://github.com/mightyiam/infra.git/'! Please make sure that the rev exists on the ref you've specified or add allRefs = true; to fetchGit.

And when dirty with -vv we get a reasonable

$ nix eval .#dpms-all -vv
evaluating file '<nix/derivation-internal.nix>'
warning: Git tree '/home/mightyiam/worktrees/infra' is dirty
evaluating file '/home/mightyiam/worktrees/infra/flake.nix'
fatal: '/home/mightyiam/worktrees/infra/forks/home-manager' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
warning: could not read HEAD ref from repo at 'file:///home/mightyiam/worktrees/infra/forks/home-manager', using 'master'
fetching Git repository 'file:///home/mightyiam/worktrees/infra/forks/home-manager'...
fatal: '/home/mightyiam/worktrees/infra/forks/home-manager' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
warning: could not update mtime for file '"/home/mightyiam/.cache/nix/gitv3/0w4hd6570ygfkhd4f02xv4vkvih5k0k7gwjqwrmywpkpyan5h3fw/refs/heads/master"': changing modification time of "/home/mightyiam/.cache/nix/gitv3/0w4hd6570ygfkhd4f02xv4vkvih5k0k7gwjqwrmywpkpyan5h3fw/refs/heads/master" (using `utimensat`): No such file or directory
fatal: Refusing to point HEAD outside of refs/
warning: could not update cached head 'master' for 'file:///home/mightyiam/worktrees/infra/forks/home-manager'
error:
       … while fetching the input 'git+file:///home/mightyiam/worktrees/infra?submodules=1'

       … while fetching the input 'git+file:///home/mightyiam/worktrees/infra/forks/home-manager?submodules=1'

       error: resolving Git reference 'master': revspec 'master' not found

[roberth]

Discussed in Nix team meeting today:

Crux of the problem seems to be the question what is the source of truth?
a. the existence of a submodule in Git
b. the existence of a submodule checkout on the local file system

We're leaning towards (a), because that the more repeatable behavior.

We suspect that this behavior may improve if we ship some sort of lazy trees implementation, as it might not have to copy the submodule at all, if it isn't referenced.

Removing the path: prefix from the URL of the home-manager input so that it is "./forks/home-manager" seems to avoid the git fetch and result in a more reasonable (albeit long) error:

Yes, but sorry. This does not seem to have anything to do with the url. It seems that the repo being in a dirty status avoids the git fetching and results in the more expected error.

That does imply that this has to do with evaluation cache?

Probably not. The local git fetching routine is substantially different from fetching from a remote (or bare) repository. That probably explains the difference.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2025-06-11-nix-team-meeting-minutes-231/65543/1

[mightyiam]

Am I to understand that when a flake input url is a path like ./foo and at that path a git submodule is detected then a git fetcher might be used to fetch it in some cases?

[mightyiam]

Here is more context. The project in which I've rubbed against this behavior. The heading that pertains to this behavior:
https://github.com/mightyiam/input-branches?tab=readme-ov-file#superproject-must-be-dirty-

[bryango]

I am also troubled by this behavior... It appears that when the workspace is clean, nix flake update nixpkgs always tries to re-fetch from the git remote, which may hang for a loooong time with no notice at all. I understand that this is the desired behavior if we aim for correctness.

To balance between convenience and correctness, maybe we could respect the --offline flag in this context: instead of always fetching from the remote, when --offline we only fetch from the local checkout (as if the workspace is dirty). Currently nix flake update --offline will still fetch from the git remotes of submodules.

By the way, I am using:

$ nix --version
nix (Nix) 2.30.2

[maxeonyx]

I have also encountered this issue, and it was a surprising head-basher. I finally figured it out then found this issue. I can summarize the behaviour that I was seeing, which is not about failures:

"Which submodule version is copied to the Nix store depends on the dirty state of the parent repo"

It's extremely intermittent and fickle and easy to accidentally fix and then encounter again.