Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transitive inputs "not in registry" #5790

Open
NobbZ opened this issue Dec 17, 2021 · 8 comments · Fixed by #6621
Open

Transitive inputs "not in registry" #5790

NobbZ opened this issue Dec 17, 2021 · 8 comments · Fixed by #6621

Comments

@NobbZ
Copy link
Contributor

NobbZ commented Dec 17, 2021

Describe the bug

I want to use follows on the input of an input.

Though I get an error, that this is not in the registry.

Steps To Reproduce

  1. Add an arbitrary nixpkgs input to a flake.nix.
  2. Add the following to your flake.nix at appropriate location:
    inputs.rnix-lsp.url = "github:nix-community/rnix-lsp/master";
    inputs.rnix-lsp.inputs.nixpkgs.follows = "nixpkgs";
    inputs.rnix-lsp.inputs.naersk.inputs.nixpkgs.follows = "nixpkgs";
  3. Update the flake.lock and observe error:
    error: cannot find flake 'flake:naersk' in the flake registries
    (use '--show-trace' to show detailed location information)
    

Expected behavior

The nixpkgs input of the naersk input of rnix-lsp is properly overwritten and follows "my" nixpkgs.

nix-env --version output

$ nix-env --version
nix-env (Nix) 2.6.0pre20211214_18e4851

Additional context

I tried this as a way to workaround #5609.

The same problem happens on 2.4 and 2.5:

$ nix shell github:nixos/nix/2.4 -c nix flake metadata
warning: Git tree '/home/…/Projects/system-config' is dirty
error: cannot find flake 'flake:naersk' in the flake registries
(use '--show-trace' to show detailed location information)
$ nix shell github:nixos/nix/2.5.0 -c nix flake metadata
warning: Git tree '/home/…/Projects/system-config' is dirty
error: cannot find flake 'flake:naersk' in the flake registries
(use '--show-trace' to show detailed location information)
@thufschmitt
Copy link
Member

I can indeed reproduce this. More self-contained repro:

#!/usr/bin/env bash

set -euo pipefail
set -x

rm -rf A B C D
mkdir -p A B C D

cat <<EOF > A/flake.nix
{ outputs = _: {}; }
EOF
cat <<EOF > B/flake.nix
{
  inputs.A.url = "path:$PWD/A";
  outputs = _: {};
}
EOF
cat <<EOF > C/flake.nix
{
  inputs.B.url = "path:$PWD/B";
  outputs = _: {};
}
EOF
cat <<EOF > D/flake.nix
{
  inputs.C.url = "path:$PWD/C";
  inputs.A.url = "path:$PWD/A";
  inputs.C.inputs.B.inputs.A.follows = "A";
  outputs = _: {};
}
EOF

nix flake update ./D

@nixos-discourse
Copy link

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-should-i-setup-nested-flake-follows-to-use-top-level-nixpkgs/12670/4

@jali-clarke
Copy link
Member

may have been fixed in #6036

@tejing1
Copy link

tejing1 commented Feb 13, 2022

Nope. Running thuf's script still fails:

$ nix shell github:NixOS/nix\#nix
$ ./tempscript
+ rm -rf A B C D
+ mkdir -p A B C D
+ cat
+ cat
+ cat
+ cat
+ nix flake update ./D
error: cannot find flake 'flake:B' in the flake registries
(use '--show-trace' to show detailed location information)
$ nix --version
nix (Nix) 2.7.0pre20220211_4d67ecb

@tomberek
Copy link
Contributor

tomberek commented Jun 2, 2022

The desired change to the resulting lockfile for the relevant "follows" line,

inputs.C.inputs.B.inputs.A.follows = "A";

should cause this:

diff --git a/flake.lock b/flake.lock.post
index 7d50628..c51a7d1 100644
--- a/flake.lock
+++ b/flake.lock.post
@@ -26,7 +26,7 @@
     },
     "B": {
       "inputs": {
-        "A": "A_2"
+        "A": ["A"]
       },
       "locked": {
         "lastModified": 1654174365,

I've tried a few combination such as bringing B to the top and they trying to override it:

{
  inputs.B.follows = "C/B";
  inputs.B.inputs.A.follows = "A";
  outputs = _: {};
}

but then the relevant line to follow A has no effect.

What does work is to bring everything to the top level:

{
  inputs.A.follows = "C/B/A";
  inputs.B.follows = "C/B";
  outputs = _: {};
}

This can work in some cases if all you need is top level access to those inputs, but it does not allow you to change them, which is where people run into this problem.

Ma27 added a commit to Ma27/nix that referenced this issue Jun 13, 2022
I recently got fairly confused why the following expression didn't have
any effect

    {
      description = "Foobar";
      inputs.sops-nix = {
        url = github:mic92/sops-nix;
        inputs.nixpkgs_22_05.follows = "nixpkgs";
      };
    }

until I found out that the input was called `nixpkgs-22_05` (please note
the dash vs. underscore).

IMHO it's not a good idea to not throw an error in that case and
probably leave end-users rather confused, so I implemented a small check
for that which basically checks whether `follows`-declaration from
overrides actually have corresponding inputs in the transitive flake.

In fact this was done by accident already in our own test-suite where
the removal of a `follows` was apparently forgotten[1].

Since the key of the `std::map` that holds the `overrides` is a vector
and we have to find the last element of each vector (i.e. the override)
this has to be done with a for loop in O(n) complexity with `n` being
the total amount of overrides (which shouldn't be that large though).

Please note that this doesn't work with nested expressions, i.e.

  inputs.fenix.inputs.nixpkgs.follows = "...";

which is a known problem[2].

[1] 2664a21
[2] NixOS#5790
Ma27 added a commit to Ma27/nix that referenced this issue Jun 13, 2022
I recently got fairly confused why the following expression didn't have
any effect

    {
      description = "Foobar";
      inputs.sops-nix = {
        url = github:mic92/sops-nix;
        inputs.nixpkgs_22_05.follows = "nixpkgs";
      };
    }

until I found out that the input was called `nixpkgs-22_05` (please note
the dash vs. underscore).

IMHO it's not a good idea to not throw an error in that case and
probably leave end-users rather confused, so I implemented a small check
for that which basically checks whether `follows`-declaration from
overrides actually have corresponding inputs in the transitive flake.

In fact this was done by accident already in our own test-suite where
the removal of a `follows` was apparently forgotten[1].

Since the key of the `std::map` that holds the `overrides` is a vector
and we have to find the last element of each vector (i.e. the override)
this has to be done with a for loop in O(n) complexity with `n` being
the total amount of overrides (which shouldn't be that large though).

Please note that this doesn't work with nested expressions, i.e.

    inputs.fenix.inputs.nixpkgs.follows = "...";

which is a known problem[2].

For the expression demonstrated above, an error like this will be
thrown:

    error: sops-nix has a `follows'-declaration for a non-existant input nixpkgs_22_05!

[1] 2664a21
[2] NixOS#5790
Ma27 added a commit to Ma27/nix that referenced this issue Jul 12, 2022
I recently got fairly confused why the following expression didn't have
any effect

    {
      description = "Foobar";
      inputs.sops-nix = {
        url = github:mic92/sops-nix;
        inputs.nixpkgs_22_05.follows = "nixpkgs";
      };
    }

until I found out that the input was called `nixpkgs-22_05` (please note
the dash vs. underscore).

IMHO it's not a good idea to not throw an error in that case and
probably leave end-users rather confused, so I implemented a small check
for that which basically checks whether `follows`-declaration from
overrides actually have corresponding inputs in the transitive flake.

In fact this was done by accident already in our own test-suite where
the removal of a `follows` was apparently forgotten[1].

Since the key of the `std::map` that holds the `overrides` is a vector
and we have to find the last element of each vector (i.e. the override)
this has to be done with a for loop in O(n) complexity with `n` being
the total amount of overrides (which shouldn't be that large though).

Please note that this doesn't work with nested expressions, i.e.

    inputs.fenix.inputs.nixpkgs.follows = "...";

which is a known problem[2].

For the expression demonstrated above, an error like this will be
thrown:

    error: sops-nix has a `follows'-declaration for a non-existant input nixpkgs_22_05!

[1] 2664a21
[2] NixOS#5790
@deemp
Copy link

deemp commented Nov 15, 2022

Perhaps this issue can be reopened as #6621 was reverted in favor of #6530

@azazel75
Copy link

Have you found any work-around for this?

@deemp
Copy link

deemp commented Jul 11, 2023

@azazel75, yes, I now use flake-compat and import + custom logic for making inputs and outputs. I reduced flake.lock sizes significantly and made flakes very flexible, but now have to write a bit of boilerplate and can't nix flake archive to cache flake inputs transitively.

https://github.com/deemp/flakes/blob/main/flake.nix
https://github.com/deemp/flakes/blob/main/flakes-tools/flake.nix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

8 participants