flake inputs cannot follow "self"

[blaggacao]

───────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: flake.nix
───────┼────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ {
   2 ~ │   description = "Test Flake For Follows";
   3 ~ │   inputs =
   4 ~ │     {
   5 ~ │       nixos-generators.url = "github:nix-community/nixos-generators";
   6 ~ │       nixos-generators.inputs.nixpkgs.follows = "self";
   7   │     };
   8   │
   9 ~ │   outputs = { ... }: {};
  10   │ }
───────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────

nix flake lock --update-input nixos-generators
warning: Git tree '/tmp/flake-test' is dirty
error: input 'nixos-generators/nixpkgs' follows a non-existent input 'self'
(use '--show-trace' to show detailed location information)

Imagine I wanted to ship a remote flake that depends on my local flake as part of my local flake ("engulfing").

With this bug present, I cant do that in flake interface land.

It is a legal operation.

[roberth]

This is a valid improvement. I see two use cases:

Plugin

This situation occurs when flake A has a plugin interface and flake B defines a plugin. The following dependencies arise:

A.lib <- B.plugin <- A.wrappedExe

Here A must be able to specify that B.inputs.A equals A aka follows self.

Megarepo

Another use case is "megarepos": repositories that define multiple related packages. Here the repo boundary is usually the same as the (human) organizational boundary. Sometimes a "high-level" package in the megarepo will depend on an external package that in turn depends on a "low-level" package that is in the same megarepo.

[RealityAnomaly]

This will require a little more thought I think, because this is technically a circular reference.

When a flake's input is being locked, it requires a reference to said input in the store; and that must be a locked reference. So you can't actually pass the current flake self in because it hasn't been evaluated yet, and to evaluate it the inputs must be locked which now depend on the current flake... you can see where this is going 😄

[roberth]

@citadelcore

When a flake's input is being locked, it requires a reference to said input in the store; and that must be a locked reference.

This isn't the case. If you look at call-flake.nix or flake-compat, you'll see that Nix takes the lockfile and connects all the inputs with mapAttrs and looking up the nodes lazily. It does not need to access dependency lockfiles during evaluation, because the transitive dependencies are also in the lockfile.

So all recursion happens in the Nix language, which is lazy, so we have no problem tying the recursive knot.

[RealityAnomaly]

Hmm, you make a good point. I'll investigate this a bit further.

[stale]

I marked this as stale due to inactivity. → More info

[alexshpilkin]

Not stale.

[thufschmitt]

This is actually possible (there’s even an example in the manual, although it’s not entirely obvious) using an empty string ( or /) in the follows. So nixos-generators.inputs.nixpkgs.follows = ""; should work in the original example

[alexshpilkin]

@thufschmitt Empty path means this flake? Whoa. That is... not how paths usually work, but I guess there is no real reason to reject that, either. What about keeping that interpretation but also making “self” components no-ops, like “.” components in filesystem paths?

[edolstra]

Given that follows takes a slash-separated path of flake input names, I think / is the most natural way to refer to the root, i.e.

nixos-generators.inputs.nixpkgs.follows = "/";

The empty string works sort of by accident, so maybe we should disallow it.

Note however that following the root flake currently does not work in pure mode since it's not locked in the lock file. So you'll get an error like:

error: in pure evaluation mode, 'fetchTarball' requires a 'sha256' argument

[thufschmitt]

Given that follows takes a slash-separated path of flake input names, I think / is the most natural way to refer to the root

Just my 2cts as to the design of the bikeshed: / is a bit surprising in that context given that it evokes an absolute path while all the flakerefs are (usually, this isn’t mandatory) written as what would syntactically be a relative path. So maybe . would be a more intuitive choice for that :)

(But I don’t have a strong opinion on that either, as long as it’s clearly documented)

[sellout]

I guess this was closed because the answer is “use ""”, which is what I have been doing for ages, and it works fine.

However, I got here because I was wondering how you can do something like inputs.foo.inputs.bar.follows = null – effectively saying “we don’t use the part of the flake that references this input, so don’t add it to the lock file.”

A likely response is “that is a bad idea – don’t do that,” which I’m on board with, but flakes still make it happen by using "" to replace arbitrary inputs with meaningless “self” references (e.g., https://github.com/nix-community/crate2nix/blob/151122427d030874ebef3517cda766a6984e6ed6/flake.nix#L39-L46). Maybe it’s simple confusion, because having "" be a self-reference isn’t an obvious consequence.

tl;dr

So, I think using .follows = "self" is the best way forward¹. Someone reading .follows = "self" can infer where it points and self is also the name provided to the flake outputs function (which, I assume, is where the OP got the idea from).

Footnotes

1. And make .follows = "" a warning – although I still don’t have a solution for .follows = null. ↩