nix flake update --single <name> is more intuitive than nix flake lock --update

[colemickens]

Why is nix flake update a think instead of just nix flake lock --update-all?

Or why is nix flake lock --update-input not nix flake update --single <name>.

Or is there some other reasoning for the current grouping? I don't get it, I have to look it up every time. It just feels inconistent

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-update-a-single-flake-input/13584/6

[kubukoz]

I strongly agree with OP, if Flakes are supposed to make Nix more approachable we should aim to minimize confusion and optimize the UX for discoverability.

[Kha]

Or even

nix flake update [option...] flake-url [input1 [input2 ...]]

[zyansheep]

might also be nice to have a nix flake update --exclude [input1] for excluding specific inputs that might not be working at that moment.

[brendanzab]

As a Nix beginner who was just wondering why nix flake update didn't let me upgrade specific inputs individually, and had to find out about nix flake lock from the forums, I can confirm this was a frustrating papercut!

[shinzui]

I have to google this command every time I want to update a single input.

[erikarvstedt]

This really needs an UX improvment.
I consider myself a fairly seasoned nix user and I recently wanted to update a single input. This is my story.

• I tried nix flake update --update-input <input>. Didn't work.
• I then read through nix help flake and the flake-related section of nix help build.
• Finally, I settled for nix eval .#randomOutput --raw --apply '_: ""' --update-input nixpkgs-unstable.
  I'd never have associated the lock verb with updating a flake.

I'd recommend the following:

• Support the common flake arg --update-input in nix flake update.
• Remove nix flake lock --recreate-lock-file.
  What's the difference between this and nix flake update?
  If there's a difference, support --recreate-lock-file in nix flake update.
• Delete subcommand lock, replace it with nix update --add-missing.

[TLATER]

I think there's plenty of agreement here, it'll take a PR to get things moving :)

[edolstra]

It just feels inconistent

It isn't.

• nix flake update: Recreates the entire lock file, thus updating everything.
• nix flake lock: Add missing locks, optionally updating some locks if --update-input is given. This is same behaviour as commands like nix build.

See also the discussion in #3781.

[kubukoz]

At the very least it's difficult to find 😅

[robx]

I just ran into this myself. I saw that nix flake update updated my inputs, and it felt very natural to assume I'd be able to restrict that invocation to a single input. In fact, my working solution for the past week was

nix flake update; git add -p; git commit -m "partial flake update"; git reset --hard

until going off on another lengthy search today and ending up at this issue.

I'd suggest it might useful to override any technical consistency of the current approach in favour of a more intuitive UX?

[kubukoz]

I think the following would be more intuitive:

• nix flake update - the main way of updating anything in the lock file. Optional argument (named or positional, up to bikeshedding) for updating a specific input
• nix flake lock - locks the current inputs (all of them) or one of them if --update-input is given. Most likely you'd only ever use this once or never (relying on nix build and others to perform the locking)

The main rationale for keeping single- and multi-input updates under the same command: it's just confusing to everyone that you currently need two.

[dschrempf]

I understand both sides but think the status quo is bad.

I think there are two solutions:

1. Either, we remove nix flake update completely, and use something like nix flake lock --update-all. (The only inconsistency would be the first invocation, because it is not an update).
2. Or we allow nix flake update to act on one input, for example, like so: nix flake update --only bla (or --single).

[thufschmitt]

We discussed that a bit during the last UX meeting.

The broad outcome was that:

• Both commands make sense individually
  • lock as a low-level “just do stuff on the lockfile”
  • update as the very handy “update everything”
• update is morally an alias for lock --recreate-lockfile

One suggestion to reduce the confusion was to simply make the fact that update is merely a shorthand alias explicit in the documentation (and maybe give --recreate-lockfile a better name as it's not entirely obvious what it does)

[ncfavier]

This doesn't seem inconsistent with making update arg1 arg2 an alias for lock --update-input arg1 --update-input arg2.

[thufschmitt]

This doesn't seem inconsistent with making update arg1 arg2 an alias for lock --update-input arg1 --update-input arg2.

It's not indeed. However doing so would be inconsistent with the fact that update arg1 means "update the inputs of the flake arg1" (and update alone is just a shorthand for update .)

[kubukoz]

Honestly, I doubt beginner users will ever want to update any other flake than ..

What's the usecase for nix flake update github:nixos/nixpkgs?

Beginners want the ability to easily update these on their own flake (.):

• all the dependencies
• one particular dependency

I don't think they're gonna want to dig into lockfile-related lock commands to find out how to do that. Just nix update / nix update arg1 should be enough, really.

One suggestion to reduce the confusion was to simply make the fact that update is merely a shorthand alias explicit in the documentation (and maybe give --recreate-lockfile a better name as it's not entirely obvious what it does)

Documentation is good, but if the UX of the tool is confusing, it should be improved at the source, and not just documented. Especially now, while flakes are experimental.

[thufschmitt]

Backwards consistency isn't really the matter here (as you say, it's experimental, we could break things).

I agree that taken individually, this command could make more sense in the form nix flake update input1 input2 .... But that would play at the detriment of global coherence: All the commands that can operate on a flake (so nearly all the commands) follow the same convention nix <command> [flake-uri] [args]. Breaking this convention would be annoying.

If anything I'd rather remove update altogether and recommend something like lock --update-inputs as a replacement (and keep lock --update-input foo --update-input bar). Or maybe have lock --update-inputs foo bar which although a bit longer would have the advantage of preserving consistency and allow a direct list of inputs to update at the same time.

[thufschmitt]

What's the usecase for nix flake update github:nixos/nixpkgs?

That one obviously doesn't make any sense, but nix flake update /some/directory does

[kubukoz]

I don't think lock --update-input helps simplify the UX. How about

• nix flake update-all - does what the current update does
• nix flake update - does what lock --update-input does

[thufschmitt]

nix flake update - does what lock --update-input does

Do you mean that you'd call it as nix flake update input1?

[kubukoz]

nix flake update - does what lock --update-input does

Do you mean that you'd call it as nix flake update input1?

@thufschmitt that's right. I see how It's not going to be consistent with the way of passing the flake reference as a positional argument, but sticking to that for the sake of consistency would make for a less intuitive UX for users.

In my view nix flake update input1 makes sense because positional arguments feel more "important" than named parameters, and the main parameter of "I want to update a single input" is the input you want to update.

Another idea - provide syntax to refer to a flake input. Then it'd be like nix flake update .#inputs.nixpkgs (updating the nixpkgs input of the cwd's flake) and nix flake update . under the same command.

[erikarvstedt]

nix flake currently supports 3 operations for operating on lockfiles, spread over 2 commands.
To improve UX, these should be bundled into one command.

Lockfile operations

1. Update all inputs / recreate the whole lock file
   nix flake update
nix flake lock --recreate-lock-file

2. Update missing inputs
   nix flake lock

3. Update a specific input
   nix flake lock --update-input <input>

Why the current UX is bad

The split into two commands is confusing. Bundling all lockfile operations into one command would lead to a more intuitive and discoverable interface.

Example user story: A user wants to update a specific input of a flake.

• The user runs nix flake --help and is currently shown the following subcommand listing, at the top of the help output:

  nix flake lock - create missing lock file entries
  ...
  nix flake update - update flake lock file
  

• The user recognizes nix flake update as the relevant command and runs nix flake update --help.
• The help shows no info relevant to the task (because only nix flake lock supports updating single inputs), the user is stuck.

Solution: Single command

update is the relevant verb for most lockfile operations, so nix flake update is the overall best fit for the command name.

Suggested output of nix flake --help:

nix flake update - update or create flake lock file

Suggested design of nix flake update:

nix flake update
nix flake update --update-input <input>
nix flake update --add-missing # replaces `nix flake lock`, which is seldomly called on its own

[edolstra]

nix flake update --add-missing # replaces nix flake lock, which is seldomly called on its own

I don't think this is true. It's pretty common to add a new input, where you want to update the lockfile to reflect the new input without upgrading every other input. In the new UI we try to avoid flags that negate what a command does, e.g. a --add-missing flag that negates the "update" part of nix flake update.

I do think nix flake update should be renamed to nix flake upgrade to better reflect what it does (namely upgrade all inputs to the latest version). That avoids the ambiguity of the word "update".

[erikarvstedt]

Nix really should fix the update single input user story which is catastrophically bad right now.
I'm convinced that a single command that makes the default case simple is by far the most user-friendly design.
Maybe we should hold a vote or do user testing on this issue.

--add-missing is no direct negation, it just modifies which inputs are updated.

[NobbZ]

I don't think this is true. It's pretty common to add a new input, where you want to update the lockfile to reflect the new input without upgrading every other input

A nix flake lock is indeed rarely necessary, as one usually adds some output using that input and one does nix build .#thatOutput, in which case nix already implicitely does whatever nix flake lock does.

If I do not want this behaviour I have to explicitely opt out using --no-write-lockfile, and I have to use that flag in many other cases when nix considers a lock file "stale".

[chreekat]

@iFreilicht your suggestion is an improvement on the status quo, so in that regard I think it's fine.

That said, I think "it's inconsistent with any other command" is kind of an anti-argument, given our regard for the status quo. 😅 Taken on its own merits, specifying which flake to update seems like a niche use case (I've never used it), so specifying it as a required argument seems un-ergonomic.

Talk is cheap, though, so please do proceed as you see fit, regardless of my opinions.

[TLATER]

I agree with @chreekat on the positional argument. I think a --flake arg is less confusing, and it's so uncommon to update a different directory that having to specify the . all the time is probably unergonomic to boot.

This is plain unreadable:

nix flake update /path/to/that/flake nixpkgs home-manager flake-utils

I have no idea what this command is doing here. Ok, so I'm updating a flake, and that first thing is a path, seems to be a flake, so given the subcommand it probably updates that. But what the hell are these other args about (especially if they're less common flake inputs than what I assume are the most common three...)?

This either needs an --only to distinguish the inputs we're updating from the target flake or the target flake needs to be omitted. Mixing the meaning of positional args always results in confusing CLIs IMO, and looks bad when reading scripts.

And yes, talk is cheap ;p

[colemickens]

re: @iFreilicht sorry if I've missed something catching up, but did you mean nix profile update there, really? Or was that maybe a typo/brain-slip? As far as I can tell that's the first mention and the profile commands aren't for working with the flake.{nix,lock}.

[iFreilicht]

Yeah after starting to look at the codebase and experimenting a little bit, flake update (and flake lock for that matter) is a bit of an outlier here. For most commands, it is very useful to be able to pass any flakeref. Put for update, not so much, the flake needs to be writable for it to have any effect. You can run nix flake update nixpkgs and it will complete, but it's basically a no-op.

So the only scenario is for a different path, not flake-refs outright, and I'd be very surprised if there were so many people doing that, that nix flake update /path/to/flake as opposed to just, cd /path/to/flake && nix flake update would be an unreasonable burden. The only real reason is explicitly specifying the flake type, like path:., but having a --flake arg as an escape hatch for that seems reasonable. It's only necessary in a few select cases.

[colemickens]

Sorry to wade back in after being absent from this thread, but I have some strong feelings.

1. I think abusing positional arguments is a very unfortunate idea and will be regretted quickly if chosen. I do not like the proposed options that make it very unclear what the args are at a quick glance. I can imagine the exact bugs it will cause right now. Rather than the design including special handling to pre-empt user misuse, we should strive for intuitive, explicit CLI UX.

2. Regarding operating on flakes outside of PWD.... I think this should be solved like Git. Which also has the nice benefit of making pt1 easier to avoid.

cd /this/is/not/a/git/worktree
git clone https://some/flake ~/myflake
git -C ~/myflake rebase -i
nix flake -C ~/myflake lock --recreate-lock-file
nix flake -C ~/myflake lock --override-input nixpkgs ~/nixpkgs_fork
nix flake -C ~/myflake update # doesn't do anything, spits out a warning and usage info
nix flake -C ~/myflake update --all # cmd2
nix flake -C ~/myflake update --input nixpkgs-wayland
nix flake -C ~/myflake update --input nixpkgs --input nixpkgs-wayland

edit: maybe for fun nix flake update could actually do a dry-run and print a summary of what inputs could be updated, and then encourage the user to nix flake update --all or nix flake update --input [one_from_the_summary]

[9999years]

nix flake -C ~/myflake update # doesn't do anything, spits out a warning and usage info

@colemickens Please show me an example from the dozens of existing dependency management tools where the-tool update prints out a warning and usage info instead of, y'know, running the update.

Edit: Here's some concrete examples with the Rust and Python dependency managers.

cargo update

$ cargo update --help
Update dependencies as recorded in the local lock file

Usage: cargo update [OPTIONS]

Options:
  -q, --quiet                 Do not print cargo log messages
  -w, --workspace             Only update the workspace packages
  -p, --package [<SPEC>]      Package to update
      --aggressive            Force updating all dependencies of SPEC as well when used with -p
      --dry-run               Don't actually write the lockfile
      --precise <PRECISE>     Update a single dependency to exactly PRECISE when used with -p
      --manifest-path <PATH>  Path to Cargo.toml
  -h, --help                  Print help
  -v, --verbose...            Use verbose output (-vv very verbose/build.rs output)
      --color <WHEN>          Coloring: auto, always, never
      --frozen                Require Cargo.lock and cache are up to date
      --locked                Require Cargo.lock is up to date
      --offline               Run without accessing the network
      --config <KEY=VALUE>    Override a configuration value
  -Z <FLAG>                   Unstable (nightly-only) flags to Cargo, see 'cargo -Z help' for details

Run `cargo help update` for more detailed information.

poetry update

$ nix run nixpkgs#poetry -- update --help

Description:
  Update the dependencies as according to the pyproject.toml file.

Usage:
  update [options] [--] [<packages>...]

Arguments:
  packages                   The packages to update

Options:
      --without=WITHOUT      The dependency groups to ignore. (multiple values allowed)
      --with=WITH            The optional dependency groups to include. (multiple values allowed)
      --only=ONLY            The only dependency groups to include. (multiple values allowed)
      --no-dev               Do not update the development dependencies. (Deprecated)
      --dry-run              Output the operations but do not execute anything (implicitly enables --verbose).
      --lock                 Do not perform operations (only update the lockfile).
  -h, --help                 Display help for the given command. When no command is given display help for the list command.
  -q, --quiet                Do not output any message.
  -V, --version              Display this application version.
      --ansi                 Force ANSI output.
      --no-ansi              Disable ANSI output.
  -n, --no-interaction       Do not ask any interactive question.
      --no-plugins           Disables plugins.
      --no-cache             Disables Poetry source caches.
  -C, --directory=DIRECTORY  The working directory for the Poetry command (defaults to the current working directory).
  -v|vv|vvv, --verbose       Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug.

Let's not pretend like this is uncharted UX territory here — there's a clear consensus that tool-name update updates the lockfile.

[iFreilicht]

I think abusing positional arguments is a very unfortunate idea and will be regretted quickly if chosen.

Just to be clear, are you talking about mixing the types of positional arguments like I proposed?

$ nix flake update . nixpkgs nixpkgs-wayland

Or are you talking about using positional arguments in general, like @TLATER and @chreekat proposed?

$ nix flake update nixpkgs nixpkgs-wayland

The former I agree with, it's not a great idea, it's confusing an error-prone. But I'm really having a hard time seeing the issue with the latter one.

I can imagine the exact bugs it will cause right now.

What would those be, roughly?

we should strive for intuitive, explicit CLI UX

Sure, but we seem to disagree what intuitive means. Intuition is built from our individual experiences, and so what we deem intuitive differs. I of course appreciate you voicing your opinion, but the general sentiment of this thread seems to be that the most common usecase should be the simplest and most obvious one, and so what's intuitive in this context is a democratic decision.

nix flake -C ~/myflake lock --recreate-lock-file

I do like the idea, kind-of, but operating on any flakeref is very useful and common for almost all nix flake commands except update. So changing the existing UX from nix flake metadata nixpkgs to nix flake -C nixpkgs metadata seems to be an overall regression. It seems like that's what you're suggesting.

maybe for fun nix flake update could actually do a dry-run and print a summary of what inputs could be updated, and then encourage the user to nix flake update --all or nix flake update --input [one_from_the_summary]

I do like that idea, and I will try to implement it for flakes where this action isn't known to be reversible. I'd still argue that offering a simple yes/no dialog is the user friendly and low-friction solution, as opposed to erroring out, but a little encouragement isn't misplaced here.

[iFreilicht]

@colemickens Ahh I just saw that you're actually the person who opened this ticket in the first place! After reading through the code for the last hours, let me give you an answer to your initial questions:

Why is nix flake update a think instead of just nix flake lock --update-all?

Or why is nix flake lock --update-input not nix flake update --single <name>.

Or is there some other reasoning for the current grouping? I don't get it, I have to look it up every time. It just feels inconistent

The reason is that Nix actually tries very hard to be 100% consistent to the point of being confusing.

Under many commands in the nix manual, you'll see a section "Common flake-related options:". Any command that can take a flake as an input can take these. I think for something like --no-use-registries, or --override-input, this makes perfect sense as it changes how a flakeref is resolved.
But --update-input and --recreate-lock-file are part of that list, and those actually (try to) mutate the flake you're referring to!

This does enable you to call nix build --recreate-lock-file instead of nix flake update; nix build, but I can't see how the former is better than the latter. No idea how many people know that this is possible or have even used this feature once.

And these flags can of course also be parsed to flake update and flake lock. Actually, nix flake update right now is the exact same as nix flake lock --recreate-lock-file, it just sets the --recreate-lock-file flag internally and ignores all --update-inputs.

I'm not entirely sure what the motivation was for this design, I guess to have primitives you could build command invocations from? No matter what, this generalization clearly gets in the way when the only thing you want to do is update inputs.

[roberth]

Discussed in meeting:

We've had a discussion about the design. We would love for this UX issue to be made some progress on, and here's our thoughts on it:

Low hanging fruit is adding nix flake update --update-input support which is consistent with the existing CLI.

Detailed notes:

• @thufschmitt (through agenda): Old issue that never got solved. We have a sensible proposal and someone willing to implement it. I say go for it!

--all

• @edolstra: --all is similar to Add nix search --all #4809. What's the expectation? I'd expect it to update everything
• @Ericson2314: the math thing suggests not to do anything (disjunction identity)
• @roberth: --all is more pain than the possible pain someone writing a script might encounter
• Conclusion: avoid --all; just update everything

-C and meaning of positional arguments

• @edolstra, @tomberek: dynamically interpreting positional argument one would be ambiguous
• @roberth: nix update instead of nix flake update so that we have an excuse to diverge from the "arguments are flakerefs" convention
• @edolstra: Avoid -C because it's inconsistent with the rest of the CLI
• Preliminary conclusion: no changes then?
• @tomberek: Can we acknowledge there's a UX issue?
• @tomberek: Switching between nix flake update and nix flake lock is counterintuitive

Locking flags on all flake commands

• @Ericson2314: Don't all commands support --update-input?
• @edolstra: Yes, except nix flake update, because it's equivalent to --recreate-lock-file, so --update-input would be meaningless
• @Ericson2314: Flags should be adjectives/adverbs not verbs
• @edolstra: However users expect nix flake update to have a flag to update just the specified inputs (e.g. --single)

Brevity and lock command expectations

• @roberth: I think this is a (rare-ish) case where the UI brevity is actually really important, so we should consider nix update as an alias for the very specific behavior that people want but is not consistent with the otherwise verbose CLI conventions
• @Ericson2314: Can we sidestep the issue by reconsidering the basics, verbs, adjectives
  • Also, looking at the implementation I see removeFlag
  • @edolstra: Well, some commands don't make sense with certain global Flake flags.
  • @Ericson2314: I don't disagree, but I think this is a problem we should be having -- if we have to filter-out global flags, this is evidence those should have never been global flags to begin with.
    • @edolstra: This is a side-effect of nix flake update being a shorthand for --recreate-lock-file. Normally you would never pass --recreate-lock-file --update-input ....
• @edolstra: --update-input is a bit weird to have in all commands, but --override-input is rather useful
• @edolstra: --update-input could be removed from the global flags
• @roberth: --override-input is like --with-overridden-input but less verbose
  • @Ericson2314: +1
  • @Ericson2314 & @roberth: conversely the persistent side effect ones (such as --update-input) don't make sense as --with-.... This is good evidence that they don't belong there! (and just belong on update/lock).
    • hypothesis: the non-adjective/adverb ones that cannot be rephrased (e.g. with --with-...) are the sketchy ones which need to be filtered out today with removeFlag!

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-08-11-nix-team-meeting-79/31627/1

[iFreilicht]

• avoid --all; just update everything

Good, it seems that's in line with the expectations of those who raised their voice so far.

• Avoid -C because it's inconsistent with the rest of the CLI

Agreed.

• nix update instead of nix flake update so that we have an excuse to diverge from the "arguments are flakerefs" convention

We have an excuse to diverge from the convention, which is; the UX is bad and there's no good reason to apply the convention in this particular case. Having nix update also makes that command seemingly unrelated to flakes when in reality it is only for flakes.

• --update-input could be removed from the global flags
  • if we have to filter-out global flags, this is evidence those should have never been global flags to begin with.
  • conversely the persistent side effect ones (such as --update-input) don't make sense as --with-.... This is good evidence that they don't belong there! (and just belong on update/lock).

I'm taking the freedom of seeing this as confirmation to my statement above and will remove --update-input and --recreate-lock-file in my draft PR. I can always revert that change before merging, but I'd like to see how it feels. This also makes --commit-lock-file pretty useless, but maybe that could be useful for something?

[9999years]

Having nix update also makes that command seemingly unrelated to flakes when in reality it is only for flakes.

IME this is a big source of confusion for new users who don't understand that everything under nix is for flakes (especially for commands like nix search which don't have equivalents in the nix2 CLI).

[Ericson2314]

@9999years Just so we are claer, it is an explicit goal of NixOS/rfcs#136 that the new CLI not be exclusively for Flakes, however. (That one is in FCP and close to being merged unless something new comes up.)

[iFreilicht]

that everything under nix is for flakes

Though not everything there is only for flakes, but installables, which indeed includes flakes, but also files with -f/--file, store paths, derivations and derivation outputs and bare Nix expressions with --expr.

But the nix flakes commands only operate on flakes.

[iFreilicht]

The PR is ready: #8817

For any curious users that want to test it, I added a Testing section to the PR. Just follow the instructions there.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/decision-making-2-0-1-0-1/12851/24