treewide: add meta.mainProgram to (almost) all packages with a single binary

[stuebinm]

Description of changes

Set meta.mainProgram automatically for "trivial" cases of packages which produce only one binary (as determined by the nixpkgs-unstable channel's program.sqlite database).

Motivation:

#246386 has deprecated the assumption that a package's main program (if not set explicitly otherwise) has the same name as the package itself. Thus using lib.getExe on a vast number of existing packages has become essentially unsupported.

Unfortunately, it seems infeasable to set mainProgram for all existing packages manually. A rough estimate of the problem's scale was given in #219567 (comment), which estimates last year that there were around ~5500 packages affected.

I've previously opened individual PRs adding mainProgram to single packages when I came across one where it was missing, but honestly I don't look forward to dealing with this until ~2030 at least (rate estimated by running the above-linked script again now), so I thought it was worth looking at another approach.

Implementation

Builing on @emilylange's script (linked above), it's pretty easy to get a list of candidate packages which produce exactly one binary (which must hence be that package's mainProgram), but which do not actually set this in their meta block. I consider these "trivial" cases to be fixed. Currently (before this PR), there are 6921 of these.

Using meta.position, it is further (mostly) easy to get each candidate's definition site (exception: things produced e.g. via the builders in build-support/trivial-builders, where meta.position points to the builder's definition instead).

Candidates can then have a meta.mainProgram entry added to their source automatically, for which I wrote a little rust tool to edit package definitions (loosely based on nix-doc-munge which was written for the doc migration from AsciiDoc).

Consideration & special cases:

The automatic rewriting does two things:

• if the package definition already has a meta block, it will add a mainProgram = ... in the next line after the description entry
• if it does not yet have a meta block, it will add a meta.mainProgram = .. attribute to the end of the attribute set
• however, some language-specific build systems (notably haskell) do not have a separate meta block and instead keep description, mainProgram, etc., as direct attributes of the package description. I have found no better way to reliably identify these than to see if the derivation has a top-level description (or similar) attribute; in this case, no meta block will be created.

Results

Of 5457 files defining the candidate packages, 5373 were edited; in 84 cases my script did nothing:

• most of these are files that occur more than once in meta.position of different packages, which I simply chose to ignore
• in 20 cases it found no suitable place for the meta attribute
• in 15 cases it found an existing meta attrset but failed to edit it (these
  lack description attributes, and it only knows how to insert something after that)

Re-searching for candidates with the changes applied reduced their number to 381. Still not nothing, but a lot more manageable.

Unexpected results

I thought this would be a no-op as far as packages are concerned, but turns out it is not:

Result of nixpkgs-review run on x86_64-linux 1

4 packages built:

• python311Packages.waitress-django
• python311Packages.waitress-django.dist
• python312Packages.waitress-django
• python312Packages.waitress-django.dist

This is a single package which has src = ./. set, so since its own package definition is part of the source, it does get changed.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[mweinelt]

I skimmed the changeset, and it looks plausible. Given that it doesn't break eval or check-meta, let's give it a shot before it rots away.

Thanks for your work on this!

[tpwrules]

There are some packages like the Python library Amaranth which have found a random utility script as the meta.mainProgram rather than what the package is about.

I don't object to this PR but I think it's worth being aware of.

[stuebinm]

@tpwrules yes, that occurred to me as well — in the end, I didn't do anything about it, because I don't think that this is, as such, wrong? It might still enable nice things to have it there (be it for nix run or lib.getExe) and even if not, it shouldn't hurt to have that line there.

(wondering out loud for a bit: perhaps it would be useful to have a meta.isLibrary-kind of flag, to explicitly say "even if this does have a main program, that's not really the point of the package? But then, this would again mean a couple thousand lines added to nixpkgs, for unclear benefits)

[KiaraGrouwstra]

wrote a little rust tool to edit package definitions

hm, this link seems dead?
i'm seeing some seemingly trivial cases like keepassxc and visidata not have the attribute set yet.

[stuebinm]

hm, this link seems dead?

ah, i renamed the repository once it turned into a slightly more general tool; I've now updated the link to again point at the version which was used for this PR (if you want to use it, be aware it's very much single-purpose & brittle)

i'm seeing some seemingly trivial cases like keepassxc and visidata not have the attribute set yet.

neither of these are "trivial" for the purposes of this PR; they both provide more than one binary (an easy way to check this is to use search.nixos.org, which uses the same programs.sqlite file to provide the "programs provided" listing).

I had deliberately excluded such packages which provide a binary named identically to their pname attribute, since that program is not always the actual "main program" in any intuitive sense (a good counterexample being the coreutils package providing a coreutils binary), so a wholly automated update would have likely led to unexpected (& unintended) consequences.