POC: devShell interface

[roberth]

The idea here is that future Nix first tries to "nix run" the pkg.devShell package, and only if that fails, fall back to the legacy nix develop (or nix-shell) behavior.

This allows the development shell to evolve with stdenv, and it allows packages to individually customize the devShell attribute, by setting passthru.devShell.

Furthermore, these shell behaviors will be pinned to the expressions, allowing changes to be made in a more agile manner, unlike Nix, which has to be very careful not to break old expressions, as users can not revert Nix.

To give it a try:

nix run .#hello.devShell

In the future this will be equivalent to:

nix develop .#hello

Isn't this the responsibility of Nix?

It is not. nix-shell and nix develop are a great user interface, that everyone loves, but their implementation is a pile of hacks on top of stdenv.
Instead of coercing stdenv to do what nix-shell needs it to, we can ask stdenv politely to provide a shell.
Now that Nix doesn't have to assume a package comes from stdenv, there's a possibility for experimental builders to provide shells too.

What does this break?

Only packages that define a devShell attribute (for some reason?) have to adapt to the suggested new Nix behavior. Note that a devShell value for the builder can be overridden by passthru without affecting the build or shell.
Packages and shells pinned to older versions can still be loaded because Nix keeps the legacy behavior as a fallback.

But this still relies on nix-shell to provide a shell???

Fair enough. This is only a proof of concept. The goal is to replace that invocation by a script or program with the same or better behavior, without relying on nix-shell as its implementation.

Does this solve the need for .env for Haskell package shells?

Not in this commit, but Haskell packages will be able to produce their own devShell attribute, which is derived from the .env derivation rather than the regular derivation.

Refs

• provide getDerivationEnvironment as a primop to reference nix develop drv from Nix code nix#7468 and a bunch of other issues where I've written about this idea.

Description of changes

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 23.05 Release Notes (or backporting 22.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
  • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
• Fits CONTRIBUTING.md.

[blaggacao]

Copying from NixOS/nix#7468 (comment)

It's possible that we need to mimic the flakes API here and do a passthru.devShells instead, where perhaps the default is a builtin attribute guaranteeing us a replica of the derivations build enviroment, but users can add others for various purposes?

I.e.:

• pkg.devShells.default instead of the so far proposed pkg.devShell

[roberth]

pkg.devShells.default instead of the so far proposed pkg.devShell

This seems more than needed. If a package is complicated enough to warrant multiple shells, that probably means that it consists of components. Presumably those could be built separately and be represented by different attributes; siblings of the package, or attributes on the package (like passthru). Selecting those components' shells is then a matter of selecting attributes.

# default shell
nix develop .#mypkg

# component-specific shell
nix develop .#mypkg.frontend

The nix develop logic can resolve the shell relative to the installable's attribute path.

So this is already a solution without having to make the package <-> nix interface more complicated.

[roberth]

See streamNixShellImage for how to do this kind of thing.

[Pandapip1]

@roberth any plans on updating this?

[roberth]

I originally wrote this PR to illustrate the design, and it worked: NixOS/nix#7501 was accepted by the Nix team.

In the foreseeable future I'm too occupied with Nix maintenance to put much effort into either the Nixpkgs or the Nix side of the proposal, so I would encourage anyone to work on it. Help is very much appreciated.

I believe the general approach taken in this PR is valid, but it was written as a proof of concept. I'll close this so nobody hesitates to open their own.

[roberth]

Making progress on some prerequisites here

• devShellTools: add environment functions #324789