Overlay for redistributable + relocatable packages

[uri-canva]

Describe the bug

The question of how to distribute nixpkgs packages to non-nix system comes up often and in many different forms. The nix ecosystem has a lot of advantages for developing and building software from the perspective of the people writing the software, but many of those advantages are not relevant from a user's perspective, so it can be limiting to require users to install and use nix to use software built with nix.

So far nix bundle with https://github.com/matthewbauer/nix-bundle is the most generally applicable solution we have for that. That however requires the use of namespaces which aren't available on macOS, has considerable invocation overhead, and can be confusing for users as it is very non-idiomatic.

An option to make many packages redistributable is to statically link them. As long as they don't reference any other packages or themselves other than via linking, the resulting package will not have any such references, making it work both on systems regardless of whether nix installed (redistributable), and regardless of which path they're copied to (relocatable). If they do reference other packages or themselves, those references should be made relative to the executable, and the executable and the referenced files should be moved together. This is especially useful on macOS since applications are packaged as directories, but even for other kinds of packages and on other systems distributing packages as archived directories is very simple.

Currently not all statically linked packages are redistributable and relocatable, and for users looking for those packages it can be confusing to navigate. An overlay separate from pkgsStatic can clarify this.

Note that non-trivial packages might require a lot of work to make relocatable, so it's also possible that statically linked packages being relocatable is an unintended side effect. In that case we should document this so that the option of static linking stop misleading people looking to redistribute packages.

Related issues:
#214611
#43795

Notify maintainers

@NixOS/static
@NixOS/darwin-maintainers

[veprbl]

So you propose to get rid of using absolute paths ever?
That would remove a need for having NixOS/nix#1971. See also NixOS/nix#7224.

[uri-canva]

At least no absolute paths to /nix/store. For darwin outputs it might have absolute paths to system libraries and files, for linux outputs it might have absolute paths to /usr/bin/env and /lib64/ld-linux-x86-64.so.2 since they exist both on nix and non-nix linux systems (though in my testing both the dynamic link interpreter reference in the elf headers and shebangs can be relative, so that might not be necessary).

Yes, it would solve the use cases described in those two issues, but this isn't about changing how nix works, it's about adding a separate type of nixpkgs outputs, and wouldn't require any implementation or design changes to nix itself. It's not about changing nix and nixpkgs so they themselves can be used without requiring /nix/store or a system wide installation of nix, it's about changing only nixpkgs so that packages can target systems without nix.

[cdepillabout]

Currently not all statically linked packages are redistributable and relocatable, and for users looking for those packages it can be confusing to navigate. An overlay separate from pkgsStatic can clarify this.

Could you give some examples of:

• What packages are good candidates for additional overrides/changes required to be redistributable and relocatable?
• What sort of changes are required for these packages to be redistributable/relocatable?
• Do you expect that this overlay would be hard to maintain? (i.e. that it would require a lot of work to keep everything compiling)

[sternenseemann]

Unfortunately I believe a relocatable overlay is a pipe dream. Somehow rewriting absolute references to relative ones would not be enough, as absolute paths can't necessarily be replaced with relative ones, as the reference point would not be fixed. Instead software would need to be patched somehow to resolve these references correctly – and already discovering your own location can be error prone. Maintaining such invasive patches that are not strictly required does not seem viable to me.

What would be possible would be to have an setup hook that verifies that a store path is “relocatable”, i.e. it has no references to other store paths, but you can kind of already see that from nix-store -q --references, so it'd just be valuable preventing regressions.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/my-issues-when-pushing-nixos-to-companies/28629/10

[SomeoneSerge]

Instead software would need to be patched somehow to resolve these references correctly

The most comprehensive namespace- and proot-free (but linux-specific) solution I heard of is guix pack -RR which uses a combination of fakechroot and rtld audit, cf. https://hpc.guix.info/blog/2020/05/faster-relocatable-packs-with-fakechroot/ and NixOS/bundlers#18 (comment). I think this only applies to executables

/lib64/ld-linux-x86-64.so.2

Well this sounds tricky. It's not deployed on NixOS by default. In case of executables, a statically linked wrapper is probably a good compromise

What sort of changes are required for these packages to be redistributable/relocatable?

Honestly, testing $ORIGIN-relative RUNPATHs would be a great start. If relative paths don't break software on Nix-ful systems (and they might; we probably can't just patchelf post hoc because of string lengths), it's a good indication that at least the dynamic loading will keep working after relocation.

What packages are good candidates for additional overrides/changes required to be redistributable and relocatable?

I'd benefit greatly from being able to "relocate" python3.withPackages (ps: [ ps.torch ]). I'm currently relying on singularityTools.buildImage instead. It's a similar situation: a no-namespaces environment