how to deal with string contexts

[MangoIV]

the issue is as follows:

• for the build time dependencies we want to look up meta/ pname/ ... to find information about our packages
• for this we must be at the nix level, i.e. with access to the attrset that mkDerivation builds
• at this point we know the drvPath so we can query the dependencies
• these dependencies don't necessarily have to be available in the nix level, reason: string contexts; simple example:

pkgs.callPackage ./buildtime-dependencies.nix {} (
  pkgs.runCommand "foo" {} ''cp -r ${pkgs.hello}/bin $out''
)

this package clearly depends on pkgs.hello but bombon doesn't recognize that.

I have not found a solution but a very sad one:

• build meta for all of the packages that might be included; the user would pass a list of package sets and dependencies to build a meta "database" keyed by the drvPath of the derivation
• build the dependency tree of some package separately
• query the "database" against the drvPaths.

[nikstur]

That's a very good observation, thank you! I don't have a clear answer yet how I can/want to solve this.

[arianvp]

We need NixOS/nix#4677

[MangoIV]

https://github.com/wireapp/wire-server/blob/develop/nix/pkg-info.nix
https://github.com/wireapp/wire-server/blob/develop/nix/all-toplevel-derivations.nix
https://github.com/wireapp/wire-server/blob/develop/hack/bin/Sbom.hs

I’ve made this abomination a while ago. It works but it probably needs a ton of adjustments if you want to use it for anything but wireserver

[MangoIV]

It goes what I described above though.

[arianvp]

because pkgs is actually not a tree but a graph so you

will go around in circle

You could use https://nix.dev/manual/nix/2.18/language/builtins#builtins-genericClosure for this

[MangoIV]

That’s not the only problem though. Nix will also memory leak like crazy. So probably not going to work anyway. You need to be careful what to recurse on. I think I also excluded a couple of attrsets to not look at.

[sternenseemann]

In Nix >= 2.6 you can use the following code to correctly and reliably determine the dependency closure of a derivation (in Nix < 2.6 you have to parse the drv files yourself which is perfectly possible):

drv:
builtins.map (builtins.getAttr "key") 
  builtins.genericClosure {
    startSet = [ { key = builtins.unsafeDiscardStringContext "${drv.drvPath}"; } ];
    operator = { key }: builtins.map (drvPath: { key = drvPath; }) (
      builtins.attrNames (builtins.getContext (builtins.readFile key))
    );
  }
)

Obviously you are left with the problem that the serialized derivation files are lossy and lack e.g. meta. This is an inherent problem with string context. OTOH this code needs to make no assumptions about how the derivations work or are structured.

[arianvp]

Came up with something very similar. But adds a fixup phase that tried to reconstruct string-context deps if they happen to match non-string-context deps

https://gist.github.com/arianvp/e96acf85734ebe7a58bf24d4758484eb

As a last resort we can at least guess the version and pname by doing string parsing on the drv path and show a warning that the drv is unknown.