dockerTools.buildLayeredImage truncates oversized closures

[gustavderdrache]

Describe the bug
When building a Docker image with dockerTools.buildLayeredImage, if the closure of contents has too many store paths, some will be ignored and not packed in the final image. I discovered this by packaging phantomjs2 into an image, but there's a shorter reproduction below.

To Reproduce

1. nix-build the following expression:

   with import <nixpkgs> {};
   dockerTools.buildLayeredImage {
     name = "test";
     tag = "latest";
   
     config.Entrypoint = "${phantomjs2}/bin/phantomjs";
   }

2. $ docker load <result

3. docker run --rm -i test

Note that the container crashes because it can't find the PhantomJS binary on the filesystem.

Expected behavior
The container should drop into a phantomjs REPL.

Additional context
I believe that this issue is caused by 3be7675. The script to create the layered image no longer accounts for multiple arguments, which would explain the symptom I saw above. Ultimately, by ignoring the extra arguments, only some of the final layer's contents are found.

To test this hypothesis, I was able to create a test case that bypasses buildLayeredImage and directly uses mkManyPureLayers.

First, the Nix expression:

let
  pkgs = import <nixpkgs> {};
  inherit (pkgs) lib;

  # Create more items than there are max layers
  items = 5;
  maxLayers = 3;

  # Minimum data needed for contentsEnv below
  created = "1970-01-01T00:00:01Z";
  config = {};

  mkDummyLayer = index:
    pkgs.runCommand
      "text-${builtins.toString index}"
      { inherit index; }
      ''
        mkdir -p $out
        echo $index > $out/text-$index
      '';

  # Note that there are 5 elements in this list:
  # one derivation for each of text-1, text-2, ..., text-5
  contents = map mkDummyLayer (lib.range 1 items);

  # config.json for Docker layers
  configJson =
    pkgs.writeText "config.json" (builtins.toJSON {
      inherit created config;
      architecture = "amd64";
      os = "linux";
    });

  # Customization layer for our closure
  contentsEnv = pkgs.symlinkJoin {
    name = "bulk-layers";
    paths = contents;
  };
in
pkgs.dockerTools.mkManyPureLayers {
  name = "test";
  closures = [ contentsEnv configJson ];
  inherit maxLayers;
  inherit configJson;
}

There are three points of interest:

1. We create 5 dummy derivations that are symlinkJoin-friendly. Really, they're just directories with a plain text file.

2. We set maxLayers to 3 - this ensures that we should be getting one layer with 3 derivations.

3. At the bottom, we create the layers derivation.

When I build this, here is what I see:

$ nix-build
[snip]
/nix/store/aysfgqsvw5wh5sgihaflmflzmkgcqb51-test-granular-docker-layers

$ for x in result/*/layer.tar; do echo $x; tar tf $x; done
result/1/layer.tar
tar: Removing leading `/' from member names
/nix/
/nix/store/
/nix/store/pzivpynp9d0zrdcz2qqmsdmpp6wr28fv-text-1/
/nix/store/pzivpynp9d0zrdcz2qqmsdmpp6wr28fv-text-1/text-1
result/2/layer.tar
tar: Removing leading `/' from member names
/nix/
/nix/store/
/nix/store/y8a7aq07hkvww0arl5xmbzb3a0xls81h-text-2/
/nix/store/y8a7aq07hkvww0arl5xmbzb3a0xls81h-text-2/text-2
result/3/layer.tar
tar: Removing leading `/' from member names
/nix/
/nix/store/
/nix/store/cjgbvimmxif7c4lbzi5skp22009vznar-text-3/
/nix/store/cjgbvimmxif7c4lbzi5skp22009vznar-text-3/text-3

Notice that we have lost the text-4 and text-5 derivations - they should have been in result/3/layer.tar, but aren't.

To confirm, I tested this exact expression against nixpkgs at the parent commit (ad7c7cf), and got this result:

$ nix-build
[snip]
/nix/store/nidh855yd67rvrn0nnphdrqij30hydfs-test-granular-docker-layers

$ for x in result/*/layer.tar; do echo $x; tar tf $x; done
result/1/layer.tar
nix/
nix/store/
nix/store/9iw5zajcymnls8f8l81bi2dnfz7k3anr-text-1/
nix/store/9iw5zajcymnls8f8l81bi2dnfz7k3anr-text-1/text-1
result/2/layer.tar
nix/
nix/store/
nix/store/c3nd7xv4br8gcs4kgs418lbfpy64jf1s-text-2/
nix/store/c3nd7xv4br8gcs4kgs418lbfpy64jf1s-text-2/text-2
result/3/layer.tar
nix/
nix/store/
nix/store/n1pmawqsibn31gy7dk4a82iabzhcd99v-text-3/
nix/store/n1pmawqsibn31gy7dk4a82iabzhcd99v-text-3/text-3
nix/store/5n9mzgz7rwdj831ljdc1ji0ng3z8r3sy-text-4/
nix/store/5n9mzgz7rwdj831ljdc1ji0ng3z8r3sy-text-4/text-4
nix/store/xdd476vic2nfrk0ib6c0bp0c8bv2pyrp-text-5/
nix/store/xdd476vic2nfrk0ib6c0bp0c8bv2pyrp-text-5/text-5

Note that the derivations text-3, text-4, and text-5 are all properly in the final layer as expected.

Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

• system: "x86_64-linux"
• host os: Linux 5.3.0-7625-generic, Pop!_OS, 19.10
• multi-user?: no
• sandbox: yes
• version: nix-env (Nix) 2.3.1
• channels(user): "nixpkgs-20.03pre210728.42a195919a1"
• nixpkgs: /home/user/.nix-defexpr/channels/nixpkgs

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:

[veprbl]

cc @grahamc @nlewo @jonringer @purefn

[purefn]

Oh, I see. It passes multiple paths to the script when there are more than max layers. I didn't realize that. Should be a relatively simple fix.

[purefn]

BTW, I'm super sorry I introduced this problem.

[gustavderdrache]

No worries! I appreciate the quick turnaround.

[nlewo]

I'm also feeling guilty... 🤦‍♂️

[jonringer]

just bumping, it's been 2 weeks