docker: Substrate Docker image error building binaries

[ltfschoen]

Is there an existing issue?

• I have searched the existing issues

Experiencing problems? Have you tried our Stack Exchange first?

• This is not a support question.

Description of bug

If I follow the instructions in the Substrate repository README here that link to the Substrate Builder Docker Image instructions here, the instructions say it might only include the following binaries:

• substrate
• subkey
• node-template
• chain-spec-builder

Then when I run ./build.sh it outputs the following error:

show error

Building parity/substrate:latest docker image, hang on!
[+] Building 2602.6s (14/15)                                                                                                              docker:desktop-linux
 => [internal] load build definition from substrate_builder.Dockerfile                                                                                    0.0s
 => => transferring dockerfile: 1.88kB                                                                                                                    0.0s
 => [internal] load .dockerignore                                                                                                                         0.0s
 => => transferring context: 136B                                                                                                                         0.0s
 => [internal] load metadata for docker.io/library/ubuntu:20.04                                                                                           5.0s
 => [internal] load metadata for docker.io/paritytech/ci-linux:production                                                                                 3.1s
 => [builder 1/4] FROM docker.io/paritytech/ci-linux:production@sha256:dc9abf9e877c5bad94828245406dac8a186530e1ad6a1b5f2072e5e19e1f64b4                   0.0s
 => CACHED [stage-1 1/6] FROM docker.io/library/ubuntu:20.04@sha256:33a5cc25d22c45900796a1aca487ad7a7cb09f09ea00b779e3b2026b4fc2faba                      0.0s
 => [internal] load build context                                                                                                                         0.6s
 => => transferring context: 46.95MB                                                                                                                      0.6s
 => CACHED [builder 2/4] WORKDIR /substrate                                                                                                               0.0s
 => CACHED [builder 3/4] COPY . /substrate                                                                                                                0.0s
 => CACHED [builder 4/4] RUN cargo build --locked --release                                                                                               0.0s
 => CACHED [stage-1 2/6] COPY --from=builder /substrate/target/release/substrate /usr/local/bin                                                           0.0s
 => ERROR [stage-1 3/6] COPY --from=builder /substrate/target/release/subkey /usr/local/bin                                                               0.0s
 => ERROR [stage-1 4/6] COPY --from=builder /substrate/target/release/node-template /usr/local/bin                                                        0.0s
 => ERROR [stage-1 5/6] COPY --from=builder /substrate/target/release/chain-spec-builder /usr/local/bin                                                   0.0s
------
 > [stage-1 3/6] COPY --from=builder /substrate/target/release/subkey /usr/local/bin:
------
------
 > [stage-1 4/6] COPY --from=builder /substrate/target/release/node-template /usr/local/bin:
------
------
 > [stage-1 5/6] COPY --from=builder /substrate/target/release/chain-spec-builder /usr/local/bin:
------
substrate_builder.Dockerfile:21
--------------------
  19 |     COPY --from=builder /substrate/target/release/subkey /usr/local/bin
  20 |     COPY --from=builder /substrate/target/release/node-template /usr/local/bin
  21 | >>> COPY --from=builder /substrate/target/release/chain-spec-builder /usr/local/bin
  22 |
  23 |     RUN useradd -m -u 1000 -U -s /bin/sh -d /substrate substrate && \
--------------------
ERROR: failed to solve: failed to compute cache key: failed to calculate checksum of ref 4b02f120-ba24-4d54-b2be-a38dc6dce883::kor4q7o2o99zc2mscfob9bp7v: "/substrate/target/release/chain-spec-builder": not found

real    43m22.870s
user    0m4.775s
sys 0m4.250s

Although the error appears obvious now, initially I thought it was because I was using a new Apple M2 chip, but it was actually just because for some reason that Docker approach only generates the substrate binary, but not the subkey, node-template, or chain-spec-builder binaries like it used to as well.

I believe the solution is to either:

1. Change line 6 of substrate_builder.Dockerfile to instead run the following to generate all the binaries in the /substrate/target/release directory, so it doesn't generate an error when they are used later on lines 19-21:

RUN cargo build --locked --release && \
    cargo build -p subkey --release --locked && \
    cargo build -p node-template --release --locked && \
    cargo build -p chain-spec-builder --release --locked

2. Alternatively remove lines 19-21 of substrate_builder.Dockerfile so it only builds and copies across the substrate binary to the Docker container. The benefit of doing this is that it should reduce the overall build time for those getting started with Substrate, as most users may only want to use the substrate binary. We could then update the Substrate Builder Docker Image instructions here to mention that if the user wants to use the subkey, node-template, or chain-spec-builder binaries too then they can simply enter the Substrate Docker container shell with the command docker run -it -d --hostname substrate --name helios parity/substrate && docker exec -it substrate /bin/bash where they could then run the substrate binary with ./target/release/substrate --help and manually generate the other binaries by running the following to build those other binaries:

cargo build -p subkey --release --locked && \
cargo build -p node-template --release --locked && \
cargo build -p chain-spec-builder --release --locked

and then run the following to create symlinks

ln -s /usr/local/bin/subkey /substrate/target/release/subkey
ln -s /usr/local/bin/node-template /substrate/target/release/node-template
ln -s /usr/local/bin/chain-spec-builder /substrate/target/release/chain-spec-builder

so they could then run those other binaries with:

./target/release/subkey --help
./target/release/node-template --help
./target/release/chain-spec-builder --help

or exit the Docker container and run the follow as it already mentions is available in the README

./run.sh subkey --help
./run.sh node-template --version
./run.sh chain-spec-builder --help

Steps to reproduce

• Go to https://github.com/paritytech/substrate#getting-started
• Click the "Docker instructions" hyperlink that takes you to https://github.com/paritytech/substrate/blob/master/docker/README.md
• Follow the instructions by first installing Docker, and then running the ./build.sh shell script in that directory, or simply run the following commands:

git clone https://github.com/paritytech/substrate
cd substrate
./docker/build.sh

• View the error