ci: add a test to check that the Docker image config works

[gustavovalverde]

Motivation

We want to test that our Docker build scripts configure Zebra correctly:

• check that $ZEBRA_CONF_PATH works
• make sure the default Docker config works with the latest Zebra version

Fixes #5168

Solution

• Add a Docker test for the default config
• Add a Docker test for $ZEBRA_CONF_PATH

Review

Anyone can review this, validating the test is using the right configuration

Reviewer Checklist

• Will the PR name make sense to users?
  • Does it need extra CHANGELOG info? (new features, breaking changes, large changes)
• Are the PR labels correct?
• Does the code do what the ticket and PR says?
  • Does it change concurrent code, unsafe code, or consensus rules?
• How do you know it works? Does it have tests?

Follow Up Work

Use a custom file instead of the existing one living inside common/configs

[teor2345]

Looks good, let's see how it goes!

[teor2345]

I don't think the test commands are quite right yet.

We also need to add the new test to the branch protection rules, did you want to do that after this PR merges?

[gustavovalverde]

In the test Docker stage we use cargo test --locked --release --features lightwalletd-grpc-tests --workspace --no-run which doesn't have zebrad as a binary

This is different from the runtime stage as it uses cargo build --locked --release --features sentry --package zebrad --bin zebrad

How should we run zebrad from the test stage @teor2345 ?

[teor2345]

I'm not sure I fully understand the question, but I'll try to say some helpful things:

In the test Docker stage we use cargo test --locked --release --features lightwalletd-grpc-tests --workspace --no-run which doesn't have zebrad as a binary

--no-run builds the binaries for the tests, without running any test binaries or the zebrad binary.

This is different from the runtime stage as it uses cargo build --locked --release --features sentry --package zebrad --bin zebrad

This builds the zebrad release binary.

How should we run zebrad from the test stage @teor2345 ?

The test stage should create a zebrad binary for the acceptance tests. It should be in a similar place to the release binary:

zebra/docker/Dockerfile

Line 132 in 7e5253e

COPY --from=release /opt/zebrad/target/release/zebrad /usr/local/bin

(It won't be in the debug directory, because we build tests with --release.)

I think one of the goals of the ticket is automating this manual check from the Zebra release process:

zebra/.github/PULL_REQUEST_TEMPLATE/release-checklist.md

Line 155 in 7e5253e

- [ ] Test the Docker image using `docker run --tty --interactive zfnd/zebra:1.0.0-rc.<version>` <!-- TODO: replace with `zfnd/zebra` when we release 1.0.0 -->

Is it possible to run this test on the release Docker image instead?
If it's not, or it's a lot of effort, that's ok. The test will still cover most of what we need.

[teor2345]

If you run docker run zfnd/zebra does it generate output or starts syncing? (I'm in a Mac with M1 and it doesn't, and the ones deployed on GCP are not syncing either)

Since this PR changes the entry point script, I'm not sure if running the main branch image will help us here.

Here is the output from Run tests using the $ZEBRA_CONF_PATH:

+ exec zebrad -c zebra/zebrad/tests/common/configs/v1.0.0-rc.2.toml start
error: Zebra could not parse the provided config file. This might mean you are using a deprecated format of the file. You can generate a valid config by running "zebrad generate", and diff it against yours to examine any format inconsistencies.
error: zebrad fatal error: config error: path error: zebra/zebrad/tests/common/configs/v1.0.0-rc.2.toml

https://github.com/ZcashFoundation/zebra/actions/runs/3963766811/jobs/6792068479#step:3:174

It looks like the config file isn't at the expected path - where is the Zebra git checkout in the Docker image?

Also, tee isn't echoing the output before it exits - since tee works with longer "docker logs" output and with other tools, it could be a bug in "docker logs" where it doesn't send an end of file or flush its internal buffers before finishing. But printing all the logs at the end seems like enough for now. I don't think we need any further fixes for the log output.

[gustavovalverde]

Since this PR changes the entry point script, I'm not sure if running the main branch image will help us here.

The main branch does not build the Docker image with the entrypoint (is not included in the runtime stage), so this PR won't have an impact there, even after merged.

https://github.com/ZcashFoundation/zebra/actions/runs/3963766811/jobs/6792068479#step:3:174

It looks like the config file isn't at the expected path - where is the Zebra git checkout in the Docker image?

Right here https://github.com/ZcashFoundation/zebra/blob/feat-conf-test/docker/Dockerfile#L105

This are the result of a pwd and ls

Requested changes were made

[teor2345]

This looks like it should work.

Did you want to add the new jobs to the branch protection rules after it merges?

[teor2345]

We'll also need to update the patch workflows with the new jobs 🙂

[gustavovalverde]

@teor2345 sorry that I dismissed your review with this change

• 1beaebd

Please have a look at it, as this will ensure we test the Dockerfile on both stages: tests and runtime, as the build process is different and this ensures it works in both cases.

[gustavovalverde]

Manual deployment to test runtime stage https://github.com/ZcashFoundation/zebra/actions/runs/3982611624

[teor2345]

Looks great, thanks for adding that extra test!

We'll have to manually check the test status in the CD workflow, but that's ok, that's what we're doing for everything else in that workflow anyway.

[teor2345]

I added these new jobs to the main branch protection rules:

• Test Zebra default Docker config file
• Test Zebra custom Docker config file