Faster post build startup:

[KevinMind]

Fixes: mozilla/addons#15239

Description

• use named volumes that mount faster and don't need renewing
• skip docker compose healthcheck
• add healthcheck script to initialize

Context

Faster post build by parallelizing initialize and startup

Testing

Verify time of startup

You can run the steps of the build independently to get a feel for relative times.

1. Build: setup and building the image

Try building the image first without cache.

Warning

This is an aggressive command that will clear local images/volumes and docker state

make clean_docker

Make sure you don't have any dangling volumes or images.. cleaning up docker can be a chore....

time make up_pre

Check the time, then run the command again. The second time should be much, much faster.

Try it after changing a file in ./src this will invalidate some of the cache and give a more realistic cached time.

2. Start: starting the containers

Now start the containers, running

time make up_start

The first time, it should take some time to download images. Run it again and it should be quite fast.

Note

Before running the second time, run make down so that the containers are not running but are pulled.

3. Initialize: setting up container state and waiting to be healthy

Now run the initialize command to initialize the container state and ensure everything is healthy.

time make up_post

If you have a truly clean environment this could take some time as it must migrate and populate the database. Run it again and it should be relatively fast.

Verify no anonymous or unnamed volumes

You can independently verify the non existence of anonymous volumes via this query that returns volumes from all containers that are:

• bind mount (anonymous volumes)
• volume mount missing source (no name)

docker compose config | yq -r '.services.*.volumes[]? | select((.type == "bind") or (has("source") | not))'

Checklist

• Add #ISSUENUM at the top of your PR to an existing open issue in the mozilla/addons repository.
• Successfully verified the change locally.
• The change is covered by automated tests, or otherwise indicated why doing so is unnecessary/impossible.
• Add before and after screenshots (Only for changes that impact the UI).
• Add or update relevant docs reflecting the changes made.

[KevinMind]

Command	Uncached Time	Cached Time	Difference (s)
make up_pre	3:58.51	2.541	236.0
make up_start	2:15.34	3.822	131.5
make up_post	1:29.98	15.859	74.1
Total	7:43.83	22.222	441.6

[eviljeff]

I got a little lost during the test steps a few times so I don't really have comparable figures for all the steps, but my time make up after a make down and changing a file in `./src/ is

real 1m52.840s
user 0m2.189s
sys 0m1.734s

which is somewhat of an improvement - but not quite as speedy as your figures.

fwiw, [+] Building 45.6s (29/29) FINISHED - so the docker part is 45 seconds; the rest is the django initialization (even with no migrations, it takes a little bit of time) + es reindexing, pretty much.

[eviljeff]

Does the syntax support brackets? It's not immediately obvious to me (or future me) if it's doing (A and B) or C or A and (B or C)

[KevinMind]

This is pretty standard JS syntax afaik (GHA operators), it's the first one. The statements are evaluated left to right.

[eviljeff]

Ah, it's 'Uncached build' if matrix.version == 'local' else 'Pull' in python syntax

[KevinMind]

I got a little lost during the test steps a few times so I don't really have comparable figures for all the steps, but my time make up after a make down and changing a file in `./src/ is

real 1m52.840s user 0m2.189s sys 0m1.734s

which is somewhat of an improvement - but not quite as speedy as your figures.

fwiw, [+] Building 45.6s (29/29) FINISHED - so the docker part is 45 seconds; the rest is the django initialization (even with no migrations, it takes a little bit of time) + es reindexing, pretty much.

Yeah the goal of this PR is to make the 2 and 3 of the 3 steps faster. It doesn't really impact the build itself. The "better" way of measuring the impact here would be to make the changes to the make recipes first, then time the original state, and then add the changes on this patch so you can get per phase timing... It's faster. At the end of this epic I plan to test the July state of the code to the end state and we can get some real data on the impact.