Fix and simplify CI caching

[TimJentzsch]

Objective

Currently, the CI cache is only renewed when the Cargo.toml files are updated. However, when a patch dependency is released which is compatible with the Cargo.toml, the CI cache will be outdated and the dependencies will need to be recompiled every time.

Additionally, our CI workflow definitions are quite verbose, because the cache paths need to be defined every time.

Solution

Use the Leafwing-Studios/cargo-cache action.
It generates the Cargo.lock file before determining the cache key, such that the exact dependencies can be used to keep the cache up-to-date and avoid recompiles. The cache falls back to similar previous caches to avoid completely starting from scratch.
This also reduces a lot of boilerplate in the CI definitions.

Additional Context

• The cache is only updated when the key is not already available. This is why the Cargo.lock file should be part of the cache key, because it determines exactly which dependencies are used.
• The Cargo.lock file is not committed to the repository, because this is not recommended for libraries. This is why the Rust toolchain action must be used before the cargo-cache action, because it uses the cargo update command to generate the Cargo.lock file.
• For Leafwing-Studios/Emergence, this reduced our CI times from ~15 min to ~2 min.

Security Considerations

Adding a new third-party action increases the attack vector and should be considered carefully.
I recommend to audit the action first, it should be easy enough to understand (it's very similar to the current system we use) and I tried to add as many comments as possible.
I also published the action under the Leafwing org to get additional eyes on every PR (e.g. Alice).
If you'd prefer, we can also target an explicit commit of the action instead of a tag, to ensure that updates can be audited properly.

[TimJentzsch]

As a small example, notice how in this CI run, there was a cache hit, but still some dependencies had to be downloaded and recompiled.

[rparrett]

At a glance, doesn't this break caching of the cargo registry, cargo binaries, etc if cargo update is run before the cache is set up?

Downloading the full registry often takes a non-trivial amount of time.

The cache falls back to similar previous caches to avoid completely starting from scratch.

Is this behavior explained somewhere?

[TimJentzsch]

At a glance, doesn't this break caching of the cargo registry, cargo binaries, etc if cargo update is run before the cache is set up?

In theory yes, but in practice this doesn't seem to matter compared to the compile times that are saved. For example, in Emergence, the toolchain action takes 12s and the cache action with cargo update takes ~1m.
Probably not ideal, but the compile step then only takes 4s which is a huge win.
I also experimented with caching the binaries before the cargo update step, but that didn't seem to make a big difference.

The cache falls back to similar previous caches to avoid completely starting from scratch.

Is this behavior explained somewhere?

This is implemented via the restore-keys attribute of the cache action, see the caching docs.
It basically allows you to fall back of prefixes of cache keys.
We first fall back to the same Cargo.toml file when the Cargo.lock file is outdated and then to the same workflow job if the Cargo.toml file is outdated.
You can view it in the action definition here:
https://github.com/Leafwing-Studios/cargo-cache/blob/e9e0c6671e9ecbb4e8c516c2e1563a543eb7f2b4/action.yml#L61-L63

[alice-i-cecile]

This has been working great for us in practice, but I'll abstain from reviewing this myself.