[automation] Use docker-worker Taskcluster caches

[data-sync-user]

This is a sub-ticket of #1273. Right now, ~/.gradle is baked into the Docker image; we really want that to be dynamically cached and shared across tasks using a docker-worker Taskcluster cache. See https://github.com/mozilla/application-services/blob/aee3c55412b35d69a774194a6197da28116a04e4/automation/taskcluster/decision_task.py#L168. This probably depends on #1461; start with that.

┆Issue is synchronized with this Jira Task

[data-sync-user]

➤ Sebastian Kaspari commented:

@ncalexan The docs say: "A cache can only be used by one task at a time."
https://docs.taskcluster.net/docs/manual/using/caching

Does this work with our parallelized builds?

[data-sync-user]

➤ Nick Alexander commented:

@ncalexan The docs say: "A cache can only be used by one task at a time."
https://docs.taskcluster.net/docs/manual/using/caching

Does this work with our parallelized builds?

So, AFAIK, what happens is that the worker type pool will have many caches, and the workers will rotate through them. That is, you can't assume that the workers all share a single cache -- but each worker will start with a cache that's relatively fresh, and then freshen it. It makes a huge difference for a-s, since we sccache all of our Rust compilation.

But! It's not suitable for, say, a Gradle build cache across simultaneous tasks. (But it would still help for this.)

[data-sync-user]

➤ Sebastian Kaspari commented:

So, AFAIK, what happens is that the worker type pool will have many caches, and the workers will rotate through them.

Okay, this sounds much better!

But! It's not suitable for, say, a Gradle build cache across simultaneous tasks. (But it would still help for this.)

Yeah, makes sense. But caching gradle task outputs may still be interesting for that, I guess...?

https://docs.gradle.org/current/userguide/build_cache.html