Engine can redistribute deployments when scaling up

[lenaschoenburg]

When scaling up, deployments containing processes, forms and drgs need to be redistributed to new partitions.

Each deployment exists with all metadata but without the actual binary resource in the state.
When redistributing, we can iterate through all deployments and rehydrate the full deployment create record.
If a resource was deleted after the deployment, we will only redistribute the metadata but not the full binary resource.

This is compatible with resource binding in that validation only occurs on the deployment partition when the deployment is created for the first time. On redistribution, validation is skipped.

We can redistribute deployments one-by-one or in batches. Since deployments tend to be large but few, batching might not be worth it.

To decide whether new deployments created during redistribution need to be enqueued for distribution to a partition immediately or not, we keep track of the type and key of the latest redistributed resource. If not all deployments are redistributed yet, a new deployment does not need to be distributed to new partitions yet.

Tasks

Preview Give feedback

1. Deployment metadata should be available in runtime state #24677
   3 of 7
   component/engine component/zeebe kind/toil +3
   
2. Reattach available resources when redistributing deployments

[lenaschoenburg]

Unfortunately this does not work out as planned because we decided a while ago to not store deployments anymore: d32354c

Instead what we'll have to do is go back to redistributing resource-by-resource again. We can still wrap them in deployment records such that we benefit from all of the resource creation logic built for that.