Support batch-level retries in dbt retry for microbatch models

[MichelleArk]

The "problem"

How should dbt retry interact with microbatch models? There are three potential states for a microbatch model:

1. Every batch succeeds
2. Every batch fails
3. Some batches succeed AND some batches fail

Cases (1) and (2) are already handled. If every batch succeeds, then the microbatch model as a whole is considered a success and nothing will happen during dbt retry. If every batch fails, then the microbatch model as whole is considered as failed and the model will be rerun during dbt retry. Then there is case (3) which doesn't fit so nicely into successorfailure`. If we consider case (3) a success, then the model won't be rerun on retry and the batches that failed won't get fixed. If we consider case (3) a failure, then the model as a whole would be considered and we'd rerun EVERY batch. This isn't great because what if you have 99 successful batches and only 1 failure? You don't want to redo the 99 previously successful batches, that's more work (and data warehouse cost) that doesn't need to be incurred.

The Solution

Microbatch models that have some successful batches and some failures should be considered a Partial Success. During dbt retry, only the previously failed batches should be retried for a microbatch model.