Clean up orphaned Blob and Cosmosdb resources

[jjcollinge]

Currently, if a job fails after committing any blob or document data - it will be rerun and the stale data will be left orphaned. We have a goroutine that cleans up completed Kubernetes jobs in the dispatcher (

ion/internal/app/dispatcher/providers/kubernetes.go

Line 176 in 9072a4a

if jobIsFinishedAndOlderThanAnHour(condition) {

) which we could extend to clean up any blob/document data associated with a failed job. This interface should probably be lifted above the specific provider to the dispatcher (

ion/internal/app/dispatcher/run.go

Line 18 in 9072a4a

func Run(cfg *types.Configuration) {

) and executed by each individual provider.