Coverage test is flaky

[RyanTheOptimist]

https://github.com/envoyproxy/envoy-mobile/actions/runs/3363462624/jobs/5576640354

[RyanTheOptimist]

Gah! Pressed enter too soon. I've had several coverage failures on PRs recently in which coverage failed with a message like:

Run tar -czvf coverage.tar.gz generated/coverage
tar: generated/coverage: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
Error: Process completed with exit code 2.

[RyanTheOptimist]

@jpsim do you have any idea what's going on here? Is there some sort of instability in our infrastructure?

[jpsim]

I think I see what's happening here, which is that if we fail to upload some BEP events to EngFlow, we don't properly post-process the coverage data in envoy/test/run_envoy_bazel_coverage.sh.

From the CI job you linked:

ERROR: The Build Event Protocol upload failed: All retry attempts failed. DEADLINE_EXCEEDED: DEADLINE_EXCEEDED: deadline exceeded after 14.999918701s. [closed=[], open=[[remote_addr=envoy.cluster.engflow.com/3.214.200.122:443]]] DEADLINE_EXCEEDED: DEADLINE_EXCEEDED: deadline exceeded after 14.999918701s. [closed=[], open=[[remote_addr=envoy.cluster.engflow.com/3.214.200.122:443]]]