[INC-263] Backoff mechanism for fetching projects in case of error

[jan-auer]

With #1426 deployed in case of invalid project state (when it could not be parsed or fetched) we will reuse the old state up until the grace period is reached.

This does not stop the Relay to try and update the state in next refresh cycle again though. We must make sure that we have a back off for these retries so we won’t DDoS our own services with requests.

[jan-auer]

@jjbayer wrote:

We have a backup mechanism, but it is only used if the upstream does not respond at all, not for invalid project configs:

relay/relay-server/src/actors/project_upstream.rs

Lines 237 to 243 in 7b39587

	// If a single request succeeded we reset the backoff. We decided to
	// only backoff if we see that the project config endpoint is
	// completely down and did not answer a single request successfully.
	//
	// Otherwise we might refuse to fetch any project configs because of a
	// single, reproducible 500 we observed for a particular project.
	slf.backoff.reset();