Implement changes for new added flag of disable_overprovisioning in ClusterLoadAssignment

[jaychenatr]

Description:
As a new API change will be introduced for disable_overprovisioning flag in #8080. Changes are required to be implemented accordingly in Envoy. If disable_overprovisioning is set to false(default value), Envoy will behave the same as overprovisioning_factor suggests, Otherwise Envoy need to ignore disable_overprovisioning and stop performing graceful failover between priorities/localities as endpoints become unhealthy.

[markdroth]

@htuch @alyssawilk @snowp

We will definitely need this for 1.14.

[alyssawilk]

Sorry, can you doc up the behavior you are expecting? I think I am not understanding what you are trying to do here.
Do you just want to serve 500s based on the percent unhealthy rather than moving your traffic to other viable endpoints? To me, disable_overprovisioning would be setting the overprovisioning factor to 1 (not over provisioning). Are you trying to disable failover?

[muxi]

The idea is to disable the failover. When disable_overprovisioning is set to true, Envoy does not automatically failover based on locality health; the xds server takes the responsibility of routing traffics to other localities by pushing new configurations.

I dug a little bit into Envoy code and I guess it's probably enough to

• Check if eds config says disable_overprovisioning==true here
• Somehow propagate the config to upstream_impl
• Have the effective weight(https://github.com/muxi/envoy/blob/master/source/common/upstream/upstream_impl.cc#L521) of the locality not affected by the availability ratio and the overprovisioning factor when disable_overprovioning == true.

I'm not sure if that is the right direction or if it covers all the necessary changes though. Would love to get some pointer.

[srini100]

/cc

[mattklein123]

@muxi that sounds about right. cc @snowp for further guidance.

[snowp]

I think in the past we've suggested people just set their overprovisioning factor really high in order to disable spillover, would this new config option just be equivalent to that?

[muxi]

You are right. I did not realize that but looking at the code in upstream_impl again it seems to be equivalent (except disable_overprovisioning is an on/off switch which has slightly different semantics). In that case I'll probably leave it as is for now. Thanks for the idea!

Besides that, hope to ask another question: is there any way to configure the default overprovisioning_factor locally for Envoy (e.g. with an environment variable or something)? It looks not likely to me because the default value seems to be a constant but in case you know another way hidden somewhere. Thanks!

[markdroth]

The need for this should be eliminated by #10136. Feel free to close.