Standardization of weights for load-balancing

[pierresouchay]

Feature Description

When using Consul to discover backends for reverse proxies, there is now way to guess the weight of a given backends.

Since backends are running on different nodes, I would be nice to standardize in Consul how much calls are redirected to a given instance, thus all proxies and libraries might be using the same rules. Historically, some proxies / libraries are using tags, some other start using Meta in instances, but all of this is a mess.

Some proxies are using DNS SRV (eg: HaProxy), some other HTTP APIs, it would be nice to have the same data for both.

I propose the following:

1. Add metadata in Services with specific semantics regarding Load-Balancing
2. Return the data in HTTP and SRV DNS interfaces
3. Allow using the warning status in DNS response to modify the weight dynamically (thus it would allow services to heavily loaded to receive less traffic, for instance when they receive too many requests and return warning state)

Implementation proposal

When defining a service, use the following optional service Meta:

{
  "Meta": {
    "_dns_passing_prio": "1",          # default value = 1
    "_dns_passing_weight": "10",   # default value = 1
    "_dns_warning_prio": "1".         # default value = 1
    "_dns_warning_weight": "1"     # default value = 0
  }
}

When DNS return SRV requests, for each SRV record, apply the corresponding _dns_<state>_weight and _dns_<state>_prio to fill DNS SRV records

This could be very easily implemented and would simplify a lot interoperability of Load balancing systems.

Optionally, we might also use similar meta data at node level (defined in the agent) in order to multiply those weights based on node characteristics (thus services would not have to know on what kind of large/tiny instance they are running) - but it could be a improvement for another PR.

What do you think?

I am ready to implement this if Hashicorp considers including it.

[bedis]

Hi,

As HAProxy's DNS stack maintainer, I'd be glad to see such feature in Consul.
Furthermore, HAProxy is already compatible with DNS SRV record's weight.

Note that HAProxy does not follow-up SRV record "prio" for now. We have some use cases in mind for this field, but are not yet sure how to implement them.

Baptiste

[pearkes]

Hey! We definitely like this idea, agree with all your points on the benefits. A couple notes.

• Give it a dedicated configuration key instead of using meta. We currently use meta with some consul- prefixed items for various internal features, but users don't interact with those during service registration. For a dedicated key we recommend something like this:

{
  "Priority": {
    "passing": 1,
    "warning": 1
  },
  "Weight": {
     "passing": 10,
     "warning": 1
   }
}

• Start with just modifying services for this behavior and possibly plan for the node characteristics feature in the future.

Let us know how we can help!

[iksaif]

cc: @wdauchy

[pierresouchay]

Let us know how we can help!

@pearkes Sure, I made a first implementation in #4468, and a review would help.

I kept the exact plan you described in your last comment - but I did not implement priority since its semantics are clear for no-one :-)

My first implementation only update weights in SRV records, but does not apply the weights in DNS A/AAAA queries. But this can be implemented later. The patch is big enough for now and I'll implement A/AAAA DNS in another review.

@banks Nice to see that our discussion allowed me to plug nicely with new Connect features as well since Weights are already implemented in it!

See #4468 (comment) for an example of input and DNS SRV output