Question about Ocelot LoadBalancer and Kubernetes Service Discovery Provider

[ussamoo]

Expected Behavior / New Feature

When using Kubernetes service discovery and LoadBalancer of type RoundRobin, Ocelot balance traffic beteween running pods.

Actual Behavior / Motivation for New Feature

Ocelot send traffic to the same pod without balancing.

Steps to Reproduce the Problem

1. Use the configuration below
   {
   "GlobalConfiguration": {
   "ServiceDiscoveryProvider": {
   "Namespace": "my-service-name",
   "Type": "Kube"
   }
   },
   "ReRoutes": [
   {
   "DownstreamPathTemplate": "/{everything}",
   "DownstreamScheme": "http",
   "ServiceName": "my-service-name",
   "UpstreamPathTemplate": "/api/{everything}",
   "LoadBalancerOptions": {
   "Type": "RoundRobin"
   },
   "UpstreamHttpMethod": []
   }
   ]
   }
2. Run 3 instance of a pod having as name "my-pod-name"
3. Create a service named "my-service-name" having as Selector "my-pod-name"

Specifications

• Version: 14.0.5
• Platform: Linux
• Subsystem:

Ocelot KubeProvider gets the information (IP and Port) correctly about the service named "my-service-name" and sends traffic TO THE SERVICE using the Ip/port.

The first issue here is that the service sends traffic always to the same pod and the second issue is that the Load Balancing is not controlled by Ocelot.

Isn't better to discover pods endpoints based on the service name and send traffic directly to pod endpoint ??

I tested this scenario with Ocelot and it works well, but i'm not sure if it's a good idea.

But doing so means that the Load Balancing can be controlled by Ocelot.

[enriko-riba]

I know this is old and you already provided a PR but just to clarify the issue.

There is nothing wrong in sending the traffic to the virtual service IP but above the TCP/IP routing is the HTTP layer with the connection concept.
Basically once you establish a HTTP connection with an endpoint you are stuck with the same connection/endpoint. The connection is kept alive until the pod resets it (which is not going to happen) or the connection times out (http linger, keepalive). IIRC the timeout is like 4 minutes.

There is one additional layer controlled via HTTP HEADER "connection: keep-alive". That's the default. You can test with postman or similar tool sending traffic to a loadbalancer service and see that only one pod consumes all traffic. If you change the connection header to "close" all pods will receive traffic in random order (not round robin).
This comes at a cost of every request having to renegotiate the connection. The TCP handshake takes roughly the same time as the whole following request/response. So congratz we just doubled the latency or halved our throughput (for small requests without body like GET).

This is something that need to be taken into account for endpoint implementation.

[ussamoo]

Thanks @enriko-riba , so what do you recommend ?

[enriko-riba]

I am by no mean an expert in this area and I had no time to take a look at the Ocelot implementation. I assume a httpClient is created via IHttpClientFactory or maybe SocketsHttpHandler? Given that both maintain an underlying connection pool I would try the following: 1. create a watch on Kubernetes API to receive endpoint changes (so we don't need to pool the K8s API per request) 2. on route init or on endpoint change: update the endpoint list per service 3. on incoming request: pick the next endpoint and create a httpclient instance via IHttpClientFactory (it will reuse an existing HttpMessageHandler from the pool) Maybe you implemented it that way already? The connection pool is shared between http client instances so it should be large enough to accommodate the sum of all endpoints. I can't speculate what number would be a good guess for the pool size. HTH

…

On Fri, Apr 3, 2020 at 2:54 PM ussamoo ***@***.***> wrote: Thanks @enriko-riba <https://github.com/enriko-riba> , so what do you recommend ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1126 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAM3LYLUANV27UN5UXUCAYDRKXMATANCNFSM4KSPSAMQ> .

[TomPallister]

fixed by #1134