Support for TCP/UDP load balancing via stream-snippets configmap key

[pleshakov]

Proposed changes

This PR introduces two ConfigMap keys:

• stream-snippets, which sets a custom snippet in stream context.
• stream-log-format, which sets a custom log format for TCP/UDP load balancing in stream context.

With stream-snippets it is possible to support TCP or UPD load balancing:

For NGINX OSS:

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-config
  namespace: nginx-ingress
data:
    stream-snippets: |
      upstream tcp-coffee {
          server coffee-svc.default.svc.cluster.local:80;
      }

      server {
          listen 4455;
          proxy_pass tcp-coffee;
      }

      upstream tcp-tea {
          server tea-svc.default.svc.cluster.local:80;
      }

      server {
          listen 4456;
          proxy_pass tcp-tea;
      }

Notes for NGINX:

• If a DNS name such as coffee-svc.default.svc.cluster.local cannot be resolved, NGINX will fail to reload
• NGINX resolves a DNS name only once. The next time the name will be resolved is during the next config reload.

For NGINX Plus:

kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-config
  namespace: nginx-ingress
data:
    stream-snippets: |
      resolver kube-dns.kube-system.svc.cluster.local valid=5s;

      upstream tcp-coffee {
          zone tcp-coffee 64k;
          server coffee-svc.default.svc.cluster.local service=_http._tcp resolve;
      }

      server {
          listen 4455;
          proxy_pass tcp-coffee;
          status_zone coffee-stream;
      }

      upstream tcp-tea {
          zone tcp-tea 64k;
          server tea-svc.default.svc.cluster.local service=_http._tcp resolve;
      }

      server {
          listen 4456;
          proxy_pass tcp-tea;
          status_zone tea-stream;
      }

Notes for NGINX Plus:

• Here were use dynamic DNS service discovery and port discovery via SRV records. See https://www.nginx.com/blog/dns-service-discovery-nginx-plus/
• To make NGINX Plus discover all the IP addresses of pods instead of the virtual IP of a service, create the service as headless. See https://kubernetes.io/docs/concepts/services-networking/service/#headless-services and https://www.nginx.com/blog/load-balancing-kubernetes-services-nginx-plus/

To apply the ConfigMap run:

$ kubectl apply -f nginx-config.yaml

You can also check if the config was applied successfully:

$  kubectl describe configmap nginx-config -n nginx-ingress
. . .
Events:
  Type    Reason   Age                From                      Message
  ----    ------   ----               ----                      -------
  Normal  Updated  17s (x2 over 28s)  nginx-ingress-controller  Configuration from nginx-ingress/nginx-config was updated

Checklist

Before creating a PR, run through this checklist and mark each as complete.

• I have read the CONTRIBUTING doc
• I have added tests that prove my fix is effective or that my feature works
• I have checked that all unit tests pass after adding my changes
• I have updated necessary documentation
• I have rebased my branch onto master
• I will ensure my PR is targeting the master branch and pulling from my branch from my own fork

[isaachawley]

Tested, works in GKE. One question then I'll approve.

[isaachawley]

This stream block will appear even if there are no stream snippets - is that required?

[isaachawley]

👍

[komapa]

The upstream is called tcp-coffee in this case. Also, I would choose different port than 80 when giving example for streams :)

[pleshakov]

good catch. thank you!