Allow Ability to Configure Upstream Keepalive

Allows Upstream Keepalive values like keepalive_timeout and
keepalive_requests to be configured via ConfigMap.

Fixes #3099

docs/user-guide/nginx-configuration/configmap.md

@@ -99,6 +99,8 @@ The following table shows a configuration option's name, type, and the default v
 |[variables-hash-bucket-size](#variables-hash-bucket-size)|int|128|
 |[variables-hash-max-size](#variables-hash-max-size)|int|2048|
 |[upstream-keepalive-connections](#upstream-keepalive-connections)|int|32|
+|[upstream-keepalive-timeout](#upstream-keepalive-timeout)|int|60|
+|[upstream-keepalive-requests](#upstream-keepalive-requests)|int|100|
 |[limit-conn-zone-variable](#limit-conn-zone-variable)|string|"$binary_remote_addr"|
 |[proxy-stream-timeout](#proxy-stream-timeout)|string|"600s"|
 |[proxy-stream-responses](#proxy-stream-responses)|int|1|
@@ -573,12 +575,35 @@ _References:_
 
 ## upstream-keepalive-connections
 
-Activates the cache for connections to upstream servers. The connections parameter sets the maximum number of idle keepalive connections to upstream servers that are preserved in the cache of each worker process. When this
-number is exceeded, the least recently used connections are closed. _**default:**_ 32
+Activates the cache for connections to upstream servers. The connections parameter sets the maximum number of idle
+keepalive connections to upstream servers that are preserved in the cache of each worker process. When this number is
+exceeded, the least recently used connections are closed. 
+_**default:**_ 32
 
 _References:_
 [http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive](http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive)
 
+
+## upstream-keepalive-timeout
+
+Sets a timeout during which an idle keepalive connection to an upstream server will stay open.
+ _**default:**_ 60
+
+_References:_
+[http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive_timeout](http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive_timeout)
+
+
+## upstream-keepalive-requests
+
+Sets the maximum number of requests that can be served through one keepalive connection. After the maximum number of
+requests is made, the connection is closed.
+_**default:**_ 100
+	
+
+_References:_
+[http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive_requests](http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive_requests)
+
+
 ## limit-conn-zone-variable
 
 Sets parameters for a shared memory zone that will keep states for various keys of [limit_conn_zone](http://nginx.org/en/docs/http/ngx_http_limit_conn_module.html#limit_conn_zone). The default of "$binary_remote_addr" variable’s size is always 4 bytes for IPv4 addresses or 16 bytes for IPv6 addresses.

internal/ingress/controller/config/config.go

@@ -393,9 +393,17 @@ type Configuration struct {
 	// upstream servers that are preserved in the cache of each worker process. When this
 	// number is exceeded, the least recently used connections are closed.
 	// http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive
-	// Default: 32
 	UpstreamKeepaliveConnections int `json:"upstream-keepalive-connections,omitempty"`
 
+	// Sets a timeout during which an idle keepalive connection to an upstream server will stay open.
+	// http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive_timeout
+	UpstreamKeepaliveTimeout int `json:"upstream-keepalive-timeout,omitempty"`
+
+	// Sets the maximum number of requests that can be served through one keepalive connection.
+	// After the maximum number of requests is made, the connection is closed.
+	// http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive_requests
+	UpstreamKeepaliveRequests int `json:"upstream-keepalive-requests,omitempty"`
+
 	// Sets the maximum size of the variables hash table.
 	// http://nginx.org/en/docs/http/ngx_http_map_module.html#variables_hash_max_size
 	LimitConnZoneVariable string `json:"limit-conn-zone-variable,omitempty"`
@@ -651,6 +659,8 @@ func NewDefault() Configuration {
 			ProxyBuffering:         "off",
 		},
 		UpstreamKeepaliveConnections: 32,
+		UpstreamKeepaliveTimeout:     60,
+		UpstreamKeepaliveRequests:    100,
 		LimitConnZoneVariable:        defaultLimitConnZoneVariable,
 		BindAddressIpv4:              defBindAddress,
 		BindAddressIpv6:              defBindAddress,

rootfs/etc/nginx/template/nginx.tmpl

@@ -416,6 +416,9 @@ http {
 
         {{ if (gt $cfg.UpstreamKeepaliveConnections 0) }}
         keepalive {{ $cfg.UpstreamKeepaliveConnections }};
+
+        keepalive_timeout  {{ $cfg.UpstreamKeepaliveTimeout }}s;
+        keepalive_requests {{ $cfg.UpstreamKeepaliveRequests }};
         {{ end }}
     }