Skip to content

Commit 9326485

Browse files
krancourkrancour
authored
Merge pull request #323 from krancour/feat/proxy_buffer_size
feat(proxyBuffers): Add router-level and app-level proxy buffer config options
2 parents 8ecb82a + 2b9074f commit 9326485

File tree

5 files changed

+179
-32
lines changed

5 files changed

+179
-32
lines changed

‎README.md

+8
Original file line numberDiff line numberDiff line change
@@ -262,6 +262,10 @@ _Note that Kubernetes annotation maps are all of Go type `map[string]string`. A
262262
| <a name="ssl-hsts-max-age"></a>deis-router | deployment | [router.deis.io/nginx.ssl.hsts.maxAge](#ssl-hsts-max-age) | `"10886400"` | Maximum number of seconds user agents should observe HSTS rewrites. |
263263
| <a name="ssl-hsts-include-sub-domains"></a>deis-router | deployment | [router.deis.io/nginx.ssl.hsts.includeSubDomains](#ssl-hsts-include-sub-domains) | `"false"` | Whether to enforce HSTS for subsequent requests to all subdomains of the original request. |
264264
| <a name="ssl-hsts-preload"></a>deis-router | deployment | [router.deis.io/nginx.ssl.hsts.preload](#ssl-hsts-preload) | `"false"` | Whether to allow the domain to be included in the HSTS preload list. |
265+
| <a name="proxy-buffers-enabled"></a>deis-router | deployment | [router.deis.io/nginx.proxyBuffers.enabled](#proxy-buffers-enabled) | `"false"` | Whether to enabled proxy buffering for all applications (this can be overridden on an application basis). |
266+
| <a name="proxy-buffers-number"></a>deis-router | deployment | [router.deis.io/nginx.proxyBuffers.number](#proxy-buffers-number) | `"8"` | `number` argument to the nginx `proxy_buffers` directive for all applications (this can be overridden on an application basis). |
267+
| <a name="proxy-buffers-size"></a>deis-router | deployment | [router.deis.io/nginx.proxyBuffers.size](#proxy-buffers-size) | `"4k"` | `size` argument to the nginx `proxy_buffers` directive expressed in bytes (no suffix), kilobytes (suffixes `k` and `K`), or megabytes (suffixes `m` and `M`). This setting applies to all applications, but can be overridden on an application basis. |
268+
| <a name="proxy-buffers-busy-size"></a>deis-router | deployment | [router.deis.io/nginx.proxyBuffers.busySize](#proxy-buffers-busy-size) | `"8k"` | nginx `proxy_busy_buffers_size` expressed in bytes (no suffix), kilobytes (suffixes `k` and `K`), or megabytes (suffixes `m` and `M`). This setting applies to all applications, but can be overridden on an application basis. |
265269
| <a name="builder-connect-timeout"></a>deis-builder | service | [router.deis.io/nginx.connectTimeout](#builder-connect-timeout) | `"10s"` | nginx `proxy_connect_timeout` setting expressed in units `ms`, `s`, `m`, `h`, `d`, `w`, `M`, or `y`. |
266270
| <a name="builder-tcp-timeout"></a>deis-builder | service | [router.deis.io/nginx.tcpTimeout](#builder-tcp-timeout) | `"1200s"` | nginx `proxy_timeout` setting expressed in units `ms`, `s`, `m`, `h`, `d`, `w`, `M`, or `y`. |
267271
| <a name="app-domains"></a>routable application | service | [router.deis.io/domains](#app-domains) | N/A | Comma-delimited list of domains for which traffic should be routed to the application. These may be fully qualified (e.g. `foo.example.com`) or, if not containing any `.` character, will be considered subdomains of the router's domain, if that is defined. |
@@ -271,6 +275,10 @@ _Note that Kubernetes annotation maps are all of Go type `map[string]string`. A
271275
| <a name="app-tcp-timeout"></a>routable application | service | [router.deis.io/tcpTimeout](#app-tcp-timeout) | router's `defaultTimeout` | nginx `proxy_send_timeout` and `proxy_read_timeout` settings expressed in units `ms`, `s`, `m`, `h`, `d`, `w`, `M`, or `y`. |
272276
| <a name="app-maintenance"></a>routable application | service | [router.deis.io/maintenance](#app-maintenance) | `"false"` | Whether the app is under maintenance so that all traffic for this app is redirected to a static maintenance page with an error code of `503`. |
273277
| <a name="ssl-enforce"></a>routable application | service | [router.deis.io/ssl.enforce](#ssl-enforce) | `"false"` | Whether to respond with a 301 for all HTTP requests with a permanent redirect to the HTTPS equivalent address. |
278+
| <a name="app-nginx-proxy-buffers-enabled"></a>routable application | service | [router.deis.io/nginx.proxyBuffers.enabled](#app-nginx-proxy-buffers-enabled) | `"false"` | Whether to enabled proxy buffering. This can be used to override the same option set globally on the router. |
279+
| <a name="app-nginx-proxy-buffers-number"></a>routable application | service | [router.deis.io/nginx.proxyBuffers.number](#app-nginx-proxy-buffers-number) | `"8"` | `number` argument to the nginx `proxy_buffers` directive. This can be used to override the same option set globally on the router. |
280+
| <a name="app-nginx-proxy-buffers-size"></a>routable application | service | [router.deis.io/nginx.proxyBuffers.size](#app-nginx-proxy-buffers-size) | `"4k"` | `size` argument to the nginx `proxy_buffers` directive expressed in bytes (no suffix), kilobytes (suffixes `k` and `K`), or megabytes (suffixes `m` and `M`). This can be used to override the same option set globally on the router. |
281+
| <a name="app-nginx-proxy-buffers-busy-size"></a>routable application | service | [router.deis.io/nginx.proxyBuffers.busySize](#app-nginx-proxy-buffers-busy-size) | `"8k"` | nginx `proxy_busy_buffers_size` expressed in bytes (no suffix), kilobytes (suffixes `k` and `K`), or megabytes (suffixes `m` and `M`). This can be used to override the same option set globally on the router. |
274282

275283
#### Annotations by example
276284

‎model/model.go

+81-12
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,8 @@
11
package model
22

33
import (
4+
"bytes"
5+
"encoding/gob"
46
"fmt"
57
"log"
68
"strings"
@@ -60,11 +62,16 @@ type RouterConfig struct {
6062
AppConfigs []*AppConfig
6163
BuilderConfig *BuilderConfig
6264
PlatformCertificate *Certificate
63-
HTTP2Enabled bool `key:"http2Enabled" constraint:"(?i)^(true|false)$"`
64-
LogFormat string `key:"logFormat"`
65+
HTTP2Enabled bool `key:"http2Enabled" constraint:"(?i)^(true|false)$"`
66+
LogFormat string `key:"logFormat"`
67+
ProxyBuffersConfig *ProxyBuffersConfig `key:"proxyBuffers"`
6568
}
6669

67-
func newRouterConfig() *RouterConfig {
70+
func newRouterConfig() (*RouterConfig, error) {
71+
proxyBuffersConfig, err := newProxyBuffersConfig(nil)
72+
if err != nil {
73+
return nil, err
74+
}
6875
return &RouterConfig{
6976
WorkerProcesses: "auto",
7077
MaxWorkerConnections: "768",
@@ -87,7 +94,8 @@ func newRouterConfig() *RouterConfig {
8794
DefaultServiceIP: "",
8895
HTTP2Enabled: true,
8996
LogFormat: `[$time_iso8601] - $app_name - $remote_addr - $remote_user - $status - "$request" - $bytes_sent - "$http_referer" - "$http_user_agent" - "$server_name" - $upstream_addr - $http_host - $upstream_response_time - $request_time`,
90-
}
97+
ProxyBuffersConfig: proxyBuffersConfig,
98+
}, nil
9199
}
92100

93101
// GzipConfig encapsulates gzip configuration.
@@ -126,17 +134,23 @@ type AppConfig struct {
126134
CertMappings map[string]string `key:"certificates" constraint:"(?i)^((([a-z0-9]+(-*[a-z0-9]+)*)|((\\*\\.)?[a-z0-9]+(-*[a-z0-9]+)*\\.)+[a-z0-9]+(-*[a-z0-9]+)+):([a-z0-9]+(-*[a-z0-9]+)*)(\\s*,\\s*)?)+$"`
127135
Certificates map[string]*Certificate
128136
Available bool
129-
Maintenance bool `key:"maintenance" constraint:"(?i)^(true|false)$"`
130-
SSLConfig *SSLConfig `key:"ssl"`
137+
Maintenance bool `key:"maintenance" constraint:"(?i)^(true|false)$"`
138+
SSLConfig *SSLConfig `key:"ssl"`
139+
Nginx *NginxAppConfig `key:"nginx"`
131140
}
132141

133-
func newAppConfig(routerConfig *RouterConfig) *AppConfig {
142+
func newAppConfig(routerConfig *RouterConfig) (*AppConfig, error) {
143+
nginxConfig, err := newNginxAppConfig(routerConfig)
144+
if err != nil {
145+
return nil, err
146+
}
134147
return &AppConfig{
135148
ConnectTimeout: "30s",
136149
TCPTimeout: routerConfig.DefaultTimeout,
137150
Certificates: make(map[string]*Certificate, 0),
138151
SSLConfig: newSSLConfig(),
139-
}
152+
Nginx: nginxConfig,
153+
}, nil
140154
}
141155

142156
// BuilderConfig encapsulates the configuration of the deis-builder-- if it's in use.
@@ -215,6 +229,55 @@ func newHSTSConfig() *HSTSConfig {
215229
}
216230
}
217231

232+
// NginxAppConfig is a wrapper for all Nginx-specific app configurations. These
233+
// options shouldn't be expected to be universally supported by alternative
234+
// router implementations.
235+
type NginxAppConfig struct {
236+
ProxyBuffersConfig *ProxyBuffersConfig `key:"proxyBuffers"`
237+
}
238+
239+
func newNginxAppConfig(routerConfig *RouterConfig) (*NginxAppConfig, error) {
240+
proxyBuffersConfig, err := newProxyBuffersConfig(routerConfig.ProxyBuffersConfig)
241+
if err != nil {
242+
return nil, err
243+
}
244+
return &NginxAppConfig{
245+
ProxyBuffersConfig: proxyBuffersConfig,
246+
}, nil
247+
}
248+
249+
// ProxyBuffersConfig represents configuration options having to do with Nginx
250+
// proxy buffers.
251+
type ProxyBuffersConfig struct {
252+
Enabled bool `key:"enabled" constraint:"(?i)^(true|false)$"`
253+
Number int `key:"number" constraint:"^[1-9]\\d*$"`
254+
Size string `key:"size" constraint:"^[1-9]\\d*[kKmM]?$"`
255+
BusySize string `key:"busySize" constraint:"^[1-9]\\d*[kKmM]?$"`
256+
}
257+
258+
func newProxyBuffersConfig(proxyBuffersConfig *ProxyBuffersConfig) (*ProxyBuffersConfig, error) {
259+
if proxyBuffersConfig != nil {
260+
var buf bytes.Buffer
261+
enc := gob.NewEncoder(&buf)
262+
dec := gob.NewDecoder(&buf)
263+
err := enc.Encode(proxyBuffersConfig)
264+
if err != nil {
265+
return nil, err
266+
}
267+
var copy *ProxyBuffersConfig
268+
err = dec.Decode(&copy)
269+
if err != nil {
270+
return nil, err
271+
}
272+
return copy, nil
273+
}
274+
return &ProxyBuffersConfig{
275+
Number: 8,
276+
Size: "4k",
277+
BusySize: "8k",
278+
}, nil
279+
}
280+
218281
// Build creates a RouterConfig configuration object by querying the k8s API for
219282
// relevant metadata concerning itself and all routable services.
220283
func Build(kubeClient *kubernetes.Clientset) (*RouterConfig, error) {
@@ -328,8 +391,11 @@ func build(kubeClient *kubernetes.Clientset, routerDeployment *v1beta1ext.Deploy
328391
}
329392

330393
func buildRouterConfig(routerDeployment *v1beta1.Deployment, platformCertSecret *v1.Secret, dhParamSecret *v1.Secret) (*RouterConfig, error) {
331-
routerConfig := newRouterConfig()
332-
err := modeler.MapToModel(routerDeployment.Annotations, "nginx", routerConfig)
394+
routerConfig, err := newRouterConfig()
395+
if err != nil {
396+
return nil, err
397+
}
398+
err = modeler.MapToModel(routerDeployment.Annotations, "nginx", routerConfig)
333399
if err != nil {
334400
return nil, err
335401
}
@@ -351,7 +417,10 @@ func buildRouterConfig(routerDeployment *v1beta1.Deployment, platformCertSecret
351417
}
352418

353419
func buildAppConfig(kubeClient *kubernetes.Clientset, service v1.Service, routerConfig *RouterConfig) (*AppConfig, error) {
354-
appConfig := newAppConfig(routerConfig)
420+
appConfig, err := newAppConfig(routerConfig)
421+
if err != nil {
422+
return nil, err
423+
}
355424
appConfig.Name = service.Labels["app"]
356425
// If we didn't get the app name from the app label, fall back to inferring the app name from
357426
// the service's own name.
@@ -363,7 +432,7 @@ func buildAppConfig(kubeClient *kubernetes.Clientset, service v1.Service, router
363432
if appConfig.Name != service.Namespace {
364433
appConfig.Name = service.Namespace + "/" + appConfig.Name
365434
}
366-
err := modeler.MapToModel(service.Annotations, "", appConfig)
435+
err = modeler.MapToModel(service.Annotations, "", appConfig)
367436
if err != nil {
368437
return nil, err
369438
}

‎model/model_test.go

+4-1
Original file line numberDiff line numberDiff line change
@@ -84,7 +84,10 @@ func TestBuildRouterConfig(t *testing.T) {
8484
},
8585
}
8686

87-
expectedConfig := newRouterConfig()
87+
expectedConfig, err := newRouterConfig()
88+
if err != nil {
89+
t.Error(err)
90+
}
8891
sslConfig := newSSLConfig()
8992
hstsConfig := newHSTSConfig()
9093
platformCert := newCertificate("foo", "bar")

‎model/model_validation_test.go

+77-17
Original file line numberDiff line numberDiff line change
@@ -343,51 +343,111 @@ func TestValidHSTSPreload(t *testing.T) {
343343
testValidValues(t, newTestHSTSConfig, "Preload", "preload", []string{"true", "false", "TRUE", "FALSE"})
344344
}
345345

346-
func testInvalidValues(t *testing.T, builder func() interface{}, fieldName string, key string, badValues []string) {
346+
func TestInvalidProxyBuffersEnabled(t *testing.T) {
347+
testInvalidValues(t, newTestProxyBuffersConfig, "Enabled", "enabled", []string{"0", "-1", "foobar"})
348+
}
349+
350+
func TestValidProxyBuffersEnabled(t *testing.T) {
351+
testValidValues(t, newTestProxyBuffersConfig, "Enabled", "enabled", []string{"true", "false", "TRUE", "FALSE"})
352+
}
353+
354+
func TestInvalidProxyBuffersNumber(t *testing.T) {
355+
testInvalidValues(t, newTestProxyBuffersConfig, "Number", "number", []string{"0", "-1", "foobar"})
356+
}
357+
358+
func TestValidProxyBuffersNumber(t *testing.T) {
359+
testValidValues(t, newTestProxyBuffersConfig, "Number", "number", []string{"1", "2", "10"})
360+
}
361+
362+
func TestInvalidProxyBuffersSize(t *testing.T) {
363+
testInvalidValues(t, newTestProxyBuffersConfig, "Size", "size", []string{"0", "-1", "foobar"})
364+
}
365+
366+
func TestValidProxyBuffersSize(t *testing.T) {
367+
testValidValues(t, newTestProxyBuffersConfig, "Size", "size", []string{"1", "2", "20", "1k", "2k", "10m", "10M"})
368+
}
369+
370+
func TestInvalidProxyBuffersBusySize(t *testing.T) {
371+
testInvalidValues(t, newTestProxyBuffersConfig, "BusySize", "busySize", []string{"0", "-1", "foobar"})
372+
}
373+
374+
func TestValidProxyBusyBuffersBusySize(t *testing.T) {
375+
testValidValues(t, newTestProxyBuffersConfig, "BusySize", "busySize", []string{"1", "2", "20", "1k", "2k", "10m", "10M"})
376+
}
377+
378+
func testInvalidValues(
379+
t *testing.T,
380+
builder func() (interface{}, error),
381+
fieldName string,
382+
key string,
383+
badValues []string,
384+
) {
347385
badMap := make(map[string]string, 1)
348386
for _, badValue := range badValues {
349387
badMap[key] = badValue
350-
model := builder()
351-
err := testModeler.MapToModel(badMap, "", model)
388+
model, err := builder()
389+
if err != nil {
390+
t.Errorf("Unexpected error: %s", err)
391+
t.FailNow()
392+
}
393+
err = testModeler.MapToModel(badMap, "", model)
352394
checkError(t, badValue, err)
353395
}
354396
}
355397

356-
func testValidValues(t *testing.T, builder func() interface{}, fieldName string, key string, goodValues []string) {
398+
func testValidValues(
399+
t *testing.T,
400+
builder func() (interface{}, error),
401+
fieldName string,
402+
key string,
403+
goodValues []string,
404+
) {
357405
goodMap := make(map[string]string, 1)
358406
for _, goodValue := range goodValues {
359407
goodMap[key] = goodValue
360-
model := builder()
361-
err := testModeler.MapToModel(goodMap, "", model)
408+
model, err := builder()
409+
if err != nil {
410+
t.Errorf("Unexpected error: %s", err)
411+
t.FailNow()
412+
}
413+
err = testModeler.MapToModel(goodMap, "", model)
362414
if err != nil {
363415
t.Errorf("Using value \"%s\", received an unexpected error: %s", goodValue, err)
364416
t.FailNow()
365417
}
366418
}
367419
}
368420

369-
func newTestRouterConfig() interface{} {
421+
func newTestRouterConfig() (interface{}, error) {
370422
return newRouterConfig()
371423
}
372424

373-
func newTestGzipConfig() interface{} {
374-
return newGzipConfig()
425+
func newTestGzipConfig() (interface{}, error) {
426+
return newGzipConfig(), nil
427+
}
428+
429+
func newTestAppConfig() (interface{}, error) {
430+
routerConfig, err := newRouterConfig()
431+
if err != nil {
432+
return nil, err
433+
}
434+
return newAppConfig(routerConfig)
375435
}
376436

377-
func newTestAppConfig() interface{} {
378-
return newAppConfig(newRouterConfig())
437+
func newTestBuilderConfig() (interface{}, error) {
438+
return newBuilderConfig(), nil
379439
}
380440

381-
func newTestBuilderConfig() interface{} {
382-
return newBuilderConfig()
441+
func newTestSSLConfig() (interface{}, error) {
442+
return newSSLConfig(), nil
383443
}
384444

385-
func newTestSSLConfig() interface{} {
386-
return newSSLConfig()
445+
func newTestHSTSConfig() (interface{}, error) {
446+
return newHSTSConfig(), nil
387447
}
388448

389-
func newTestHSTSConfig() interface{} {
390-
return newHSTSConfig()
449+
func newTestProxyBuffersConfig() (interface{}, error) {
450+
return newProxyBuffersConfig(nil)
391451
}
392452

393453
func checkError(t *testing.T, value string, err error) {

‎nginx/config.go

+9-2
Original file line numberDiff line numberDiff line change
@@ -146,7 +146,10 @@ http {
146146
}
147147
148148
location / {
149-
proxy_buffering off;
149+
proxy_buffering {{ if $routerConfig.ProxyBuffersConfig.Enabled }}on{{ else }}off{{ end }};
150+
proxy_buffer_size {{ $routerConfig.ProxyBuffersConfig.Size }};
151+
proxy_buffers {{ $routerConfig.ProxyBuffersConfig.Number }} {{ $routerConfig.ProxyBuffersConfig.Size }};
152+
proxy_busy_buffers_size {{ $routerConfig.ProxyBuffersConfig.BusySize }};
150153
proxy_set_header Host $host;
151154
proxy_set_header X-Forwarded-For $remote_addr;
152155
proxy_set_header X-Forwarded-Proto $access_scheme;
@@ -242,7 +245,11 @@ http {
242245
add_header X-Correlation-Id $correlation_id always;
243246
{{end}}
244247
245-
{{ if $appConfig.Maintenance }}return 503;{{ else if $appConfig.Available }}proxy_buffering off;
248+
{{ if $appConfig.Maintenance }}return 503;{{ else if $appConfig.Available }}
249+
proxy_buffering {{ if $appConfig.Nginx.ProxyBuffersConfig.Enabled }}on{{ else }}off{{ end }};
250+
proxy_buffer_size {{ $appConfig.Nginx.ProxyBuffersConfig.Size }};
251+
proxy_buffers {{ $appConfig.Nginx.ProxyBuffersConfig.Number }} {{ $appConfig.Nginx.ProxyBuffersConfig.Size }};
252+
proxy_busy_buffers_size {{ $appConfig.Nginx.ProxyBuffersConfig.BusySize }};
246253
proxy_set_header Host $host;
247254
proxy_set_header X-Forwarded-For $remote_addr;
248255
proxy_set_header X-Forwarded-Proto $access_scheme;

0 commit comments

Comments
 (0)