From 74351d9325e342fc9c29517ee4f321e26eba6661 Mon Sep 17 00:00:00 2001 From: Nick Young Date: Thu, 20 Jun 2019 16:04:00 +1000 Subject: [PATCH 1/4] Add Secure Communication design doc Updates #881 Updates #862 Signed-off-by: Nick Young --- design/secure-communication.md | 94 ++++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+) create mode 100644 design/secure-communication.md diff --git a/design/secure-communication.md b/design/secure-communication.md new file mode 100644 index 00000000000..d80ad328914 --- /dev/null +++ b/design/secure-communication.md @@ -0,0 +1,94 @@ +# Secure Communication for Contour and Envoy + +Status: Draft + +This document outlines a design for securing gRPC communication between Contour and Envoy, including all the xDS endpoints. + +## Goals + +- Achieve secure communication between Contour and Envoy, which will allow a split deployment. +- Allow use of the SDS service in Contour with authentication. + +## Non Goals + +- Certificate management, including creation or rotation. These are left as an exercise for the reader at this time. + +## Definitions + +### mTLS + +In this document, we use **mTLS** as shorthand for 'mutual TLS client certificate authentication', without implying a key lifetime. That is, we're just talking about both sides having TLS certs, not implying that we're using the more modern form which implies dynamic, short-lifetime certificates managed by an automated mechanism. + +## Background + +Currently, the default deployment of Contour includes a Contour and an Envoy container, that connect via the Pod's loopback interface. This uses the pod-level security boundary to provide transport security (no-one can listen in) and authentication (no-one other than Envoy can connect). + +In order to be able to split the deployment of Contour and Envoy, we must be able to provide an acceptable level of both transport security and authentication between the different pods. + +Currently, if you do split the deployment, Contour will assume that anything that talks to the xDS endpoints is authorized for all information in them. Notably, this includes any certificates used in any endpoint. Antyhing that can speak SDS can connect to contour and ask for all the certificates it knows about via SDS. This design is intended to mitigate this problem. + +It's also important to note that the certificates talked about in this document are **only** the certificates used to secure the gRPC channel between Contour and Envoy, not any certificates used to serve TLS out of Envoy. As such, this design is wholly separate to any certificate functionality on either Ingress or IngressRoute. + +## High-Level Design + +We will add configuration to Contour, both for `contour serve` and `contour bootstrap`, to have Contour and Envoy use a shared CA and a keypair each. This will allow a basic split deployment with minimal changes to the current code, using simple Secret mounts into file locations in both a Contour Deployment and Envoy deployment or DaemonSet. + +The generation of the certificates and their placement into Secrets in an operable and maintainable way is not in scope for this change, only a basic illustration in the example deployment YAMLs and Howto. However, see [Future Work](#future-work) for more info. + +## Detailed Design - mTLS + +In order for mTLS to work, each end must know: +- The CA that the TLS relationship will descend from. +- Its certificate +- Its key + +This applies both to Contour and Envoy. + +To accomplish this, we will add the following new command line options: + +To `contour serve`, covering the Contour side: +- `--contour-cafile`: the CA Bundle for the mutual TLS relationship +- `--contour-mtls-cert`: The cert for Contour to use to identify itself. (`serve`) +- `--contour-mtls-key`: The key for Contour to use for its side of the mTLS pairing. (`serve`) + +And to `contour bootstrap`, covering Envoy configuration: +- `--envoy-cafile`: The CA Bundle for the mutual TLS relationship +- `--envoy-mtls-cert`: The cert for Envoy to use to identify itself. +- `--envoy-mtls-key`: The key for Envoy to use to identify itself. + +If any one of the three options is passed to either `serve` or `bootstrap`, the other two are required, and we will error out if they are not passed. + +We will wire these options into the gRPC server establishment process on the Contour side, and supply them in the bootstrap on the Envoy side. This is where the most significant code changes will be required. + +We will update the example deployment manifests for the separate deployment options to use these new options, and provide a basic sample howto on generating a CA keypair and mTLS keypairs using command line tools like openssl. + +## Security Considerations + +In the current design, these certs require the rolling of the deployment to change, they should be long-lived certs (days, weeks, or months), not short-lived (minutes or hours). The CA Keypair should be very long-lived and very tightly controlled, as Contour/Envoy connection security is only as secure as the CA keypair. + +As noted above, this is an initial design to *start* the process of being able to run reasonably secure, authenticated, separate deployments of Contour and Envoy. Some things that we *will* address at a later date, but are out of scope for now: + +- Certificate generation (apart from a very simple howto to allow users to see how the feature works) +- Certificate rotation + +Implementing this change will, however, improve the security of the split-deployment scenario. + +## Future Work + +This work is the first phase of a planned three-phase rollout. + +### Phase 1: mTLS + +This document - allows a more secure split deployment in a very basic way. + +### Phase 2: User Experience + +This phase will add additional functionality and/or tools to help with the certificate generation and rolling process. Watch [#1184][1] to track this phase. + +### Phase 3: Secure by default + +This phase will flip the example deployments to all use a secure mode, make those command line options default, and include an option to restore the current behaviour. Watch [#1185][2] to track this phase. + + +[1]: https://github.com/heptio/contour/issues/1184 +[2]: https://github.com/heptio/contour/issues/1185 \ No newline at end of file From 0d93b12fe092a4219d1b020ae109a85280242dd9 Mon Sep 17 00:00:00 2001 From: Nick Young Date: Fri, 21 Jun 2019 09:42:15 +1000 Subject: [PATCH 2/4] Update design doc with PR comments Signed-off-by: Nick Young --- design/secure-communication.md | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/design/secure-communication.md b/design/secure-communication.md index d80ad328914..a85cb96f2ae 100644 --- a/design/secure-communication.md +++ b/design/secure-communication.md @@ -2,16 +2,15 @@ Status: Draft -This document outlines a design for securing gRPC communication between Contour and Envoy, including all the xDS endpoints. +This document outlines a design for securing gRPC communication between Contour and Envoy. ## Goals -- Achieve secure communication between Contour and Envoy, which will allow a split deployment. -- Allow use of the SDS service in Contour with authentication. +- Achieve secure communication between Contour and Envoy, which will allow a secure split deployment. ## Non Goals -- Certificate management, including creation or rotation. These are left as an exercise for the reader at this time. +- Certificate management, including creation or rotation. These must be provided by the user. In the future we plan to automated this process to allow split deployments to become the default configuration. ## Definitions @@ -21,7 +20,7 @@ In this document, we use **mTLS** as shorthand for 'mutual TLS client certificat ## Background -Currently, the default deployment of Contour includes a Contour and an Envoy container, that connect via the Pod's loopback interface. This uses the pod-level security boundary to provide transport security (no-one can listen in) and authentication (no-one other than Envoy can connect). +Currently, the default deployment of Contour colocates the Envoy and Contour containers in the same pod communicating over the pod's loopback interface. This uses the pod-level security boundary to provide transport security (no-one can listen in) and authentication (no-one other than Envoy can connect). In order to be able to split the deployment of Contour and Envoy, we must be able to provide an acceptable level of both transport security and authentication between the different pods. @@ -31,9 +30,9 @@ It's also important to note that the certificates talked about in this document ## High-Level Design -We will add configuration to Contour, both for `contour serve` and `contour bootstrap`, to have Contour and Envoy use a shared CA and a keypair each. This will allow a basic split deployment with minimal changes to the current code, using simple Secret mounts into file locations in both a Contour Deployment and Envoy deployment or DaemonSet. +We will add configuration to Contour, both for `contour serve` and `contour bootstrap`, to have Contour and Envoy use a shared CA and a keypair each. This will allow a basic split deployment, using simple Secret mounts into file locations in both a Contour Deployment and Envoy deployment or DaemonSet. -The generation of the certificates and their placement into Secrets in an operable and maintainable way is not in scope for this change, only a basic illustration in the example deployment YAMLs and Howto. However, see [Future Work](#future-work) for more info. +The generation of the certificates and their placement into Secrets is not in scope for this change, only a basic illustration in the example deployment YAMLs and Howto. However, see [Future Work](#future-work) for more info. ## Detailed Design - mTLS @@ -64,14 +63,14 @@ We will update the example deployment manifests for the separate deployment opti ## Security Considerations -In the current design, these certs require the rolling of the deployment to change, they should be long-lived certs (days, weeks, or months), not short-lived (minutes or hours). The CA Keypair should be very long-lived and very tightly controlled, as Contour/Envoy connection security is only as secure as the CA keypair. +In the current design, the process must be restarted to pick up a change to these certs. Because of this, they should be long-lived certs (days, weeks, or months), not short-lived (minutes or hours). The CA Keypair should be very long-lived and very tightly controlled, as Contour/Envoy connection security is only as secure as the CA keypair. As noted above, this is an initial design to *start* the process of being able to run reasonably secure, authenticated, separate deployments of Contour and Envoy. Some things that we *will* address at a later date, but are out of scope for now: - Certificate generation (apart from a very simple howto to allow users to see how the feature works) - Certificate rotation -Implementing this change will, however, improve the security of the split-deployment scenario. +Implementing this change will, however, enable a secure split-deployment scenario. ## Future Work @@ -87,7 +86,7 @@ This phase will add additional functionality and/or tools to help with the certi ### Phase 3: Secure by default -This phase will flip the example deployments to all use a secure mode, make those command line options default, and include an option to restore the current behaviour. Watch [#1185][2] to track this phase. +This phase will flip the example deployments to all use a secure mode, make those command line options default, and include an option to restore the current behavior. Watch [#1185][2] to track this phase. [1]: https://github.com/heptio/contour/issues/1184 From 48295645764484ba6cb9abfdf62e2fb77bdd91bb Mon Sep 17 00:00:00 2001 From: Nick Young Date: Fri, 21 Jun 2019 11:35:56 +1000 Subject: [PATCH 3/4] Move to accepted. Signed-off-by: Nick Young --- design/secure-communication.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/design/secure-communication.md b/design/secure-communication.md index a85cb96f2ae..4c9ac1724c6 100644 --- a/design/secure-communication.md +++ b/design/secure-communication.md @@ -1,6 +1,6 @@ # Secure Communication for Contour and Envoy -Status: Draft +Status: Accepted This document outlines a design for securing gRPC communication between Contour and Envoy. From 82f1b8a72ab947167fa39e2753ae334759beb678 Mon Sep 17 00:00:00 2001 From: Nick Young Date: Fri, 21 Jun 2019 12:50:39 +1000 Subject: [PATCH 4/4] Add note about config file (#1130) Signed-off-by: Nick Young --- design/secure-communication.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/design/secure-communication.md b/design/secure-communication.md index 4c9ac1724c6..e5659489ac1 100644 --- a/design/secure-communication.md +++ b/design/secure-communication.md @@ -50,6 +50,8 @@ To `contour serve`, covering the Contour side: - `--contour-mtls-cert`: The cert for Contour to use to identify itself. (`serve`) - `--contour-mtls-key`: The key for Contour to use for its side of the mTLS pairing. (`serve`) +(Note that these `contour serve` options may also move to a configuration file under [#1130][1]) + And to `contour bootstrap`, covering Envoy configuration: - `--envoy-cafile`: The CA Bundle for the mutual TLS relationship - `--envoy-mtls-cert`: The cert for Envoy to use to identify itself. @@ -82,12 +84,12 @@ This document - allows a more secure split deployment in a very basic way. ### Phase 2: User Experience -This phase will add additional functionality and/or tools to help with the certificate generation and rolling process. Watch [#1184][1] to track this phase. +This phase will add additional functionality and/or tools to help with the certificate generation and rolling process. Watch [#1184][2] to track this phase. ### Phase 3: Secure by default -This phase will flip the example deployments to all use a secure mode, make those command line options default, and include an option to restore the current behavior. Watch [#1185][2] to track this phase. - +This phase will flip the example deployments to all use a secure mode, make those command line options default, and include an option to restore the current behavior. Watch [#1185][3] to track this phase. -[1]: https://github.com/heptio/contour/issues/1184 -[2]: https://github.com/heptio/contour/issues/1185 \ No newline at end of file +[1]: https://github.com/heptio/contour/issues/1130 +[2]: https://github.com/heptio/contour/issues/1184 +[3]: https://github.com/heptio/contour/issues/1185