From 15926050b02f3e25abaf16e8bdee880eb26bf4f1 Mon Sep 17 00:00:00 2001 From: Harvey Tuch Date: Wed, 2 Sep 2020 13:41:21 -0400 Subject: [PATCH 1/5] dependencies: initial external dependency policy. This will apply to all changes to external dependencies in future PRs. Signed-off-by: Harvey Tuch --- DEPENDENCY_POLICY.md | 91 +++++++++++++++++++++++++++++++++ EXTENSION_POLICY.md | 2 + tools/protodoc/requirements.txt | 13 ++++- 3 files changed, 105 insertions(+), 1 deletion(-) create mode 100644 DEPENDENCY_POLICY.md diff --git a/DEPENDENCY_POLICY.md b/DEPENDENCY_POLICY.md new file mode 100644 index 000000000000..9ff439afda3b --- /dev/null +++ b/DEPENDENCY_POLICY.md @@ -0,0 +1,91 @@ +# Envoy External Dependency Policy + +Envoy has an evolving policy on external dependencies, tracked at +https://github.com/envoyproxy/envoy/issues/10471. This will become stricter over time, below we +detail the policy as it currently applies. + +## Declaring external dependencies + +In general, all external dependencies for the Envoy proxy binary build and test should be declared +in either [bazel/repository_locations.bzl](bazel/repository_locations.bzl) or +[api/bazel/repository_locations.bzl](api/bazel/repository_locations.bzl), unless listed under +[policy exceptions](#policy-exceptions). + +An example entry for the `nghttp2` dependency is: + +```python +com_github_nghttp2_nghttp2 = dict( + project_name = "Nghttp2", + project_url = "https://nghttp2.org", + version = "1.41.0", + sha256 = "eacc6f0f8543583ecd659faf0a3f906ed03826f1d4157b536b4b385fe47c5bb8", + strip_prefix = "nghttp2-{version}", + urls = ["https://github.com/nghttp2/nghttp2/releases/download/v{version}/nghttp2-{version}.tar.gz"], + use_category = ["dataplane"], + cpe = "cpe:2.3:a:nghttp2:nghttp2:*", +), +``` + +Dependency declarations must: + +* Provide a meaningful project name and URL. +* State the version in the `version` field. String interpolation should be used in `strip_prefix` + and `urls` to reference the version. If you need to reference version `X.Y.Z` as `X_Y_Z`, this + may appear in a string as `{version}.replace('.','_')`. +* Provide accurate entries for `use_category`. Please think carefully about whether there are data + or control plane implications of the dependency. +* CPEs are compulsory for all dependencies that are not purely build/test. + [CPEs](https://en.wikipedia.org/wiki/Common_Platform_Enumeration) provide metadata that allow us + to correlate with related CVEs in dashboards and other tooling, and also provide a machine + consumable join key. You can consult the latest [CPE + dictionary](https://nvd.nist.gov/products/cpe) to find a CPE for a dependency.`"N/A"` should only + be used if no CPE for the project is available in the CPE database. CPEs should be _versionless_ + with a `:*` suffix, since the version can be computed from `version`. + +When build or test code references Python modules, they should be imported via `pip3_import` in +[bazel/repositories_extra.bzl](bazel/repositories_extra.bzl). Python modules should not be listed in +`repository_locations.bzl` entries. `requirements.txt` files for Python dependencies must pin to +exact versions, e.g. `PyYAML==5.3.1` and ideally also include a [SHA256 +checksum](https://davidwalsh.name/hashin). + +Pure developer tooling and documentation builds may reference Python via standalone +`requirements.txt`, following the above policy. + +## New external dependencies + +* Any new dependency on the Envoy data or control plane that impacts Envoy core (i.e. is not + specific to a single non-core extension) must be cleared with the OSS Envoy security team, please + file an issue and tag + [@envoyproxy/security-team](https://github.com/orgs/envoyproxy/teams/security-team). While policy + is still [evolving](robust_to_untrusted_downstream_and_upstream), criteria that will be used in + evaluation include: + * Does the project have release versions? How often do releases happen? + * Does the project have a security vulnerability disclosure process and contact details? + * Does the project have effective governance, e.g. multiple maintainers, a governance policy? + * Does the project have a code review culture? Are patches reviewed by independent maintainers + prior to merge? + * Does the project enable mandatory GitHub 2FA for contributors? + * Does the project have evidence of high test coverage, fuzzing, static analysis (e.g. CodeQL), + etc.? + +* Dependencies for extensions that are tagged as `robust_to_untrusted_downstream` or + `robust_to_untrusted_downstream_and_upstream` should be sensitive to the same set of concerns + as the core data plane. + +## Maintaining existing dependencies + +We rely on community volunteers to help track the latest versions of dependencies. On a best effort +basis: + +* Core Envoy dependencies will be updated by the Envoy maintainers/security team. + +* Extension [CODEOWNERS](CODEOWNERS) should update extension specific dependencies. + +## Policy exceptions + +The following dependencies are exempt from the policy: + +* Any developer-only facing tooling or the documentation build. + +* Transitive build time dependencies, e.g. Go projects vendored into + [protoc-gen-validate](https://github.com/envoyproxy/protoc-gen-validate). diff --git a/EXTENSION_POLICY.md b/EXTENSION_POLICY.md index d7f1df8334c5..5b096f5b8cbf 100644 --- a/EXTENSION_POLICY.md +++ b/EXTENSION_POLICY.md @@ -33,6 +33,8 @@ The following procedure will be used when proposing new extensions for inclusion 4. Any extension added via this process becomes a full part of the repository. This means that any API breaking changes in the core code will be automatically fixed as part of the normal PR process by other contributors. + 5. Any new dependencies added for this extension must comply with [DEPENDENCY_POLICY.md], please + follow the steps detailed there. ## Removing existing extensions diff --git a/tools/protodoc/requirements.txt b/tools/protodoc/requirements.txt index 7a997b5e44bd..c327c23fc27a 100644 --- a/tools/protodoc/requirements.txt +++ b/tools/protodoc/requirements.txt @@ -1 +1,12 @@ -PyYAML==5.3.1 +PyYAML==5.3.1 \ + --hash=sha256:06a0d7ba600ce0b2d2fe2e78453a470b5a6e000a985dd4a4e54e436cc36b0e97 \ + --hash=sha256:240097ff019d7c70a4922b6869d8a86407758333f02203e0fc6ff79c5dcede76 \ + --hash=sha256:4f4b913ca1a7319b33cfb1369e91e50354d6f07a135f3b901aca02aa95940bd2 \ + --hash=sha256:69f00dca373f240f842b2931fb2c7e14ddbacd1397d57157a9b005a6a9942648 \ + --hash=sha256:73f099454b799e05e5ab51423c7bcf361c58d3206fa7b0d555426b1f4d9a3eaf \ + --hash=sha256:74809a57b329d6cc0fdccee6318f44b9b8649961fa73144a98735b0aaf029f1f \ + --hash=sha256:7739fc0fa8205b3ee8808aea45e968bc90082c10aef6ea95e855e10abf4a37b2 \ + --hash=sha256:95f71d2af0ff4227885f7a6605c37fd53d3a106fcab511b8860ecca9fcf400ee \ + --hash=sha256:b8eac752c5e14d3eca0e6dd9199cd627518cb5ec06add0de9d32baeee6fe645d \ + --hash=sha256:cc8955cfbfc7a115fa81d85284ee61147059a753344bc51098f3ccd69b0d7e0c \ + --hash=sha256:d13155f591e6fcc1ec3b30685d50bf0711574e2c0dfffd7644babf8b5102ca1a From 6e22ae18352fda25521e0416a4f3aa6fdfb171a8 Mon Sep 17 00:00:00 2001 From: Harvey Tuch Date: Wed, 2 Sep 2020 15:13:36 -0400 Subject: [PATCH 2/5] Update GOVERNANCE.md, and switch to simpler string interpolation. Signed-off-by: Harvey Tuch --- DEPENDENCY_POLICY.md | 3 ++- GOVERNANCE.md | 5 +++++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/DEPENDENCY_POLICY.md b/DEPENDENCY_POLICY.md index 9ff439afda3b..9efeb0df4d67 100644 --- a/DEPENDENCY_POLICY.md +++ b/DEPENDENCY_POLICY.md @@ -31,7 +31,8 @@ Dependency declarations must: * Provide a meaningful project name and URL. * State the version in the `version` field. String interpolation should be used in `strip_prefix` and `urls` to reference the version. If you need to reference version `X.Y.Z` as `X_Y_Z`, this - may appear in a string as `{version}.replace('.','_')`. + may appear in a string as `{underscore_version}`, similarly for `X-Y-Z` you can use + `{dash_version}`. * Provide accurate entries for `use_category`. Please think carefully about whether there are data or control plane implications of the dependency. * CPEs are compulsory for all dependencies that are not purely build/test. diff --git a/GOVERNANCE.md b/GOVERNANCE.md index ce60a6e82ca7..64b818a6ed48 100644 --- a/GOVERNANCE.md +++ b/GOVERNANCE.md @@ -155,6 +155,11 @@ the maintainers per the voting process below. Adding new [extensions](REPO_LAYOUT.md#sourceextensions-layout) has a dedicated policy. Please see [this](./EXTENSION_POLICY.md) document for more information. +# Exernal dependency policy + +Adding new external dependencies has a dedicated policy. Please see [this](DEPENDENCY_POLICY.md) +document for more information. + # Conflict resolution and voting In general, we prefer that technical issues and maintainer membership are amicably worked out From 66597ee6ce8f1bfc0e5eb7e76c50c6a8bfc755e3 Mon Sep 17 00:00:00 2001 From: Harvey Tuch Date: Wed, 2 Sep 2020 15:16:48 -0400 Subject: [PATCH 3/5] Fix link in EXTENSION_POLICY.md. Signed-off-by: Harvey Tuch --- EXTENSION_POLICY.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/EXTENSION_POLICY.md b/EXTENSION_POLICY.md index 5b096f5b8cbf..bad23f6dffab 100644 --- a/EXTENSION_POLICY.md +++ b/EXTENSION_POLICY.md @@ -33,8 +33,8 @@ The following procedure will be used when proposing new extensions for inclusion 4. Any extension added via this process becomes a full part of the repository. This means that any API breaking changes in the core code will be automatically fixed as part of the normal PR process by other contributors. - 5. Any new dependencies added for this extension must comply with [DEPENDENCY_POLICY.md], please - follow the steps detailed there. + 5. Any new dependencies added for this extension must comply with + [DEPENDENCY_POLICY.md](DEPENDENCY_POLICY.md), please follow the steps detailed there. ## Removing existing extensions From d17e071a64a88b5af523e94e2ddfe397d1e6725f Mon Sep 17 00:00:00 2001 From: Harvey Tuch Date: Thu, 3 Sep 2020 13:58:30 -0400 Subject: [PATCH 4/5] Review feedback. Signed-off-by: Harvey Tuch --- DEPENDENCY_POLICY.md | 14 ++++++++++++-- tools/protodoc/requirements.txt | 13 +------------ 2 files changed, 13 insertions(+), 14 deletions(-) diff --git a/DEPENDENCY_POLICY.md b/DEPENDENCY_POLICY.md index 9efeb0df4d67..677a990544ad 100644 --- a/DEPENDENCY_POLICY.md +++ b/DEPENDENCY_POLICY.md @@ -4,6 +4,11 @@ Envoy has an evolving policy on external dependencies, tracked at https://github.com/envoyproxy/envoy/issues/10471. This will become stricter over time, below we detail the policy as it currently applies. +## External dependencies dashboard + +The list of external dependencies in Envoy with their current version is available at +https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/security/external_deps + ## Declaring external dependencies In general, all external dependencies for the Envoy proxy binary build and test should be declared @@ -33,6 +38,8 @@ Dependency declarations must: and `urls` to reference the version. If you need to reference version `X.Y.Z` as `X_Y_Z`, this may appear in a string as `{underscore_version}`, similarly for `X-Y-Z` you can use `{dash_version}`. +* Versions should prefer release versions over master branch GitHub SHA tarballs. A comment is + necessary if the latter is used. * Provide accurate entries for `use_category`. Please think carefully about whether there are data or control plane implications of the dependency. * CPEs are compulsory for all dependencies that are not purely build/test. @@ -55,8 +62,8 @@ Pure developer tooling and documentation builds may reference Python via standal ## New external dependencies * Any new dependency on the Envoy data or control plane that impacts Envoy core (i.e. is not - specific to a single non-core extension) must be cleared with the OSS Envoy security team, please - file an issue and tag + specific to a single non-core extension) must be cleared with the Envoy security team, please file + an issue and tag [@envoyproxy/security-team](https://github.com/orgs/envoyproxy/teams/security-team). While policy is still [evolving](robust_to_untrusted_downstream_and_upstream), criteria that will be used in evaluation include: @@ -82,6 +89,9 @@ basis: * Extension [CODEOWNERS](CODEOWNERS) should update extension specific dependencies. +Where possible, we prefer the latest release version for external dependencies, rather than master +branch GitHub SHA tarballs. + ## Policy exceptions The following dependencies are exempt from the policy: diff --git a/tools/protodoc/requirements.txt b/tools/protodoc/requirements.txt index c327c23fc27a..7a997b5e44bd 100644 --- a/tools/protodoc/requirements.txt +++ b/tools/protodoc/requirements.txt @@ -1,12 +1 @@ -PyYAML==5.3.1 \ - --hash=sha256:06a0d7ba600ce0b2d2fe2e78453a470b5a6e000a985dd4a4e54e436cc36b0e97 \ - --hash=sha256:240097ff019d7c70a4922b6869d8a86407758333f02203e0fc6ff79c5dcede76 \ - --hash=sha256:4f4b913ca1a7319b33cfb1369e91e50354d6f07a135f3b901aca02aa95940bd2 \ - --hash=sha256:69f00dca373f240f842b2931fb2c7e14ddbacd1397d57157a9b005a6a9942648 \ - --hash=sha256:73f099454b799e05e5ab51423c7bcf361c58d3206fa7b0d555426b1f4d9a3eaf \ - --hash=sha256:74809a57b329d6cc0fdccee6318f44b9b8649961fa73144a98735b0aaf029f1f \ - --hash=sha256:7739fc0fa8205b3ee8808aea45e968bc90082c10aef6ea95e855e10abf4a37b2 \ - --hash=sha256:95f71d2af0ff4227885f7a6605c37fd53d3a106fcab511b8860ecca9fcf400ee \ - --hash=sha256:b8eac752c5e14d3eca0e6dd9199cd627518cb5ec06add0de9d32baeee6fe645d \ - --hash=sha256:cc8955cfbfc7a115fa81d85284ee61147059a753344bc51098f3ccd69b0d7e0c \ - --hash=sha256:d13155f591e6fcc1ec3b30685d50bf0711574e2c0dfffd7644babf8b5102ca1a +PyYAML==5.3.1 From cff986f87fbff46f0be84f0024a97353a1e3b6c5 Mon Sep 17 00:00:00 2001 From: Harvey Tuch Date: Tue, 8 Sep 2020 11:40:45 -0400 Subject: [PATCH 5/5] Non-release comment format. Signed-off-by: Harvey Tuch --- DEPENDENCY_POLICY.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/DEPENDENCY_POLICY.md b/DEPENDENCY_POLICY.md index 677a990544ad..50aad88708aa 100644 --- a/DEPENDENCY_POLICY.md +++ b/DEPENDENCY_POLICY.md @@ -39,7 +39,8 @@ Dependency declarations must: may appear in a string as `{underscore_version}`, similarly for `X-Y-Z` you can use `{dash_version}`. * Versions should prefer release versions over master branch GitHub SHA tarballs. A comment is - necessary if the latter is used. + necessary if the latter is used. This comment should contain the reason that a non-release + version is being used and the YYYY-MM-DD when the last update was performed. * Provide accurate entries for `use_category`. Please think carefully about whether there are data or control plane implications of the dependency. * CPEs are compulsory for all dependencies that are not purely build/test.