forked from envoyproxy/envoy
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Reconcile envoyproxy/data-plane-api and envoyproxy/envoy (envoyproxy#…
…3036) This PR implements the planned merge of envoyproxy/data-plane-api into envoyproxy/envoy as described in envoyproxy#2934 and https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!topic/envoy-dev/KcVHFH-zQwQ. Risk Level: Medium (there might be unintentional breakage of dependent builds). Testing: CI passes. There is now an additional bazel.api do_ci.sh target to build and run API tests. Fixes envoyproxy#2934. Signed-off-by: Harvey Tuch <htuch@google.com>
- Loading branch information
Showing
381 changed files
with
28,242 additions
and
129 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,3 @@ | ||
| licenses(["notice"]) # Apache 2 | ||
|
|
||
| exports_files(["VERSION"]) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| 1.7.0-dev |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,128 @@ | ||
| # Envoy v2 APIs for developers | ||
|
|
||
| ## Goals | ||
|
|
||
| This repository contains both the implemented and draft v2 JSON REST and gRPC | ||
| [Envoy](https://github.com/envoyproxy/envoy/) APIs. | ||
|
|
||
| Version 2 of the Envoy API evolves existing APIs and introduces new APIs to: | ||
|
|
||
| * Allow for more advanced load balancing through load and resource utilization reporting to management servers. | ||
| * Improve N^2 health check scalability issues by optionally offloading health checking to other Envoy instances. | ||
| * Support Envoy deployment in edge, sidecar and middle proxy deployment models via changes to the listener model and CDS/SDS APIs. | ||
| * Allow streaming updates from the management server on change, instead of polling APIs from Envoy. gRPC APIs will be supported | ||
| alongside JSON REST APIs to provide for this. | ||
| * Ensure all Envoy runtime configuration is dynamically discoverable via API | ||
| calls, including listener configuration, certificates and runtime settings, which are today sourced from the filesystem. There | ||
| will still remain a static bootstrap configuration file that will specify items | ||
| unlikely to change during runtime, including the Envoy node identity, xDS | ||
| management server addresses, administration interface and tracing | ||
| configuration. | ||
| * Revisit and where appropriate cleanup any v1 technical debt. | ||
|
|
||
| ## Status | ||
|
|
||
| See | ||
| [here](https://www.envoyproxy.io/docs/envoy/latest/configuration/overview/v2_overview.html#status) | ||
| for the current status of the v2 APIs. | ||
|
|
||
| See [here](CONTRIBUTING.md#api-changes) for the v2 API change process. | ||
|
|
||
| ## Principles | ||
|
|
||
| * [Proto3](https://developers.google.com/protocol-buffers/docs/proto3) will be | ||
| used to specify the canonical API. This will provide directly the gRPC API and | ||
| via gRPC-JSON transcoding the JSON REST API. A textual YAML input will be | ||
| supported for filesystem configuration files (e.g. the bootstrap file), in | ||
| addition to JSON, as a syntactic convenience. YAML file contents will be | ||
| internally converted to JSON and then follow the standard JSON-proto3 | ||
| conversion during Envoy config ingestion. | ||
|
|
||
| * xDS APIs should support eventual consistency. For example, if RDS references a | ||
| cluster that has not yet been supplied by CDS, it should be silently ignored | ||
| and traffic not forwarded until the CDS update occurs. Stronger consistency | ||
| guarantees are possible if the management server is able to sequence the xDS | ||
| APIs carefully (for example by using the ADS API below). By following the | ||
| `[CDS, EDS, LDS, RDS]` sequence for all pertinent resources, it will be | ||
| possible to avoid traffic outages during configuration update. | ||
|
|
||
| * The API is primarily intended for machine generation and consumption. It is | ||
| expected that the management server is responsible for mapping higher level | ||
| configuration concepts to API responses. Similarly, static configuration | ||
| fragments may be generated by templating tools, etc. The APIs and tools | ||
| used to generate xDS configuration are beyond the scope of the definitions in | ||
| this repository. | ||
|
|
||
| * REST-JSON API equivalents will be provided for the basic singleton xDS | ||
| subscription services CDS/EDS/LDS/RDS/SDS. Advanced APIs such as HDS, ADS and | ||
| EDS multi-dimensional LB will be gRPC only. This avoids having to map | ||
| complicated bidirectional stream semantics onto REST. | ||
|
|
||
| * Listeners will be immutable. Any updates to a listener via LDS will require | ||
| the draining of existing connections for the specific bound IP/port. As a | ||
| result, new requests will only be guaranteed to observe the new configuration | ||
| after existing connections have drained or the drain timeout. | ||
|
|
||
| * Versioning will be expressed via [proto3 package | ||
| namespaces](https://developers.google.com/protocol-buffers/docs/proto3#packages), | ||
| i.e. `package envoy.api.v2;`. | ||
|
|
||
| * Custom components (e.g. filters, resolvers, loggers) will use a reverse DNS naming scheme, | ||
| e.g. `com.google.widget`, `com.lyft.widget`. | ||
|
|
||
| ## APIs | ||
|
|
||
| Unless otherwise stated, the APIs with the same names as v1 APIs have a similar role. | ||
|
|
||
| * [Cluster Discovery Service (CDS)](envoy/api/v2/cds.proto). | ||
| * [Endpoint Discovery Service (EDS)](envoy/api/v2/eds.proto). This has the same role as SDS in the [v1 API](https://www.envoyproxy.io/docs/envoy/latest/api-v1/cluster_manager/sds), | ||
| the new name better describes what the API does in practice. Advanced global load balancing capable of utilizing N-dimensional upstream metrics is now supported. | ||
| * [Health Discovery Service (HDS)](envoy/service/discovery/v2/hds.proto). This new API supports efficient endpoint health discovery by the management server via the Envoy instances it manages. Individual Envoy instances | ||
| will typically receive HDS instructions to health check a subset of all | ||
| endpoints. The health check subset may not be a subset of the Envoy instance's | ||
| EDS endpoints. | ||
| * [Listener Discovery Service (LDS)](envoy/api/v2/lds.proto). This new API supports dynamic discovery of the listener configuration (which ports to bind to, TLS details, filter chains, etc.). | ||
| * [Metric Service (MS)](envoy/service/metrics/v2/metrics_service.proto). This new API allows Envoy to push (stream) metrics forever for servers to consume. | ||
| * [Rate Limit Service (RLS)](envoy/service/ratelimit/v2/rls.proto) | ||
| * [Route Discovery Service (RDS)](envoy/api/v2/rds.proto). | ||
| * [Secret Discovery Service (SDS)](envoy/service/discovery/v2/sds.proto). | ||
|
|
||
| In addition to the above APIs, an aggregation API will be provided to allow for | ||
| fine grained control over the sequencing of API updates across discovery | ||
| services: | ||
|
|
||
| * [Aggregated Discovery Service (ADS)](envoy/api/v2/discovery.proto). See | ||
| the [ADS overview](https://www.envoyproxy.io/docs/envoy/latest/configuration/overview/v2_overview#aggregated-discovery-service). | ||
|
|
||
| A protocol description for the xDS APIs is provided [here](XDS_PROTOCOL.md). | ||
|
|
||
| ## Terminology | ||
|
|
||
| Some relevant [existing terminology](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/terminology.html) is | ||
| repeated below and some new v2 terms introduced. | ||
|
|
||
| * Cluster: A cluster is a group of logically similar endpoints that Envoy | ||
| connects to. In v2, RDS routes points to clusters, CDS provides cluster configuration and | ||
| Envoy discovers the cluster members via EDS. | ||
|
|
||
| * Downstream: A downstream host connects to Envoy, sends requests, and receives responses. | ||
|
|
||
| * Endpoint: An endpoint is an upstream host that is a member of one or more clusters. Endpoints are discovered via EDS. | ||
|
|
||
| * Listener: A listener is a named network location (e.g., port, unix domain socket, etc.) that can be connected to by downstream clients. Envoy exposes one or more listeners that downstream hosts connect to. | ||
|
|
||
| * Locality: A location where an Envoy instance or an endpoint runs. This includes | ||
| region, zone and sub-zone identification. | ||
|
|
||
| * Management server: A logical server implementing the v2 Envoy APIs. This is not necessarily a single physical machine since it may be replicated/sharded and API serving for different xDS APIs may be implemented on different physical machines. | ||
|
|
||
| * Region: Geographic region where a zone is located. | ||
|
|
||
| * Sub-zone: Location within a zone where an Envoy instance or an endpoint runs. | ||
| This allows for multiple load balancing targets within a zone. | ||
|
|
||
| * Upstream: An upstream host receives connections and requests from Envoy and returns responses. | ||
|
|
||
| * xDS: CDS/EDS/HDS/LDS/RLS/RDS/SDS APIs. | ||
|
|
||
| * Zone: Availability Zone (AZ) in AWS, Zone in GCP. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| # Contributing guide | ||
|
|
||
| ## API changes | ||
|
|
||
| All API changes should follow the [style guide](STYLE.md). | ||
|
|
||
| API changes are regular PRs in https://github.com/envoyproxy/envoy for the API/configuration | ||
| changes. They may be as part of a larger implementation PR. Please follow the standard Bazel and CI | ||
| process for validating build/test sanity of `api/` before submitting a PR. | ||
|
|
||
| *Note: New .proto files should be also included to [build.sh](https://github.com/envoyproxy/envoy/blob/master/docs/build.sh) and | ||
| [BUILD](https://github.com/envoyproxy/envoy/blob/master/api/docs/BUILD) in order to get the RSTs generated.* | ||
|
|
||
| ## Documentation changes | ||
|
|
||
| The Envoy project takes documentation seriously. We view it as one of the reasons the project has | ||
| seen rapid adoption. As such, it is required that all features have complete documentation. This is | ||
| generally going to be a combination of API documentation as well as architecture/overview | ||
| documentation. | ||
|
|
||
| ### Building documentation locally | ||
|
|
||
| The documentation can be built locally in the root of https://github.com/envoyproxy/envoy via: | ||
|
|
||
| ``` | ||
| docs/build.sh | ||
| ``` | ||
|
|
||
| Or to use a hermetic docker container: | ||
|
|
||
| ``` | ||
| ./ci/run_envoy_docker.sh './ci/do_ci.sh docs' | ||
| ``` | ||
|
|
||
| This process builds RST documentation directly from the proto files, merges it with the static RST | ||
| files, and then runs [Sphinx](http://www.sphinx-doc.org/en/stable/rest.html) over the entire tree to | ||
| produce the final documentation. The generated RST files are not committed as they are regenerated | ||
| every time the documentation is built. | ||
|
|
||
| ### Viewing documentation | ||
|
|
||
| Once the documentation is built, it is available rooted at `generated/docs/index.html`. The | ||
| generated RST files are also viewable in `generated/rst`. | ||
|
|
||
| Note also that the generated documentation can be viewed in CI: | ||
|
|
||
| 1. Open docs job in CircleCI. | ||
| 2. Navigate to "artifacts" tab. | ||
| 3. Expand files and click on `index.html`. | ||
|
|
||
| If you do not see an artifacts tab this is a bug in CircleCI. Try logging out and logging back in. | ||
|
|
||
| ### Documentation guidelines | ||
|
|
||
| The following are some general guidelines around documentation. | ||
|
|
||
| * Cross link as much as possible. Sphinx is fantastic at this. Use it! See ample examples with the | ||
| existing documentation as a guide. | ||
| * Please use a **single space** after a period in documentation so that all generated text is | ||
| consistent. | ||
| * Comments can be left inside comments if needed (that's pretty deep, right?) via the `[#comment:]` | ||
| special tag. E.g., | ||
|
|
||
| ``` | ||
| // This is a really cool field! | ||
| // [#comment:TODO(mattklein123): Do something cooler] | ||
| string foo_field = 3; | ||
| ``` | ||
|
|
||
| * Prefer *italics* for emphasis as `backtick` emphasis is somewhat jarring in our Sphinx theme. | ||
| * All documentation is expected to use proper English grammar with proper punctuation. If you are | ||
| not a fluent English speaker please let us know and we will help out. | ||
| * Tag messages/enum/files with `[#proto-status: draft|experimental|frozen]` to | ||
| reflect their [API | ||
| status](https://www.envoyproxy.io/docs/envoy/latest/configuration/overview/v2_overview#status). | ||
| Frozen entities do not need to be tagged except when overriding an outer scope | ||
| draft or experimental status. |
Oops, something went wrong.