Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(lifecycle): document waiting for sidecar to be ready #1449

1 change: 1 addition & 0 deletions app/_redirects
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
/transparent-proxying-reachable-services/ /docs/LATEST_RELEASE/reference/kubernetes-annotations/#kumaiotransparent-proxying-reachable-services 302
/virtual-probes-port/ /docs/LATEST_RELEASE/reference/kubernetes-annotations/#kumaiovirtual-probes-port 302
/virtual-probes/ /docs/LATEST_RELEASE/reference/kubernetes-annotations/#kumaiovirtual-probes 302
/wait-for-dataplane-ready/ /docs/LATEST_RELEASE/reference/kubernetes-annotations/#kumaiowait-for-dataplane-ready 302

# kuma.io subdomain redirects
https://prometheus.metrics.kuma.io/port /docs/LATEST_RELEASE/reference/kubernetes-annotations/#prometheus-metrics-kuma-io-port/ 302
Expand Down
26 changes: 26 additions & 0 deletions app/_src/production/dp-config/dpp-on-kubernetes.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,32 @@ To join the mesh in a graceful way, we need to first make sure the application i

When `Pod` is converted to a `Dataplane` object it will be marked as unhealthy until Kubernetes considers all containers to be ready.

{% if_version gte:2.4.x %}
### Waiting for the dataplane to be ready

By default, containers start in any order, so an app container can start even though a dataplane container might not be ready to receive traffic.

Making initial requests, such as connecting to a database, can fail for a brief period after the pod starts.

To mitigate this problem try setting
* `runtime.kubernetes.injector.sidecarContainer.waitForDataplaneReady` to `true`, or
* [kuma.io/wait-for-dataplane-ready](/docs/{{ page.version }}/reference/kubernetes-annotations/#kumaiowait-for-dataplane-ready) annotation to `true`
so that the app container waits for the dataplane container to be ready to serve traffic.

{% warning %}

The `waitForDataplaneReady` setting relies on the fact that defining a `postStart` hook causes Kubernetes to run containers sequentially based on their order of occurrence in the `containers` list.
This isn't documented and could change in the future.
It also depends on injecting the kuma-sidecar container as the first container in the pod, which isn't guaranteed since other mutating webhooks can rearrange the containers.

<!-- vale off -->
A better solution will be available when [sidecar containers](https://kubernetes.io/blog/2023/08/25/native-sidecar-containers/) are more stable and widely available.
slonka marked this conversation as resolved.
Show resolved Hide resolved
slonka marked this conversation as resolved.
Show resolved Hide resolved
<!-- vale on -->
{% endwarning %}

{% endif_version %}


### Leaving the mesh

To leave the mesh in a graceful shutdown, we need to remove the traffic destination from all the clients before shutting it down.
Expand Down
8 changes: 8 additions & 0 deletions app/_src/reference/kubernetes-annotations.md
Original file line number Diff line number Diff line change
Expand Up @@ -605,6 +605,14 @@ spec:
[...]
```

{% if_version gte:2.4.x %}
### `kuma.io/wait-for-dataplane-ready`

Define if you want the kuma-sidecar container to wait for the dataplane to be ready before starting app container.
Read relevant [Data plane on Kubernetes](/docs/{{ page.version }}/production/dp-config/dpp-on-kubernetes/#waiting-for-the-dataplane-to-be-ready) section for more information.

{% endif_version %}

### `prometheus.metrics.kuma.io/aggregate-<name>-enabled`

Define if `kuma-dp` should scrape metrics from the application that has been defined in the `Mesh` configuration. Default value: `true`. For more details see the [applications metrics docs](/docs/{{ page.version }}/policies/traffic-metrics#expose-metrics-from-applications)
Expand Down
Loading