Skip to content

Commit

Permalink
docs: document /dev/net/tun compatibility
Browse files Browse the repository at this point in the history
Fixes #9309

Co-authored-by: Jean-François Roy <jf@devklog.net>
Signed-off-by: Andrey Smirnov <andrey.smirnov@siderolabs.com>
  • Loading branch information
smira and jfroy committed Sep 17, 2024
1 parent 18daedb commit 7bd26df
Show file tree
Hide file tree
Showing 3 changed files with 158 additions and 0 deletions.
8 changes: 8 additions & 0 deletions hack/release.toml
Original file line number Diff line number Diff line change
Expand Up @@ -248,6 +248,14 @@ Talos Linux supports supplying zstd-compressed, base64-encoded machine configura
description = """\
Talos Linux now supports removing parts of the configuration using the `$patch: delete` syntax similar to the kubernetes.
More information can be found [here](https://www.talos.dev/v1.8/talos-guides/configuration/patching/#strategic-merge-patches).
"""

[notes.dev-tun]
title = "Accessing `/dev/net/tun` in Kubernetes Pods"
description = """\
Talos Linux ships with `runc` 1.2, which [drops](https://github.com/opencontainers/runc/pull/3468) legacy rule to expose `/dev/net/tun` devices by default in the container.
If you need to access `/dev/net/tun` in your Kubernetes pods (e.g. running Tailscale as a Kubernetes pod), you can add use [device plugins](https:/www.talos.dev/v1.8/kubernetes-guides/configuration/device-plugins/) to expose `/dev/net/tun` to the pod.
"""

[make_deps]
Expand Down
5 changes: 5 additions & 0 deletions website/content/v1.8/introduction/what-is-new/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,11 @@ If you are running Talos virtualized in QEMU (e.g., Proxmox), you can add this a
You can refer to the [Image Factory or Imager documentation]({{< relref "../../talos-guides/install/boot-assets" >}}) for instructions on how to do this.
This change addresses issues such as slow boot or lack of console output on bare metal hardware without a serial console.

### Accessing `/dev/net/tun` in Kubernetes Pods

Talos Linux includes `runc` 1.2, which [no longer](https://github.com/opencontainers/runc/pull/3468) exposes `/dev/net/tun` devices by default in containers.
If you require access to `/dev/net/tun` in your Kubernetes pods (such as when running Tailscale as a pod), you can use [device plugins]({{< relref "../../kubernetes-guides/configuration/device-plugins" >}}) to expose `/dev/net/tun` to the pod.

## Disk Management

The disk management backend has been rewritten to support more complex configurations, but the existing configuration should continue to work as before.
Expand Down
145 changes: 145 additions & 0 deletions website/content/v1.8/kubernetes-guides/configuration/device-plugins.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,145 @@
---
title: "Device Plugins"
description: "In this guide you will learn how to expose host devices to the Kubernetes pods."
---

[Kubernetes Device Plugins](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/) can be used to expose host devices to the Kubernetes pods.
This guide will show you how to deploy a device plugin to your Talos cluster.
In this guide, we will use [Kubernetes Generic Device Plugin](https://github.com/squat/generic-device-plugin), but there are other implementations available.

## Deploying the Device Plugin

The Kubernetes Generic Device Plugin is a DaemonSet that runs on each node in the cluster, exposing the devices to the pods.
The device plugin is configured with a [list of devices to expose](https://github.com/squat/generic-device-plugin#overview), e.g.
`--device='{"name": "video", "groups": [{"paths": [{"path": "/dev/video0"}]}]}`.

In this guide, we will demonstrate how to deploy the device plugin with a configuration that exposes the `/dev/net/tun` device.
This device is commonly used for user-space Wireguard, including Tailscale.

```yaml
# generic-device-plugin.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: generic-device-plugin
namespace: kube-system
labels:
app.kubernetes.io/name: generic-device-plugin
spec:
selector:
matchLabels:
app.kubernetes.io/name: generic-device-plugin
template:
metadata:
labels:
app.kubernetes.io/name: generic-device-plugin
spec:
priorityClassName: system-node-critical
tolerations:
- operator: "Exists"
effect: "NoExecute"
- operator: "Exists"
effect: "NoSchedule"
containers:
- image: squat/generic-device-plugin
args:
- --device
- |
name: tun
groups:
- count: 1000
paths:
- path: /dev/net/tun
name: generic-device-plugin
resources:
requests:
cpu: 50m
memory: 10Mi
limits:
cpu: 50m
memory: 20Mi
ports:
- containerPort: 8080
name: http
securityContext:
privileged: true
volumeMounts:
- name: device-plugin
mountPath: /var/lib/kubelet/device-plugins
- name: dev
mountPath: /dev
volumes:
- name: device-plugin
hostPath:
path: /var/lib/kubelet/device-plugins
- name: dev
hostPath:
path: /dev
updateStrategy:
type: RollingUpdate
```
Apply the manifest to your cluster:
```sh
kubectl apply -f generic-device-plugin.yaml
```

Once the device plugin is deployed, you can verify that the nodes have a new resource: `squat.ai/tun` (the `tun` name comes from the name of the group in the device plugin configuration).:

```sh
$ kubectl describe node worker-1
...
Allocated resources:
Resource Requests Limits
-------- -------- ------
...
squat.ai/tun 0 0
```

## Deploying a Pod with the Device

Now that the device plugin is deployed, you can deploy a pod that requests the device.
The request for the device is specified as a [resource](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) in the pod spec.

```yaml
requests:
limits:
squat.ai/tun: "1"
```
Here is an example non-privileged pod spec that requests the `/dev/net/tun` device:

```yaml
# tun-pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: tun-test
spec:
containers:
- image: alpine
name: test
command:
- sleep
- inf
resources:
limits:
squat.ai/tun: "1"
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
add:
- NET_ADMIN
dnsPolicy: ClusterFirst
restartPolicy: Always
```

When running the pod, you should see the `/dev/net/tun` device available:

```sh
$ ls -l /dev/net/tun
crw-rw-rw- 1 root root 10, 200 Sep 17 10:30 /dev/net/tun
```

0 comments on commit 7bd26df

Please sign in to comment.