Hyperlight on Kubernetes

Run Hyperlight sandboxes in Kubernetes without privileged containers.

Quick Start

Target	Commands
Local (KIND)	just local-up && just plugin-build && just plugin-local-push && just plugin-local-deploy
Azure (AKS)	just azure-up && just get-aks-credentials && just plugin-build && just plugin-acr-push && just plugin-azure-deploy

# Check status
just status

# View logs
just logs

Verify Device Injection

Deploy a test pod to verify the hypervisor device is properly injected:

# Deploy test pod (after device plugin is running)
kubectl apply -f deploy/manifests/examples/test-pod-kvm.yaml

# Check it's running
kubectl get pod hyperlight-test-kvm

# View logs - should show /dev/kvm exists
kubectl logs hyperlight-test-kvm

# Cleanup
kubectl delete pod hyperlight-test-kvm

Expected output:

=== Hyperlight KVM Test Pod ===
Checking for /dev/kvm...
✓ /dev/kvm exists
crw-rw---- 1 nobody nobody 10, 232 Jan  6 12:00 /dev/kvm

Environment variables:
HYPERLIGHT_HYPERVISOR=kvm
HYPERLIGHT_DEVICE_PATH=/dev/kvm

=== Test Complete ===

How It Works

A Kubernetes Device Plugin exposes hypervisor devices (/dev/kvm or /dev/mshv) to pods using the Container Device Interface (CDI). Pods request hyperlight.dev/hypervisor and get the device injected securely.

flowchart LR
    subgraph Node["Kubernetes Node"]
        DP["Device Plugin<br/>(DaemonSet)"]
        K[Kubelet]
        C[containerd]
        P["Pod"]
        CDI["/var/run/cdi/<br/>hyperlight.json"]
        
        DP -->|"1. Registers<br/>hyperlight.dev/hypervisor"| K
        DP -->|"2. Writes CDI spec"| CDI
        K -->|"3. Schedules pod"| C
        C -->|"4. Reads CDI spec"| CDI
        C -->|"5. Injects /dev/kvm or /dev/mshv device"| P
    end

Loading

Using Hyperlight in Your Pods

Request the hyperlight.dev/hypervisor resource and apply security best practices:

apiVersion: v1
kind: Pod
metadata:
  name: my-hyperlight-app
spec:
  nodeSelector:
    hyperlight.dev/hypervisor: kvm  # or mshv
  automountServiceAccountToken: false
  securityContext:
    runAsNonRoot: true
    runAsUser: 65534
    seccompProfile:
      type: RuntimeDefault
  containers:
    - name: app
      image: your-hyperlight-app:latest
      resources:
        limits:
          hyperlight.dev/hypervisor: "1"
      securityContext:
        allowPrivilegeEscalation: false
        readOnlyRootFilesystem: true
        capabilities:
          drop: ["ALL"]

See deploy/manifests/examples/ for complete examples.

Example Hyperlight Application

The hyperlight-app/ directory contains an example demonstrating best practices:

Feature	Implementation
Minimal Image	scratch base - just the binaries (~2.7MB)
Static Binary	musl libc, no runtime dependencies
Non-root	Runs as UID 65534 (nobody), no privilege escalation
Read-only FS	readOnlyRootFilesystem: true
Seccomp	RuntimeDefault profile
No Capabilities	All capabilities dropped
No K8s API	automountServiceAccountToken: false
No Host Access	hostNetwork/hostPID/hostIPC disabled
Masked /proc	procMount: Default

# Build and deploy to AKS
just app-build && just app-acr-push && just app-azure-deploy

# View logs
kubectl logs -l app=hyperlight-hello -f

Documentation

Guide	Description
Command Reference	All just commands explained
Local Development	Test with KIND + local registry
Azure Deployment	Production on AKS + ACR
GHCR Publishing	Publish images to GitHub
Architecture	How the device plugin works

Project Structure

.
├── device-plugin/           # Device plugin source code
│   ├── main.go              # Plugin implementation
│   └── Dockerfile
├── hyperlight-app/          # Example Hyperlight application
│   ├── host/                # Host binary (runs in container)
│   ├── guest/               # Guest binary (runs in VM)
│   ├── k8s/                 # App deployment manifests
│   └── Dockerfile           # Multi-stage build (scratch)
├── deploy/
│   ├── manifests/           # Production Kubernetes manifests
│   │   ├── device-plugin.yaml
│   │   └── examples/        # Test pods and deployments
│   ├── local/               # KIND-specific manifests and setup
│   │   ├── setup.sh
│   │   ├── teardown.sh
│   │   └── device-plugin.yaml
│   └── azure/               # Azure deployment
│       ├── setup.sh
│       ├── teardown.sh
│       └── config.env
├── docs/                    # Documentation
├── scripts/                 # Build and test scripts
└── justfile                 # Build/deploy commands

Available Commands

just --list                  # Show all commands

# Device Plugin
just plugin-build            # Build binary + image
just plugin-local-push       # Push to local registry
just plugin-local-deploy     # Deploy to KIND
just plugin-acr-push         # Push to Azure Container Registry
just plugin-azure-deploy     # Deploy to AKS
just plugin-ghcr-push        # Push to ghcr.io/hyperlight-dev

# Example Hyperlight App
just app-build               # Build app (scratch image)
just app-local-deploy        # Deploy to KIND
just app-azure-deploy        # Deploy to AKS

# Cluster Management
just local-up                # Create KIND cluster + registry
just local-down              # Tear down KIND
just azure-up                # Create Azure infrastructure
just azure-stop              # Stop AKS cluster 
just azure-start             # Start AKS cluster
just azure-down              # Delete all Azure resources

# Utilities
just status                  # Show device plugin status
just logs                    # View device plugin logs
just check                   # Verify prerequisites

Requirements

Platform

OS	Support	Notes
Linux	✅ Full	Native support
Windows	✅ Via WSL2	Run everything inside WSL2 (Ubuntu recommended)

Windows users: Install WSL2 first, then work entirely inside WSL2. Docker Desktop should be configured to use WSL2 backend.

# PowerShell (as admin)
wsl --install -d Ubuntu

Then open Ubuntu and continue from there.

All targets

Tool	Purpose	Install
just	Command runner	cargo install just or binaries
Docker	Build images	See Docker docs (use WSL2 backend on Windows)
kubectl	Kubernetes CLI	curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
Go 1.21+	Device plugin	apt install golang or download
envsubst	Template substitution	apt install gettext-base (usually pre-installed)

Local development (KIND)

Tool	Purpose	Install
KIND 0.20+	Local K8s	go install sigs.k8s.io/kind@latest
/dev/kvm	Hypervisor	See below

Minimum versions: Kubernetes 1.26+, KIND 0.20+. Older versions may have container runtime issues.

Azure deployment

Tool	Purpose	Install
Azure CLI	Azure management	curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash

Building Hyperlight apps (optional)

Tool	Purpose	Install
Rust	Hyperlight apps	curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
cargo-hyperlight	Guest binaries	cargo install --locked cargo-hyperlight

Note: Rust is only needed if you're building Hyperlight applications locally. The example app builds inside Docker, so Rust isn't required on your host machine.