router - single port/ingress; alt server certs

charts/ziti-router/Chart.yaml

@@ -1,6 +1,6 @@
-apiVersion: v3
-appVersion: 1.1.9
+apiVersion: v2
+appVersion: 1.1.15
 description: Host an OpenZiti router in Kubernetes
 name: ziti-router
 type: application
-version: 1.0.10
+version: 1.0.11

charts/ziti-router/README.md.gotmpl

@@ -20,38 +20,29 @@
 helm repo add openziti https://docs.openziti.io/helm-charts/
 ```
 
-## Minimal Installation
+## Public Router
 
-After adding the charts repo to Helm, then you may install the chart in the same cluster where the controller is running by using the cluster-internal service of the control plane endpoint. This default values used in this minimal approach is suitable for a Kubernetes distribution like K3S or Minikube that configures pass-through TLS for Service resources of type LoadBalancer. 
+The default configuration listens for incoming edge connections and router links. Set a public address for this listener (`edge.advertisedHost`) or disable it (`linkListeners.transport.enabled`) to avoid routers continually failing to dial into it.
 
 ```bash
 # get a router enrollment token from the controller's management API
-ziti edge create edge-router router1 \
-  --role-attributes default --tunneler-enabled --jwt-output-file /tmp/router1.jwt
+ziti edge create edge-router "router1" \
+  --tunneler-enabled --jwt-output-file /tmp/router1.jwt
 
 # subscribe to the openziti Helm repo
 helm repo add openziti https://openziti.github.io/helm-charts/
 
-# install the router chart
-helm install \
-  --namespace ziti-router --create-namespace --generate-name \
+# install the router chart with a public address
+helm upgrade --install \
+  "ziti-router-123456789" \
   openziti/ziti-router \
     --set-file enrollmentJwt=/tmp/router1.jwt \
-    --set advertisedHost=ziti-router.example.com \
-    --set ctrl.endpoint=ziti-controller-ctrl.ziti-controller.svc:6262 
+    --set ctrl.endpoint=ctrl.ziti.example.com:443 \
+    --set edge.advertisedHost=router1.ziti.example.com \
 ```
+### Ingress TLS Passthrough
 
-You must supply some values when you install the chart:
-
-| Key | Type | Default | Description |
-|-----|------|---------|-------------|
-|enrollmentJwt|string|nil|the router enrollment token from the Ziti management API|
-|advertisedHost|string|nil|the DNS name that edge clients will resolve to reach this router's edge listener|
-|ctrl.endpoint|string|nil|the DNS name:port of the router control plane endpoint provided by the Ziti controller|
-
-## Managed Kubernetes Installation
-
-Managed Kubernetes providers typically configure server TLS for a Service of type LoadBalancer. Ziti needs pass-through TLS because edge clients authenticate to the router with client certificates. We'll accomplish this by changing the Service type to ClusterIP and creating Ingress resources with pass-through TLS for each cluster service.
+All router TLS listeners must terminate TLS, so it's essential that Ingress resources use TLS passthrough.
 
 This example demonstrates creating TLS pass-through Ingress resources for use with [ingress-nginx](https://docs.nginx.com/nginx-ingress-controller/installation/installing-nic/installation-with-helm/).
 
@@ -83,10 +74,10 @@ helm install \
 Create a Helm chart values file for this router chart.
 
 ```yaml
-# /tmp/router-values.yml
+# router-values.yml
 ctrl:
-  endpoint: ziti-controller-ctrl.ziti-controller.svc:6262 
-advertisedHost: ziti-router.example.com
+  endpoint: ziti-controller-ctrl.ziti-controller.svc:1280
+advertisedHost: router1.ziti.example.com
 edge:
   advertisedPort: 443
   service:
@@ -102,51 +93,43 @@ edge:
 Now upgrade your router chart release with the values file.
 
 ```bash
-# will attempt enrollment again if it failed initially
-helm upgrade \
-  --namespace ziti-router ziti-router-123456789 \
+helm upgrade --install \
+  "ziti-router-1" \
   openziti/ziti-router \
     --set-file enrollmentJwt=/tmp/router1.jwt \
     --values /tmp/router-values.yml
 ```
 
-## Router Transport Links
-
-The minimal installation guided you to install a router in the same cluster as the controller, and the managed Kubernetes upgrade guided you to expose the router's edge listener as a pass-through TLS Ingress. Building on those concepts, let's expand your mesh of Ziti routers. For this you will need to configure router link listeners, i.e. router-to-router links. This is accomplished in this chart by setting some additional values.
+## Private Router
 
-Merge the following with your router values.
+Disable the link listener if the router does not have a public address set (`edge.advertisedHost`). Ziti identities inside the cluster can still use the private router's edge listener ClusterIP service by authorizing them with a Ziti edge router policy.
 
-```yaml
-linkListeners:
-  transport:
-    advertisedHost: router1-transport.example.com
-    advertisedPort: 443
-    service:
-      enabled: true
-      type: ClusterIP
-    ingress:
-      enabled: true
-      ingressClassName: nginx
-      annotations:
-        kubernetes.io/ingress.allow-http: "false"
-        nginx.ingress.kubernetes.io/ssl-passthrough: "true"
+```bash
+helm upgrade --install \
+  "ziti-router-1" \
+  openziti/ziti-router \
+    --set-file enrollmentJwt=/tmp/router1.jwt \
+    --set ctrl.endpoint=ctrl.ziti.example.com:443 \
+    --set linkListeners.transport.enabled=false
 ```
 
-Notice that we've chosen a distinct DNS name for this new ingress. This allows us to have any number of 443/tcp virtual servers on the same IP address. You may find it convenient to delegate a DNS zone with a wildcard record resolving to your Nginx LoadBalancer IP.
+## Tunnel Modes
+
+### Host tunnel mode
 
-Now upgrade your router chart release with the merged values file.
+Default: `tunnel.mode=none`
+
+Host mode enables a router's identity to reverse proxy Ziti service traffic to a target address on the regular network. Enable this mode by updating the router's identity in the controller to enable tunneling, then set `tunnel.mode=host` and upgrade the Helm release to start hosting Ziti services.
 
 ```bash
-helm upgrade \
-  --namespace ziti-router ziti-router-123456789 \
-  openziti/ziti-router \
-    --set-file enrollmentJwt=/tmp/router1.jwt \
-    --values /tmp/router-values.yml
+ziti edge update identity "router1" --tunneler-enabled
 ```
 
-## Proxy tunnel mode
+### Proxy tunnel mode
+
+`tunnel.mode=proxy`
 
-The Openziti router supports the [Proxy Tunnel mode](https://docs.openziti.io/docs/reference/configuration/router#tunnel-listeners). If you need to deploy Kubernetes services together with your Ziti Router in order to make these ports available as ClusterIP, NodePort or LoadBalancer services within your cluster, you can let this helm chart deploy those services for you. In some specific cases, it is not enough to have just one Kubernetes service making accessible all ports assigned to specific Openziti services, but to have some of these proxy ports on one, and others on another Kubernetes service (for example, if you want to expose one of the proxy services as a ClusterIP service, but another as a LoadBalancer service).
+Proxy mode enables the router to publish Ziti services as Kubernetes services.
 
 Here's an example router values' snippet to merge with your other values:
 
@@ -172,13 +155,91 @@ tunnel:
         metallb.universe.tf/loadBalancerIPs: 192.168.1.100
 ```
 
-## Values Reference
+## Additional Listeners and Volumes
 
-{{ template "chart.valuesTable" . }}
+You can configure an additional edge listener by setting `edge.additionalListeners`. This is useful for making a WebSocket edge listener available for BrowZer clients that require a trusted server certificate.
 
-## TODO's
+This example configures a wss listener and requests a certificate from cert-manager. The alternative certificate must have a DNS SAN that is distinct from the public address of the default edge listener (`edge.advertisedHost`). This cert-manager approach has the advantage of automatically renewing the certificate and ensuring the DNS SAN of the certificate matches an additional listener's advertised host.
 
-* replicas - does it make sense? afaik every replica needs it's own identity - how does this fit in?
-* lower CA / Cert lifetime; refresh certificates on update
+```yaml
+edge:
+  advertisedHost: router1.ziti.example.com
+  advertisedPort: 443
+  ingress:
+    annotations:
+      kubernetes.io/ingress.allow-http: "false"
+      nginx.ingress.kubernetes.io/ssl-passthrough: "true"
+    enabled: true
+    ingressClassName: nginx
+  service:
+    enabled: true
+    type: ClusterIP
+  additionalListeners:
+    - name: router1-edge-wss
+      protocol: wss
+      containerPort: 3023                           # must be unique
+      advertisedHost: router1-wss.ziti.example.com  # must be distinct from edge.advertisedHost
+      advertisedPort: 443
+      addHostToSan: false                           # must be false to avoid colliding DNS SANs between listeners
+      service:
+        enabled: true
+        type: ClusterIP
+      ingress:
+        enabled: true
+        annotations:
+          kubernetes.io/ingress.allow-http: "false"
+          nginx.ingress.kubernetes.io/ssl-passthrough: "true"
+        ingressClassName: nginx
+
+identity:
+  altServerCerts:
+    - name: alt-server-cert-1
+      mode: certManager
+      secretName: ziti-router1-alt-server-certs1
+      additionalListenerName: router1-edge-wss
+      mountPath: /etc/ziti/alt-server-cert-1
+      issuerRef:
+        group: cert-manager.io
+        kind: ClusterIssuer
+        name: cloudflare-dns01-issuer-staging
+```
+
+You don't have to use cert-manager. If you have a TLS secret named `ziti-router1-alt-server-certs1` from some other issuer in the same namespace as the router containing the certificate and key, you can use it by setting values like these. You must also configure the additional listener as in the prior example with an advertisedHost that matches a DNS SAN of the alternative certificate.
+
+```yaml
+# this is an generic approach for mounting configmaps, secrets, csi volumes, etc.
+additionalVolumes:
+  - name: alt-server-cert-2
+    volumeType: secret
+    mountPath: /etc/ziti/alt-server-cert-2
+    secretName: ziti-router1-alt-server-cert-2
+
+# this looks up a TLS secret's mountpoint to configure the router's identity
+identity:
+  altServerCerts:
+    - mode: secret
+      secretName: ziti-router1-alt-server-cert-2
+```
+
+You may also specify matching file paths for an additional volume and alternative certificate if the volume is not a TLS secret.
+
+```yaml
+additionalVolumes:
+  - name: alt-server-cert-3
+    volumeType: csi
+    driverName: csi.bpfd.dev
+    attributes: volumeAttributes
+    mountPath: /etc/ziti/alt-server-cert-3
+
+identity:
+  altServerCerts:
+    - mode: localFile
+      serverCert: /etc/ziti/alt-server-cert-3/server3.crt
+      serverKey: /etc/ziti/alt-server-cert-3/server3.key
+```
+
+## Values Reference
+
+{{ template "chart.valuesTable" . }}
 
 <!-- README.md generated by helm-docs from README.md.gotmpl -->

charts/ziti-router/templates/_helpers.tpl

@@ -7,20 +7,20 @@ Expand the name of the chart.
 {{- end }}
 
 {{/*
-Create a default fully qualified app name.
-We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
-If release name contains chart name it will be used as a full name.
+Create a default fully qualified app name.  We truncate at 63 chars because some
+Kubernetes name fields are limited to this (by the DNS naming spec).  If release
+name contains chart name it will be used as a full name.
 */}}
 {{- define "ziti-router.fullname" -}}
     {{- if .Values.fullnameOverride }}
         {{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
     {{- else }}
         {{- $name := default ( trimPrefix "ziti-" .Chart.Name ) .Values.nameOverride }}
-    {{- if contains $name .Release.Name }}
-        {{- .Release.Name | trunc 63 | trimSuffix "-" }}
-    {{- else }}
-        {{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
-    {{- end }}
+        {{- if contains $name .Release.Name }}
+            {{- .Release.Name | trunc 63 | trimSuffix "-" }}
+        {{- else }}
+            {{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
+        {{- end }}
     {{- end }}
 {{- end }}
 
@@ -61,3 +61,75 @@ Create the name of the service account to use
 {{- default "default" .Values.serviceAccount.name }}
 {{- end }}
 {{- end }}
+
+{{/*
+help the alt-certificate template find its DNS SAN by looking up the advertised
+host of an additional listener
+*/}}
+{{- define "ziti-router.lookupAltServerCertHost" -}}
+{{- $listenerName := .additionalListenerName -}}
+{{- $additionalListeners:= .additionalListeners -}}
+{{- $matchedListenerHost := "" -}}
+{{- range $additionalListeners }}
+  {{- if eq .name $listenerName }}
+    {{- $matchedListenerHost = .advertisedHost }}
+  {{- end }}
+{{- end }}
+{{- if $matchedListenerHost }}
+  {{- $matchedListenerHost }}
+{{- else }}
+  {{- fail "No matched listener host found" }}
+{{- end }}
+{{- end }}
+
+{{/*
+help the alt-certificate template find the members of identity.altServerCerts
+that are managed by cert-manager
+*/}}
+{{- define "ziti-router.getCertManagerAltServerCerts" -}}
+{{- $filteredCerts := list -}}
+{{- range . -}}
+  {{- if eq .mode "certManager" -}}
+    {{- $filteredCerts = append $filteredCerts . -}}
+  {{- end -}}
+{{- end -}}
+{{- dict "certManagerCerts" $filteredCerts | toJson -}}
+{{- end -}}
+
+{{/*
+help the configmap template find the mount path of an alternative server
+certificate by looking up the secret name in the list of additional volumes
+*/}}
+{{- define "ziti-router.lookupVolumeMountPath" -}}
+{{- $secretName := .secretName -}}
+{{- $matchingVolumeMountPath := "" -}}
+{{- range .additionalVolumes }}
+  {{- if and (eq .volumeType "secret") (eq .secretName $secretName) }}
+    {{- $matchingVolumeMountPath = .mountPath }}
+  {{- end }}
+{{- end }}
+{{- if $matchingVolumeMountPath }}
+  {{- $matchingVolumeMountPath }}
+{{- else }}
+  {{- fail (printf "No matching additionalVolume found for secretName: %s" $secretName) }}
+{{- end }}
+{{- end -}}
+
+{{/*
+render as an inline template if the value is a string containing a go template,
+else return the literal value
+*/}}
+{{- define "ziti-router.tplOrLiteral" -}}
+{{- $value := .value -}}
+{{- $context := .context -}}
+{{- if typeIs "string" $value -}}
+  {{- $trimmed := trim $value -}}
+  {{- if and (hasPrefix "{{" $trimmed) (hasSuffix "}}" $trimmed) -}}
+    {{- tpl $value $context -}}
+  {{- else -}}
+    {{- $value -}}
+  {{- end -}}
+{{- else -}}
+  {{- $value -}}
+{{- end -}}
+{{- end -}}

charts/ziti-router/templates/alt-certificate.yaml

@@ -0,0 +1,27 @@
+# openziti-helm-charts/charts/ziti-router/templates/alt-certificate.yaml
+{{- if gt (len $.Values.identity.altServerCerts) 0 -}}
+{{- $root := . }}
+{{- $values := $.Values }}
+{{- $certs := ((include "ziti-router.getCertManagerAltServerCerts" $values.identity.altServerCerts) | fromJson).certManagerCerts -}}
+
+{{- if gt (len $certs) 0 }}
+  {{- range $index, $cert := $certs }}
+---
+apiVersion: cert-manager.io/v1
+kind: Certificate
+metadata:
+  name: {{ printf "%s-alt-cert-%d" (include "ziti-router.fullname" $root) $index }}
+  namespace: {{ $root.Release.Namespace }}
+spec:
+  secretName: {{ $cert.secretName | quote }}
+  issuerRef:
+    {{- toYaml $cert.issuerRef | nindent 4 }}
+  dnsNames:
+    - {{ (include "ziti-router.lookupAltServerCertHost" (dict "additionalListenerName" $cert.additionalListenerName "additionalListeners" $values.edge.additionalListeners)) | quote }}
+  usage:
+    - digital signature
+    - key encipherment
+    - server auth
+  {{- end }}
+{{- end }}
+{{- end }}