Revamp TalosOS Provisioning & Bootstrapping. (#20)

* Change from individual paches to Jinjia Template. * Migrate to dagger functions. * Daggerize most of k8s initalization. * Update docs for daggerized Talos. * Revamped Talos Bootstraping & Provisioning.
KGB33 · Mar 24, 2024 · 1aae70d · 1aae70d
1 parent ef80306
commit 1aae70d
Show file tree

Hide file tree

Showing 24 changed files with 405 additions and 207 deletions.
diff --git a/ansible/playbooks/k8s/argo.yaml b/ansible/playbooks/k8s/argo.yaml
diff --git a/ansible/roles/ArgoCD/tasks/main.yaml b/ansible/roles/ArgoCD/tasks/main.yaml
diff --git a/docs/src/SUMMARY.md b/docs/src/SUMMARY.md
@@ -8,10 +8,10 @@
 - [Proxmox]()
   - [Proxmox Install](pve/install.md)
   - [TLS Certificates](pve/tls_certs.md)
+  - [Talos VMs](pve/talos_vms.md)
 
 - [Kubernetes](k8s/README.md)
-  - [Starting from Scratch]()
-      - [Proxmox VMs](k8s/tofu_vms.md)
+  - [Starting from Scratch](k8s/from_scratch.md)
       - [Talos](k8s/talos.md)
       - [Cilium](k8s/cilium.md)
       - [ArgoCD](k8s/argocd.md)

diff --git a/docs/src/k8s/argocd.md b/docs/src/k8s/argocd.md
@@ -1,12 +1,6 @@
 # ArgoCD
 
-`cd` into the Ansible folder and Install ArgoCD:
-
-```bash
-ansible-playbook playbooks/k8s/argo.yaml
-```
-
-# Argo Login
+## Argo Login
 
 Grab the initial secret:
 
@@ -35,8 +29,7 @@ Once the password has been changed, delete the initial secret:
 kubectl delete secret -n argocd argocd-initial-admin-secret
 ```
 
-
-# Apps-of-Apps
+## Apps-of-Apps
 
 Apply the meta definition:
 

diff --git a/docs/src/k8s/from_scratch.md b/docs/src/k8s/from_scratch.md
@@ -0,0 +1,66 @@
+# Starting from Scratch
+
+First, make sure to create the Talos VMs as described [here](/pve/talos_vms.html),
+then, `cd` into the `talos` directory.
+
+From here, you can use Dagger to automatically provision the nodes. Each step
+is also detailed in the sub-chapters - if you would prefer a manual approach.
+
+```bash
+$ dagger functions
+Name        Description
+argocd      Step 4: Start ArgoCD.
+base-img    Builds a Alpine image with talosctl installed and ready to go.
+bootstrap   Step 2: Bootstrap etcd.
+cilium      Step 3: Apply Cilium.
+provision   Step 1: Provision the nodes.
+```
+
+# Step 1: Provision the Nodes
+
+After the brand new Talos VMs load up - and the `STAGE` is `Maintance` - run:
+
+```bash
+dagger call \
+  --raw-template-file=./templates/talos.yaml.j2 \
+  --talos-dir=_out \
+  provision
+```
+
+# Step 2: Bootstrap Etcd
+
+After all the nodes have rebooted (~1min), bootstrap Etcd. The `STAGE` on
+`teemo` will change from `Installing` to `Booting` when its ready to be
+bootstraped. 
+
+```bash
+dagger call \
+  --raw-template-file=./templates/talos.yaml.j2 \
+  --talos-dir=_out \
+  bootstrap
+```
+
+# Step 3: Apply Cilium
+
+Once Etcd has started, apply cilium:
+
+```bash
+dagger call \
+  --raw-template-file=./templates/talos.yaml.j2 \
+  --talos-dir=_out \
+  cilium
+```
+
+# Step 4: Start ArgoCD
+
+Once the Cilium step has compleated (it'll show a nice status dashboard), start ArgoCD.
+
+```bash
+dagger call \
+  --raw-template-file=./templates/talos.yaml.j2 \
+  --talos-dir=_out \
+  argocd
+```
+Importantly, this step ends by  printing out the default ArgoCD password. **You
+still need to manually change the password and sync the apps-of-apps; see
+[here](/k8s/argocd.html).**
diff --git a/docs/src/k8s/tofu_vms.md → docs/src/pve/talos_vms.md b/docs/src/k8s/tofu_vms.md → docs/src/pve/talos_vms.md
diff --git a/flake.lock b/flake.lock
diff --git a/flake.nix b/flake.nix
@@ -3,22 +3,24 @@
 
   inputs = {
     nixpkgs = { url = "github:NixOS/nixpkgs/nixos-unstable-small"; };
+    dagger = { url = "github:dagger/nix"; };
     flake-utils = { url = "github:numtide/flake-utils"; };
   };
 
-  outputs = { self, nixpkgs, flake-utils }:
+  outputs = { self, nixpkgs, flake-utils, dagger }:
     flake-utils.lib.eachDefaultSystem (system:
       let
+	dag = dagger.packages.${system};
         pkgs = nixpkgs.legacyPackages.${system};
         pyPkgs = pkgs.python311Packages;
-
       in
       {
         # enable `nix fmt`
         formatter = pkgs.nixpkgs-fmt;
 
         devShell = pkgs.mkShell {
           buildInputs = [
+	    dag.dagger
             pyPkgs.ansible-core
             pyPkgs.ansible
             pyPkgs.kubernetes
@@ -31,6 +33,7 @@
             pkgs.jsonnet
             pkgs.jsonnet-bundler
             pkgs.just
+            pkgs.k9s
             pkgs.kubectl
             pkgs.kubeseal
             pkgs.mdbook

diff --git a/talos/README.md b/talos/README.md
@@ -20,52 +20,16 @@ talosctl gen config \
 talosctl --talosconfig talosconfig config endpoint 10.0.0.116
 talosctl --talosconfig talosconfig config node 10.0.0.116
 ```
-And generate the cilium config:
 
-```bash
- helm template cilium cilium/cilium \
-    --version 1.13.1 --namespace kube-system \
-    --set ipam.mode=kubernetes \
-    --set kubeProxyReplacement=strict \
-    --set k8sServiceHost="10.0.0.116" \
-    --set k8sServicePort="6443" \
-    --set=securityContext.capabilities.ciliumAgent="{CHOWN,KILL,NET_ADMIN,NET_RAW,IPC_LOCK,SYS_ADMIN,SYS_RESOURCE,DAC_OVERRIDE,FOWNER,SETGID,SETUID}" \
-    --set=securityContext.capabilities.cleanCiliumState="{NET_ADMIN,SYS_ADMIN,SYS_RESOURCE}" \
-    --set=cgroup.autoMount.enabled=false \
-    --set=cgroup.hostRoot=/sys/fs/cgroup \
-    --set hubble.listenAddress=":4244" \
-    --set hubble.relay.enabled=true \
-    --set hubble.ui.enabled=true > cilium.yaml
-```
-
-## Start the Control plane node
-
-There is only one control plane node:
-  - `teemo.kgb33.dev`
-  - `10.0.0.116`
-
-However, the commands used will allow more control plane nodes to be
-added in the future.
-
-Run `control.py`, then watch the tty in proxmox and wait for the node to
-come back online before preceding.
+## Daggerized Provsioning
 
-## Start the worker nodes
-
-Just like the control plane nodes, run `./workers.py` and wait for the nodes
-to reboot and come back online in Proxmox.
-
-## Bootstrap etcd
-
-Next, run `talosctl --talosconfig talosconfig bootstrap`
-
-Then grab the kubeconfig:
+Run each of the dagger functions in order, waiting for the nodes to come back online after each one:
 
 ```
-talosctl --talosconfig talosconfig kubeconfig
-cp kubeconfig ~/.kube/config
+$ dagger functions
+argocd      Step 4: Start ArgoCD.
+base-img    Builds a Alpine image with talosctl installed and ready to go.
+bootstrap   Step 2: Bootstrap etcd.
+cilium      Step 3: Apply Cilium.
+provision   Step 1: Provision the nodes.
 ```
-
-## Install Cilium
-
-Lastly, run `kubectl apply -f cilium.yaml && cilium status --wait`
diff --git a/talos/dagger.json b/talos/dagger.json
@@ -0,0 +1,6 @@
+{
+  "name": "talos",
+  "sdk": "python",
+  "source": "dagger",
+  "engineVersion": "v0.10.1"
+}
diff --git a/talos/dagger/.gitattributes b/talos/dagger/.gitattributes
@@ -0,0 +1 @@
+/sdk/** linguist-generated
diff --git a/talos/dagger/.gitignore b/talos/dagger/.gitignore
@@ -0,0 +1 @@
+/sdk
diff --git a/talos/dagger/pyproject.toml b/talos/dagger/pyproject.toml
@@ -0,0 +1,6 @@
+[project]
+name = "main"
+version = "0.0.0"
+dependencies = [
+    "Jinja2",
+]