Skip to content

Commit

Permalink
pkgs(kubenix): overhaul and drop support for the helm CLI (#24)
Browse files Browse the repository at this point in the history
This is a relatively large re-design which

 - removes usage of the Helm CLI
 - expects users to override the default package
 - performs an interactive diff, confirm, apply by default
 - prunes removed resources
  • Loading branch information
hall authored Jul 7, 2023
1 parent ccfd0d1 commit 07ba711
Show file tree
Hide file tree
Showing 4 changed files with 156 additions and 130 deletions.
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

## [0.2.0] - 2023-07-07

### Breaking

- removed usage of the `helm` CLI within the `kubenix` CLI

This simplifies design by removing overlapping responsibilities but means extra functionality provided by the `helm` CLI is no longer available; specifically:

- hooks are no longer ordered (but can still be excluded with `noHooks`)
- `helm` subcommands (e.g., `list` or `rollback`) will not be able to operate on resources

### Added

- the CLI now prunes resources and performs an interactive diff by default

## [0.1.0] - 2023-07-06

### Added
Expand Down
61 changes: 43 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Kubernetes management with Nix
<img src="./docs/static/logo.svg" alt="nixos logo in kubernetes blue" width="350"/>
</p>

> **WARN**: this is a work in progress, expect breaking changes
> **WARN**: this is a work in progress, expect breaking [changes](./CHANGELOG.md)
## Usage

Expand All @@ -20,7 +20,7 @@ A minimal example `flake.nix` (build with `nix build`):
in {
packages.${system}.default = (kubenix.evalModules.${system} {
module = { kubenix, ... }: {
imports = with kubenix.modules; [k8s];
imports = [ kubenix.modules.k8s ];
kubernetes.resources.pods.example.spec.containers.nginx.image = "nginx";
};
}).config.kubernetes.result;
Expand All @@ -33,42 +33,67 @@ Or, if you're not using flakes, a `default.nix` file (build with `nix-build`):
```nix
{ kubenix ? import (builtins.fetchGit {
url = "https://github.com/hall/kubenix.git";
rev = "aa734afc9cf7a5146a7a9d93fd534e81572c8122";
rev = "main";
}) }:
(kubenix.evalModules.x86_64-linux {
module = {kubenix, ... }: {
imports = with kubenix.modules; [k8s];
imports = [ kubenix.modules.k8s ];
kubernetes.resources.pods.example.spec.containers.nginx.image = "nginx";
};
}).config.kubernetes.result
```

Either way the JSON manifests will be written to `./result`.

See the [examples](/examples/pod) for more.
See the [examples](https://kubenix.org/examples/pod) for more.

## CLI

> **NOTE**: this is a WIP CLI which currently reads the `kubenix` package on a local flake
While kubenix is compatible with just about any deployment system, there's a simple builtin CLI which can:

Render all resources with
- show a diff, prompt for confirmation, then apply
- prune removed resources
- pipe manifests through [vals](https://github.com/helmfile/vals) for the ability to inject secrets without writing them to the nix store

nix run github:hall/kubenix -- render
To configure this, override the default package, passing the arguments of [evalModules](https://nixos.org/manual/nixpkgs/stable/#module-system-lib-evalModules).

> **HINT**: use ` --help` for more commands
```nix
{
kubenix = inputs.kubenix.packages.${pkgs.system}.default.override {
module = import ./cluster;
# optional; pass custom values to the kubenix module
specialArgs = { flake = self; };
};
}
```

### Support
Then apply the resources with

nix run '.#kubenix'

which will print a diff and prompt for confirmation:

```diff
diff -N -u -I ' kubenix/hash: ' -I ' generation: ' /tmp/LIVE-2503962153/apps.v1.Deployment.default.home-assistant /tmp/MERGED-231044561/apps.v1.Deployment.default.home-assistant
--- /tmp/LIVE-2503962153/apps.v1.Deployment.default.home-assistant 2023-07-06 23:33:29.841771295 -0400
+++ /tmp/MERGED-231044561/apps.v1.Deployment.default.home-assistant 2023-07-06 23:33:29.842771296 -0400
@@ -43,7 +43,7 @@
spec:
automountServiceAccountToken: true
containers:
- - image: homeassistant/home-assistant:2023.5
+ - image: homeassistant/home-assistant:2023.6
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
apply? [y/N]:
```

The following table gives a general overview of currently supported/planned functionality.
> **HINT**: use ` --help` for more commands
| | kubectl | helm |
| --------- | :-----: | :---: |
| render | x | x[^2] |
| diff | x | x |
| apply[^1] | x | x |
Optionally, write the resources to `./result/manifests.json`:

[^1]: currently create-only
[^2]: piping rendered helm charts to kubectl is a lossy process (e.g., [hooks](https://helm.sh/docs/topics/charts_hooks/) will not work)
nix build '.#kubenix'

## Attribution

Expand Down
4 changes: 2 additions & 2 deletions flake.nix
Original file line number Diff line number Diff line change
Expand Up @@ -144,10 +144,10 @@
inherit (pkgs) kubernetes kubectl;
}
// {
cli = pkgs.callPackage ./pkgs/kubenix.nix {
default = pkgs.callPackage ./pkgs/kubenix.nix {
inherit (self.packages.${system});
evalModules = self.evalModules.${system};
};
default = self.packages.${system}.cli;
docs = import ./docs {
inherit pkgs;
options =
Expand Down
206 changes: 96 additions & 110 deletions pkgs/kubenix.nix
Original file line number Diff line number Diff line change
@@ -1,113 +1,99 @@
{
jq,
kubectl,
kubernetes-helm,
nix,
vals,
writeShellScriptBin,
}:
writeShellScriptBin "kubenix" ''
set -Eeuo pipefail
function _help() {
echo "
kubenix - Kubernetes management with Nix
commands:
apply - create resources in target cluster
diff - show a diff between configured and live resources
render - print resource manifests to stdout
options:
-h --help - show this menu
-v --verbose - increase output details
"
}
# path to nix binary (useful to inject flags, e.g.)
_nix="${nix}/bin/nix"
SYSTEM=$($_nix show-config --json | jq -r '.system.value')
function _helm() {
$_nix eval ".#kubenix.$SYSTEM.config.kubernetes.helm" --json | jq -c '.releases[] | del(.objects)' | while read -r release; do
values=$(mktemp)
echo "$release" | jq -r '.values' | ${vals}/bin/vals eval > $values
name=$(echo "$release" | jq -r '.name')
chart=$(echo "$release" | jq -r '.chart')
namespace=$(echo "$release" | jq -r '.namespace // "default"')
args="-n $namespace $name $chart -f $values"
# only apply when there are changes
if [[ "$1" == "upgrade" ]]; then
if ${kubernetes-helm}/bin/helm diff upgrade $args --allow-unreleased --detailed-exitcode 2> /dev/null; then
continue
fi
fi
${kubernetes-helm}/bin/helm $@ $args
done
}
function _kubectl() {
MANIFESTS=$(mktemp)
# TODO: find a better filter, not just not-helm, not-crd
resources=$($_nix build ".#kubenix.$SYSTEM.config.kubernetes.result" --json | jq -r '.[0].outputs.out')
cat $resources | jq '.items[]
| select(.metadata.labels."app.kubernetes.io/managed-by" != "Helm")
| select(.kind != "CustomResourceDefinition")' > $MANIFESTS
[ -s "$MANIFESTS" ] || return 0
case $1 in
render)
cat $MANIFESTS;;
*)
cat $MANIFESTS | ${vals}/bin/vals eval | ${kubectl}/bin/kubectl $@ -f - || true;;
esac
}
# if no args given, add empty string
[ $# -eq 0 ] && set -- ""
# use kubeconfig, if given
kubeconfig=$($_nix eval ".#kubenix.$SYSTEM.config.kubernetes.kubeconfig" --raw)
[ -n "$kubeconfig" ] && export KUBECONFIG=$kubeconfig
# parse arguments
while test $# -gt 0; do
case "$1" in
apply)
_kubectl apply
_helm upgrade --atomic --install --create-namespace
shift;;
diff)
_kubectl diff
_helm diff upgrade --allow-unreleased
shift;;
render)
_kubectl render
_helm template
shift;;
-h|--help|"")
_help
exit 0;;
-v|--verbose)
_nix="$_nix --show-trace"
set -x
shift;;
*)
_help
exit 1;;
esac
done
''
colordiff,
evalModules,
runCommand,
writeShellScript,
module ? {},
specialArgs ? {},
}: let
kubernetes =
(evalModules {
inherit module specialArgs;
})
.config
.kubernetes
or {};
in
runCommand "kubenix"
{
kubeconfig = kubernetes.kubeconfig or "";
result = kubernetes.result or "";

# kubectl does some parsing which removes the -I flag so
# as workaround, we write to a script and call that
# https://github.com/kubernetes/kubernetes/pull/108199#issuecomment-1058405404
diff = writeShellScript "kubenix-diff" ''
${colordiff}/bin/colordiff --nobanner -N -u -I ' kubenix/hash: ' -I ' generation: ' $@
'';
} ''
set -euo pipefail
mkdir -p $out/bin
# write the manifests for use with `nix build`
ln -s $result $out/manifest.json
# create a script for `nix run`
cat <<EOF> $out/bin/kubenix
set -uo pipefail
export KUBECONFIG=$kubeconfig
export KUBECTL_EXTERNAL_DIFF=$diff
function _help() {
echo "
kubenix - Kubernetes management with Nix
commands:
"" - run diff, prompt for confirmation, then apply
apply - create resources in target cluster
diff - show a diff between configured and live resources
render - print resource manifests to stdout
options:
-h --help - show this menu
"
}
function _kubectl() {
${vals}/bin/vals eval -fail-on-missing-key-in-map < $result | ${kubectl}/bin/kubectl \$@
}
# if no args given, add empty string
[ \$# -eq 0 ] && set -- ""
# parse arguments
while test \$# -gt 0; do
case "\$1" in
-h|--help)
_help
exit 0;;
"")
_kubectl diff -f - --prune
if [[ "\$?" -eq 1 ]]; then
read -p 'apply? [y/N]: ' response
[[ \$response == "y" ]] && _kubectl apply -f - --prune --all
fi
shift;;
render)
${vals}/bin/vals eval < $result
shift;;
apply|diff)
_kubectl \$@ -f - --prune
shift;;
*)
_kubectl \$@
shift;;
esac
done
EOF
chmod +x $out/bin/kubenix
''

0 comments on commit 07ba711

Please sign in to comment.