Skip to content

Latest commit

 

History

History
229 lines (186 loc) · 9.79 KB

README.md

File metadata and controls

229 lines (186 loc) · 9.79 KB
Raw

nix-configs

Repository containing my personal Nix (NixOS, Home Manager etc.) configurations.

Basic structure:

  • modules/ - all modules live here, they MUST be turned off by default (side-effect free imports),
    • modules/default.nix - holds imports to all the modules and basic Nix package manager configuration,
    • modules/profile/machine - large NixOS configuration bundles
    • modules/profile/user - user specific profiles
  • packages/, some personal/in-house/out-of-band tools

Generally I aim to hide as much as possible behind *.enable options.

Overview

This is an incomplete list of incorporated software/systems:

  • disks: ZFS on LUKS through disko
  • desktop: Sway (Wayland), as much as possible through Home Manager
  • shell: Fish
  • users: Home Manager
  • MISSING: development environment

Notes

Custom ISO installer

see https://bmcgee.ie/posts/2022/12/setting-up-my-new-laptop-nix-style/

# building the installer from packages/install-iso
sudo dd if="$(nom build '.#install-iso' --no-link --print-out-paths --print-build-logs)/iso/nixos.iso" of=/dev/disk/by-id/usb-SanDisk_Cruzer_Blade_02000515031521144721-0:0 status=progress
sudo dd if="$(nom build '.#install-iso' --no-link --print-out-paths --print-build-logs)/iso/nixos.iso" of=/dev/disk/by-id/usb-_Patriot_Memory_070133F17AC22052-0:0 status=progress
# boot the machine and ssh into it
ssh -o StrictHostKeyChecking=no kdn@nixos

Golden path for bootstrapping new physical machine

  1. build the install-iso and boot it

  2. prepare a new config at modules/profile/host/${HOSTNAME}/default.nix

    • put kdn.profile.machine.baseline.enable = true
    • look through kdn.hardware.{gpu,cpu}.{intel,amd}
    • set up zramSwap & boot.tmp.tmpfsSize
    • figure out missing boot.initrd.{availableK,k}ernelModules

  3. set up a disk configuration:

    (
  # hashtag comment denotes defaults
  # /* */ are just comments
  let
    cfg = config.kdn.hardware.disks;
    d1 = "<DISK_1_NAME>-${config.networking.hostName}";
    d1Cfg = cfg.luks.volumes."${d1}";
    d2 = "<DISK_2_NAME>-${config.networking.hostName}";
    d2Cfg = cfg.luks.volumes."${d2}";
  in
  {
    kdn.hardware.disks.enable = true;
    #kdn.hardware.disks.zpool-main.name = "<HOSTNAME>-main";
    #kdn.hardware.disks.devices."boot" = { type = "gpt"; content.partitions.ESP = { size = "4096M"; ... }; };
    #kdn.hardware.disks.zpools."${cfg.zpool-main.name}" = { };
    #disko.devices.zpool."${cfg.zpool-main.name}".datasets = {
    # "${hostname}/nix-system/nix-store" = { mountpoint = "/nix/store"; ... };
    # "${hostname}/nix-system/nix-var" = { mountpoint = "/nix/var"; ... };
    #}

    /* point it at the right detached `/boot` disk (USB flash drive) */
    kdn.hardware.disks.devices."boot".path = "/dev/disk/by-id/usb-<CORRECT_IDENTIFIER>";
    #disko.devices.disk.boot = { content.type = "gpt"; content.partitions = { ... }; ... };
    #kdn.hardware.disks.devices."${d1}" = { type = "luks"; path = d1Cfg.targetSpec.path; };
    #disko.devices.disk."${d1}" = { type = "luks"; name = "${d1}-crypted"; ... };
    #kdn.hardware.disks.devices."${d2}" = { type = "luks"; path = d2Cfg.targetSpec.path; };
    #disko.devices.disk."${d1}" = { type = "luks"; name = "${d1}-crypted"; ... };

    kdn.hardware.disks.luks.volumes."${d1}" = {
      #target.deviceKey = d1;
      targetSpec.path = "/dev/disk/by-id/<DISK_PATH>";
      uuid = "<uuidgen result>";
      #keyFile = "/tmp/${d1}.key";
      #header.deviceKey = "boot";
      #header.partitionKey = d1;
      headerSpec.num = 2;
      #zpool.name = cfg.zpool-main.name;
    };

    kdn.hardware.disks.luks.volumes."${d2}" = {
      #target.deviceKey = d2;
      targetSpec.path = "/dev/disk/by-id/<DISK_PATH>";
      uuid = "<uuidgen result>";
      #keyFile = "/tmp/${d1}.key";
      #header.deviceKey = "boot";
      #header.partitionKey = d2;
      headerSpec.num = 3;
      #zpool.name = cfg.zpool-main.name;
    };

    #kdn.hardware.disks.impermanence."sys/config".snapshots = true;
    #kdn.hardware.disks.impermanence."sys/cache".snapshots = false;
    #kdn.hardware.disks.impermanence."sys/data".snapshots = true;
    #kdn.hardware.disks.impermanence."sys/state".snapshots = false;
    #kdn.hardware.disks.impermanence."usr/config".snapshots = true;
    #kdn.hardware.disks.impermanence."usr/cache".snapshots = false;
    #kdn.hardware.disks.impermanence."usr/data".snapshots = true;
    #kdn.hardware.disks.impermanence."usr/state".snapshots = false;
    
    /* just a single impermanence example goes below */
    #kdn.hardware.disks.impermanence."usr/data" = {
    #  #zpool.name = cfg.zpool-main.name;
    #  #zfsPrefix = "${config.networking.hostName}/impermanence";
    #  #zfsPath = "${zfsPrefix}/${"usr/data"}";
    #  imp.directories = [
    #    "/var/lib/libvirt/images"
    #  ];
    #  imp.users.root.directories = [
    #    # ...
    #  ];
    #  imp.users.kdn.directories = [
    #    ".local/share/atuin"
    #    ".local/share/nix"
    #    ".local/share/containers"
    #    "dev"
    #  ];
    #};
    #environment.persistence."/nix/persist/usr/data" = {
    #  enable = true; 
    #  hideMounts = true; 
    #  users.root.home = "/root";
    #} // cfg.impermanence."usr/data".imp;
    #disko.devices.zpool."${cfg.zpool-main.name}".datasets."${cfg.impermanence."usr/data".zfsPath}" = { ... };

    #kdn.hardware.disks.tmpfs.size = "16M";
    #disko.devices.nodev."/" = { fsType = "tmpfs"; mountPoints = ["size=${cfg.tmpfs.size}" ... ]; };
    #disko.devices.nodev."/" = { fsType = "tmpfs"; mountPoints = ["size=${cfg.tmpfs.size}" ... ]; };
  }
)

  4. add all your required environment.persistence entries

  5. set up keyfiles for each disk:

    dd if=/dev/random bs=1 count=2048 of=/dev/stdout | pass insert --force --multiline luks/${DISK_NAME}-${HOST_NAME}/keyfile

  6. boot the install-iso

  7. run nixos-anywhere to deploy

    nixos-anywhere --no-reboot --disk-encryption-keys /tmp/${DISK_NAME}-${HOST_NAME}.key "$(pass show luks/${DISK_NAME}-${HOST_NAME}/keyfile | psub)" --flake '.#${HOST_NAME}' nixos.lan.

  8. set up either of for each disk:

    • (unattended) TPM2 unlock: 
      ssh nixos.lan. sudo systemd-cryptenroll --unlock-key-file=/tmp/${DISK_NAME}-${HOST_NAME}.key --tpm2-device=auto /dev/disk/by-partlabel/${DISK_NAME}-${HOST_NAME}-header
    • (attended) YubiKey FIDO2 (touch required, without PIN) unlock: 
      ssh nixos.lan. sudo systemd-cryptenroll --unlock-key-file=/tmp/${DISK_NAME}-${HOST_NAME}.key --fido2-device=auto --fido2-with-client-pin=false --fido2-with-user-verification=false /dev/disk/by-partlabel/${DISK_NAME}-${HOST_NAME}-header

  9. add SSH key to /.sops.yaml

    • ssh-to-age </etc/ssh/ssh_host_ed25519_key.pub
    • reboot

Building on Hetzner Cloud from NixOS installer image

  1. mount the NixOS installer image
  2. run the build: 
    APPLY=1 HOST="<HOST>" bash <(curl -L 'https://raw.githubusercontent.com/nazarewk-iac/nix-configs/main/hetzner.sh')

Interaction between NixOS and Home Manager

How to find out what uses the specific store path?

Find immediate parents/reverse dependencies: nix-store --query --referrers <paths...>.

Find the root using paths: nix-store --query --roots <paths...>.

Fix nix store errors

Fix errors like /nix/store/*-source not found: sudo nix-store --repair --verify --check-contents.

GitHub rate-limiting unauthenticated requests

see https://discourse.nixos.org/t/flakes-provide-github-api-token-for-rate-limiting/18609

add token to ~/.config/nix/nix.conf:

access-tokens = github.com=github_pat_XXXX

example of standalone Nix module

https://gist.github.com/nazarewk/8988facb6118f73d2db3d28b64463cba

systemd-cryptenroll --fido2-with-user-presence=false doesn't work with YubiKey

YubiKey's FIDO2 applet seems to ALWAYS require touch, so it cannot be used for unattended unlock of LUKS volume:

> systemd-cryptenroll --unlock-key-file=$(cat /tmp/emmc-etra.key | psub) --fido2-device=auto --fido2-with-client-pin=false --fido2-with-user-verification=false --fido2-with-user-presence=false /dev/disk/by-partlabel/disk-boot-emmc-etra-header
Initializing FIDO2 credential on security token.
👆 (Hint: This might require confirmation of user presence on security token.)
Generating secret key on FIDO2 security token.
👆 Locking without user presence test requested, but FIDO2 device /dev/hidraw1 requires it, enabling.
New FIDO2 token enrolled as key slot 1.

Keeping nixpkgs fork with arbitrary patches up to date

How to use https://github.com/katrinafyi/nix-patcher to maintain nixpkgs fork:

  1. Create required nixpkgs inputs, see nix-patcher repo for more details:
    • nixpkgs-upstream - desired upstream nixpkgs branch
    • nixpkgs - the fork you have access to that will be managed by nix-patcher
    • nixpkgs-patch-<num> - numbered patches to be applied in specific order on top of nixpkgs-upstream and pushed to nixpkgs fork
  2. run nix flake update && GITHUB_TOKEN="$(pass show python-keyring/git/github.com/nazarewk)" nix-patcher --update "$@"
  3. In your root flake.nix create the nixpkgs input pointing to the same input as the sub-flake.
  4. Run nix flake update nixpkgs in your root flake.

Most of it is wrapped in /nixpkgs-update.sh and top entries of /flake.nix.

Note this can be used to patch arbitrary flake, not just nixpkgs! Just create <input>-upstream, <input> & <input>-patch-<num> inputs.