I Provisioned My Cluster with Ansible Dynamic Inventories

[fanuch]

Dynamic Inventory Provisioning

Just wanted to share that I have provisioned my cluster with a Proxmox dynamic inventory

Thought it might be of interest.

You can include "tags" on your qemu guests through either the Ansible community.general.proxmox_kvm module or through the Terraform Telmate/Proxmox provider; I chose the latter.

Then you simply create a file called blah.proxmox.yaml (I called mine inventory.proxmox.yaml) and we can start querying for those tags we used!

Dynamic Inventory Config

plugin: community.general.proxmox
url: https://ipaddress:8006
user: username
password: password
# token_id: ansible
# token_secret: <token>
want_facts: true
validate_certs: false
want_proxmox_nodes_ansible_host: false
compose:
  ansible_host: proxmox_ipconfig0.ip | ipaddr('address')

Rendering Dynamic Hosts

When I run ansible-inventory -i inventory.proxmox.yaml --graph I get back something like:

@all:
  |--@proxmox:
  |  |--proxmox1
  |  |--proxmox2
  |  |--proxmox3
  |--@proxmox_all_lxc:
  |  |--adguard
  |  |--caddy
  |  |--frigate
  |  |--grafana
  |  |--kali-01
  |  |--uptimekuma
  |--@proxmox_all_qemu:
  |  |--Fixing
  |  |--docker1
  |  |--docker2
  |  |--docker3
  |  |--flatcar-template
  |  |--games1
  |  |--gitlab1
  |  |--gl-runner1
  |  |--haos8.1
  |  |--webhost1
  |--@proxmox_all_running:
  ...

Here is where it gets interesting ....

Creating Dynamic Groups to Tie into this Repo

If you append this to the inventory.proxmox.yaml file:

groups:
  k3s_cluster: "'k3s' in (proxmox_tags_parsed|list)"
  master: "'k3s-master' in (proxmox_tags_parsed|list)"
  node: "'k3s-nodes' in (proxmox_tags_parsed|list)"

(Yes I separated my provisioned master and nodes for easier management as IaC)

Then we can get back from our ansible-inventory:

@all:
  |--@k3s_cluster:
  |  |--k3s-control-01
  |  |--k3s-control-02
  |  |--k3s-control-03
  |  |--k3s-node-01
  |  |--k3s-node-02
  |  |--k3s-node-03
  |--@master:
  |  |--k3s-control-01
  |  |--k3s-control-02
  |  |--k3s-control-03
  |--@node:
  |  |--k3s-node-01
  |  |--k3s-node-02
  |  |--k3s-node-03
...

Deploy to N Number of Hosts

Back to this repo, you can run the magic command:

ansible-playbook site.yml -i inventory.proxmox.yaml -e @inventory/my-cluster/group_vars/all.yml

What this will do is create that dynamic inventory from Proxmox and the tags and blast away creating everything for you!

Caveats

• As we are not using the inventory directory I needed to yoink in my variables with the ...-e @... command
• Getting the IP address from the hosts dynamically is a PITA unless you have the same interface name or provision static IP through Cloud-Init
  • I ended up doing the latter for simplicity
  • Note the ansible_host: parameter in the inventory.proxmox.yaml file
  • For hosts I provisioned with DHCP and with the agent enabled, I was able to pull the IP by:
    • proxmox_agent_interfaces | selectattr('name','equalto','ens18')|map(attribute='ip-addresses')|first)[0] | ipaddr('address')
    • where ens18 was my interface name
    • You can keep iterating though ansible-inventory -i inventory.proxmox.yaml --host blah to see all labels from that host

P.S.

_{^{I don't care that a static inventory is "easier".
This transfers to other cloud providers and scaled much better}}