Upgrade fails when ansible_host is not set for control plane nodes

[dgerhardt]

With the lastest kubespray release, the cluster upgrade (initial installation not tested but likely effected in the same way) fails when ansible_host is not set for nodes in the inventory. This issue did not occur with previous releases using the same inventory config.

This issue was introduced with acb6f24 (#9194). After reverting this commit, the upgrade succeeded.

Environment:

• Cloud provider or hardware configuration:
  Bare metal/KVM (5 nodes)

• OS (printf "$(uname -srm)\n$(cat /etc/os-release)\n"):
  Linux 5.10.0-17-amd64 x86_64
  Debian GNU/Linux 11 (bullseye)

• Version of Ansible (ansible --version):
  ansible [core 2.12.5]

• Version of Python (python --version):
  Python 3.8.10

Kubespray version (commit) (git rev-parse --short HEAD):
v2.20.0 (18efdc2)

Network plugin used:
Calico

Full inventory with variables (ansible -i inventory/sample/inventory.ini all -m debug -a "var=hostvars[inventory_hostname]"):

"hostvars[inventory_hostname].groups":

    {
        "all": [
            "h4.example.com",
            "h4n1.example.com",
            "h4n2.example.com",
            "h4n3.example.com",
            "h4n4.example.com",
            "h4n5.example.com",
            "h4n6.example.com"
        ],
        "clusters": [
            "h4.example.com",
            "h4n1.example.com",
            "h4n2.example.com",
            "h4n3.example.com",
            "h4n4.example.com",
            "h4n5.example.com",
            "h4n6.example.com"
        ],
        "etcd": [
            "h4n3.example.com"
        ],
        "h4_nodes": [
            "h4n1.example.com",
            "h4n2.example.com",
            "h4n3.example.com",
            "h4n4.example.com",
            "h4n5.example.com",
            "h4n6.example.com"
        ],
        "myisp": [
            "h4.example.com",
            "h4n1.example.com",
            "h4n2.example.com",
            "h4n3.example.com",
            "h4n4.example.com",
            "h4n5.example.com",
            "h4n6.example.com"
        ],
        "k8s_cluster": [
            "h4n2.example.com",
            "h4n3.example.com",
            "h4n4.example.com",
            "h4n5.example.com",
            "h4n6.example.com"
        ],
        "kube_control_plane": [
            "h4n2.example.com"
        ],
        "kube_node": [
            "h4n4.example.com",
            "h4n5.example.com",
            "h4n6.example.com"
        ],
        "kvm_hosts": [
            "h4.example.com"
        ],
        "mail": [
            "h4n1.example.com"
        ],
        "ungrouped": []
    }

ansible -i my.inventory.yml -m debug -a var=hostvars[inventory_hostname].ansible_host -l k8s_cluster

h4n2.example.com | SUCCESS => {
    "hostvars[inventory_hostname].ansible_host": "VARIABLE IS NOT DEFINED!"
}
h4n3.example.com | SUCCESS => {
    "hostvars[inventory_hostname].ansible_host": "VARIABLE IS NOT DEFINED!"
}
h4n4.example.com | SUCCESS => {
    "hostvars[inventory_hostname].ansible_host": "VARIABLE IS NOT DEFINED!"
}
h4n5.example.com | SUCCESS => {
    "hostvars[inventory_hostname].ansible_host": "VARIABLE IS NOT DEFINED!"
}
h4n6.example.com | SUCCESS => {
    "hostvars[inventory_hostname].ansible_host": "VARIABLE IS NOT DEFINED!"
}

Command used to invoke ansible:
ansible-playbook -u myuser --become -i my.inventory.yml upgrade-cluster.yml

Output of ansible run:

[...]

TASK [kubernetes/node : Write kubelet systemd init file] ***************************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ansible.errors.AnsibleUndefinedVariable: {{ groups['kube_control_plane'] | map('extract', hostvars, ['ansible_host']) | join(' ') }}: 'ansible.vars.hostvars.HostVarsVars object' has no attribute 'ansible_host'
fatal: [h4n2.example.com]: FAILED! => {"changed": false, "msg": "AnsibleUndefinedVariable: {{ groups['kube_control_plane'] | map('extract', hostvars, ['ansible_host']) | join(' ') }}: 'ansible.vars.hostvars.HostVarsVars object' has no attribute 'ansible_host'"}

Anything else do we need to know:
kubelet_systemd_hardening: false
Template line which causes the error: acb6f24#diff-184b5937f6be4d63c1e833fd13903f1560034737d2e7ca93c631ea605ed2092eR29

[rickerc]

Confirmation that this also impacts new installs (as expected)