-
Notifications
You must be signed in to change notification settings - Fork 664
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
add example of integration with kubevirt
Signed-off-by: Jose Angel Morena <jmorenas@redhat.com>
- Loading branch information
Showing
8 changed files
with
390 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,106 @@ | ||
# Using Kubevirt | ||
|
||
Below you can see a scenario that is using [Kubevirt VMs](https://kubevirt.io/user-guide/) as test hosts. For Ansible to connect with the SSH in the KubeVirt VMs, it will be made accessible through the Service NodePort. | ||
When you run `molecule test --scenario kubevirt` the `create`, `converge` and | ||
`destroy` steps will be run one after another. | ||
|
||
This example is using Ansible playbooks and it does not need any molecule | ||
plugins to run. You can fully control which test requirements you need to be | ||
installed. | ||
|
||
## Prerequisites | ||
|
||
The `create.yml` and `destroy.yml` Ansible playbooks require the Ansible collection `kubernetes.core`. For seamless communication with the Kubernetes API server, the collection uses the following environment variables: | ||
|
||
- `K8S_AUTH_API_KEY`: This is the token from the service account used to authenticate with the Kubernetes cluster. | ||
|
||
- `K8S_AUTH_HOST`: This points to the URL of the Kubernetes cluster's API. | ||
|
||
- `K8S_AUTH_VERIFY_SSL`: If set to `false`, this disables the verification of SSL/TLS certificates, which might pose a security risk. It's mainly used for testing environments, particularly when dealing with self-signed certificates. | ||
|
||
Additionally, for the playbooks to work, the Kubernetes service account needs specific roles and role bindings to operate in a particular namespace. This ensures the playbook has sufficient privileges to execute commands on the Kubernetes resources. These roles include getting, listing, watching, creating, deleting, and editing virtual machines and services. | ||
|
||
```yaml | ||
--- | ||
apiVersion: v1 | ||
kind: ServiceAccount | ||
metadata: | ||
name: <Molecule Kubernetes Serviceaccount> | ||
namespace: <Kubernetes VM Namespace> | ||
--- | ||
apiVersion: rbac.authorization.k8s.io/v1 | ||
kind: Role | ||
metadata: | ||
namespace: <Kubernetes VM Namespace> | ||
name: <Molecule Kubernetes Role> | ||
rules: | ||
- apiGroups: ["kubevirt.io"] | ||
resources: ["virtualmachines"] | ||
verbs: ["get", "list", "watch", "create", "delete", "patch", "edit"] | ||
- apiGroups: [""] | ||
resources: ["services"] | ||
verbs: ["get", "list", "watch", "create", "delete", "patch", "edit"] | ||
--- | ||
apiVersion: rbac.authorization.k8s.io/v1 | ||
kind: RoleBinding | ||
metadata: | ||
name: <Molecule Kubernetes Rolebinding> | ||
namespace: <Kubernetes VM Namespace> | ||
subjects: | ||
- kind: ServiceAccount | ||
name: <Molecule Kubernetes Serviceaccount> | ||
namespace: <Kubernetes VM Namespace> | ||
roleRef: | ||
kind: Role | ||
name: <Molecule Kubernetes Role> | ||
apiGroup: rbac.authorization.k8s.io | ||
``` | ||
You will need to substitute the following placeholders: | ||
- `<Molecule Kubernetes Serviceaccount>`: This refers to the name of the Kubernetes Serviceaccount that the molecule test utilizes to create the KubeVirt VM. | ||
- `<Kubernetes VM Namespace>`: This denotes the name of the Kubernetes namespace where the VMs will be instantiated. | ||
- `<Molecule Kubernetes Role>`: This is the name of the Kubernetes role which encapsulates the necessary permissions for the molecule test to function. | ||
- `<Molecule Kubernetes Rolebinding>`: This represents the name of the Kubernetes rolebinding that associates the role `<Molecule Kubernetes Role>` with the serviceaccount `<Molecule Kubernetes Serviceaccount>`. | ||
|
||
## Considerations | ||
- This example is using ephemeral VMs, which enhance the speed of VM creation and cleanup. However, it is important to note that any data in the system will not be retained if the VM is rebooted. | ||
- You don't need to worry about setting up SSH keys. The `create.yml` Ansible playbook takes care of configuring a temporary SSH key. | ||
|
||
## Config playbook | ||
|
||
```yaml title="molecule.yml" | ||
{!../molecule/kubevirt/molecule.yml!} | ||
``` | ||
|
||
Please, replace the following parameters: | ||
- `<Kubernetes VM Namespace>`: This should be replaced with the namespace in Kubernetes where you intend to create the KubeVirt VMs. | ||
- `<Kubernetes Node FQDN>`: Change this to the fully qualified domain name (FQDN) of the Kubernetes node that Ansible will attempt to SSH into via the Service NodePort. | ||
|
||
```yaml title="requirements.yml" | ||
{!../molecule/kubevirt/requirements.yml!} | ||
``` | ||
|
||
## Create playbook | ||
|
||
```yaml title="create.yml" | ||
{!../molecule/kubevirt/create.yml!} | ||
``` | ||
|
||
```yaml title="tasks/create_vm.yml" | ||
{!../molecule/kubevirt/tasks/create_vm.yml!} | ||
``` | ||
|
||
```yaml title="tasks/create_vm_dictionary.yml" | ||
{!../molecule/kubevirt/tasks/create_vm_dictionary.yml!} | ||
``` | ||
|
||
## Converge playbook | ||
|
||
```yaml title="converge.yml" | ||
{!../molecule/kubevirt/converge.yml!} | ||
``` | ||
|
||
## Destroy playbook | ||
|
||
```yaml title="destroy.yml" | ||
{!../molecule/kubevirt/destroy.yml!} | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
- name: Fail if molecule group is missing | ||
hosts: localhost | ||
tasks: | ||
- name: Print some info | ||
ansible.builtin.debug: | ||
msg: "{{ groups }}" | ||
|
||
- name: Assert group existence | ||
ansible.builtin.assert: | ||
that: "'molecule' in groups" | ||
fail_msg: | | ||
molecule group was not found inside inventory groups: {{ groups }} | ||
- name: Converge | ||
hosts: molecule | ||
# We disable gather facts because it would fail due to our container not | ||
# having python installed. This will not prevent use from running 'raw' | ||
# commands. Most molecule users are expected to use containers that already | ||
# have python installed in order to avoid notable delays installing it. | ||
gather_facts: false | ||
tasks: | ||
- name: Check uname | ||
ansible.builtin.raw: uname -a | ||
register: result | ||
changed_when: false | ||
|
||
- name: Print some info | ||
ansible.builtin.assert: | ||
that: result.stdout | regex_search("^Linux") |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,96 @@ | ||
- name: Create | ||
hosts: localhost | ||
connection: local | ||
gather_facts: false | ||
vars: | ||
temporary_ssh_key_size: 2048 # Variable for the size of the SSH key | ||
tasks: | ||
- name: Set default SSH key path # Sets the path of the SSH key | ||
set_fact: | ||
tempoary_ssh_key_path: "{{ molecule_ephemeral_directory }}/identity_file" | ||
|
||
- name: Generate SSH key pair # Generates a new SSH key pair | ||
community.crypto.openssh_keypair: | ||
path: "{{ tempoary_ssh_key_path }}" | ||
size: "{{ temporary_ssh_key_size }}" | ||
register: temporary_ssh_keypair # Stores the output of this task in a variable | ||
|
||
- name: Set SSH public key # Sets the SSH public key from the key pair | ||
set_fact: | ||
temporary_ssh_public_key: "{{ temporary_ssh_keypair.public_key }}" | ||
|
||
- name: Create VM in KubeVirt # Calls another file to create the VM in KubeVirt | ||
include_tasks: tasks/create_vm.yml | ||
loop: "{{ molecule_yml.platforms }}" # Loops over all platforms defined in molecule_yml | ||
loop_control: | ||
loop_var: vm # Sets the variable for the current item in the loop | ||
|
||
- name: Create Nodeport service if ssh_type is set to NodePort # Conditional block, executes if vm.ssh_service.type is NodePort | ||
block: | ||
- name: Create ssh NodePort Kubernetes Services # Creates a new NodePort service in Kubernetes | ||
kubernetes.core.k8s: | ||
state: present | ||
definition: | ||
apiVersion: v1 | ||
kind: Service | ||
metadata: | ||
name: "{{ vm.name }}" | ||
namespace: "{{ vm.namespace }}" | ||
spec: | ||
ports: | ||
- port: 22 | ||
protocol: TCP | ||
targetPort: 22 | ||
selector: | ||
kubevirt.io/domain: "{{ vm.name }}" | ||
type: NodePort | ||
loop: "{{ molecule_yml.platforms }}" # Loops over all platforms defined in molecule_yml | ||
loop_control: | ||
loop_var: vm # Sets the variable for the current item in the loop | ||
|
||
- name: Retrieve Service Info # Retrieves information about the service | ||
kubernetes.core.k8s_info: | ||
api_version: v1 | ||
kind: Service | ||
name: "{{ vm.name }}" | ||
namespace: "{{ vm.namespace }}" | ||
loop: "{{ molecule_yml.platforms }}" # Loops over all platforms defined in molecule_yml | ||
loop_control: | ||
loop_var: vm # Sets the variable for the current item in the loop | ||
register: node_port_services # Stores the output of this task in a variable | ||
when: "vm.ssh_service.type == 'NodePort'" # The block is executed when this condition is met | ||
|
||
- name: Create VM dictionary # Calls another file to create a dictionary with information about the VM | ||
include_tasks: tasks/create_vm_dictionary.yml | ||
loop: "{{ molecule_yml.platforms }}" # Loops over all platforms defined in molecule_yml | ||
loop_control: | ||
loop_var: vm # Sets the variable for the current item in the loop | ||
|
||
- name: Create ansible inventory from dictionary # Creates an Ansible inventory file from the dictionary | ||
vars: | ||
molecule_inventory: | ||
all: | ||
children: | ||
molecule: | ||
hosts: "{{ molecule_systems }}" | ||
ansible.builtin.copy: | ||
content: "{{ molecule_inventory | to_nice_yaml }}" | ||
dest: "{{ molecule_ephemeral_directory }}/inventory/molecule_inventory.yml" | ||
mode: 0600 # Sets the permissions of the file to -rw------- | ||
|
||
- name: Refresh inventory # Refreshes the inventory | ||
ansible.builtin.meta: refresh_inventory | ||
|
||
- name: Assert molecule group exists # Checks if the 'molecule' group exists in the inventory | ||
ansible.builtin.assert: | ||
that: "'molecule' in groups" | ||
fail_msg: "Molecule group was not found in inventory groups: {{ groups }}" | ||
run_once: true # Ensures this task is only run once, not on every host in 'hosts' | ||
|
||
- name: Validate that inventory was refreshed # New playbook to validate the inventory | ||
hosts: molecule # Runs on hosts in the 'molecule' group | ||
gather_facts: false # Disables fact gathering | ||
tasks: | ||
- name: Wait for the host to be reachable # Waits for the host to become reachable | ||
ansible.builtin.wait_for_connection: | ||
timeout: 120 # Waits for up to 120 seconds |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
--- | ||
- name: Destroy | ||
hosts: localhost | ||
connection: local | ||
gather_facts: false | ||
tasks: | ||
- name: Delete VM Instance in KubeVirt | ||
kubernetes.core.k8s: | ||
state: absent | ||
kind: VirtualMachine | ||
name: "{{ vm.name }}" | ||
namespace: "{{ vm.namespace }}" | ||
loop: "{{ molecule_yml.platforms }}" | ||
loop_control: | ||
loop_var: vm | ||
|
||
- name: Delete VM Instance in KubeVirt | ||
kubernetes.core.k8s: | ||
state: absent | ||
kind: Service | ||
name: "{{ vm.name }}" | ||
namespace: "{{ vm.namespace }}" | ||
loop: "{{ molecule_yml.platforms }}" | ||
loop_control: | ||
loop_var: vm |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
--- | ||
dependency: | ||
name: galaxy | ||
options: | ||
requirements-file: requirements.yml | ||
role-file: requirements.yml | ||
platforms: | ||
- name: rhel9 | ||
image: registry.redhat.io/rhel9/rhel-guest-image | ||
namespace: <Kubernetes VM Namespace> | ||
ssh_service: | ||
type: NodePort | ||
nodeport_host: <Kubernetes Node FQDN> | ||
ansible_user: cloud-user | ||
memory: 1Gi | ||
- name: rhel8 | ||
image: registry.redhat.io/rhel8/rhel-guest-image | ||
namespace: <Kubernetes VM Namespace> | ||
ssh_service: | ||
type: NodePort | ||
nodeport_host: <Kubernetes Node FQDN> | ||
ansible_user: cloud-user | ||
memory: 1Gi | ||
provisioner: | ||
name: ansible | ||
config_options: | ||
defaults: | ||
interpreter_python: auto_silent | ||
callback_whitelist: profile_tasks, timer, yaml | ||
ssh_connection: | ||
pipelining: false | ||
log: true | ||
verifier: | ||
name: ansible | ||
scenario: | ||
test_sequence: | ||
- dependency | ||
- destroy | ||
- syntax | ||
- create | ||
- converge | ||
- idempotence | ||
- side_effect | ||
- verify | ||
- destroy |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
--- | ||
collections: | ||
- name: kubernetes.core | ||
- name: community.crypto |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,59 @@ | ||
--- | ||
- name: Create VM in KubeVirt | ||
kubernetes.core.k8s: # Uses the k8s module from the kubernetes.core Ansible collection | ||
state: present # Ensures the VM exists. If it doesn't, it will be created. | ||
definition: | ||
apiVersion: kubevirt.io/v1 # KubeVirt's API version | ||
kind: VirtualMachine # The type of Kubernetes resource to create | ||
metadata: | ||
labels: | ||
kubevirt.io/domain: "{{ vm.name }}" # Labels for the VM | ||
name: "{{ vm.name }}" # Name of the VM | ||
namespace: "{{ vm.namespace }}" # Namespace where the VM will be created | ||
spec: | ||
running: true # Starts the VM after creation | ||
template: | ||
metadata: | ||
labels: | ||
kubevirt.io/domain: "{{ vm.name }}" # Labels for the VM's template | ||
spec: | ||
domain: | ||
devices: | ||
disks: | ||
- disk: | ||
bus: virtio # Type of disk bus | ||
name: containerdisk # Name of the container disk | ||
- disk: | ||
bus: virtio # Type of disk bus | ||
name: cloudinitdisk # Name of the cloud-init disk | ||
- name: emptydisk # Name of the empty disk | ||
disk: | ||
bus: virtio # Type of disk bus | ||
resources: | ||
requests: | ||
memory: "{{ vm.memory | default('1Gi') }}" # Amount of memory requested for the VM | ||
volumes: | ||
- name: emptydisk | ||
emptyDisk: | ||
capacity: "{{ vm.capacity | default('2Gi') }}" # Capacity of the empty ephemeral disk | ||
- containerDisk: | ||
image: "{{ vm.image }}" # The image used for the container disk | ||
name: containerdisk | ||
- cloudInitNoCloud: # Cloud-init configuration | ||
userData: | # User-data script | ||
#cloud-config | ||
preserve_hostname: true | ||
hostname: "{{ vm.name }}" # Sets the hostname | ||
fqdn: "{{ vm.name }}" # Fully Qualified Domain Name | ||
prefer_fqdn_over_hostname: true | ||
users: | ||
- default | ||
- name: {{ vm.ansible_user }} | ||
lock_passwd: true # Locks the password | ||
ssh_authorized_keys: | ||
- "{{ temporary_ssh_public_key }}" # SSH public key | ||
runcmd: | ||
- [ sh, -c, "hostnamectl set-hostname {{ vm.name }}" ] # Sets the hostname | ||
- [ sudo, yum, install, -y, qemu-guest-agent ] # Installs qemu-guest-agent | ||
- [ sudo, systemctl, start, qemu-guest-agent ] # Starts qemu-guest-agent | ||
name: cloudinitdisk |
Oops, something went wrong.