Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use the api to create a set of reusable fixtures #43

Merged
merged 4 commits into from
Nov 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,10 @@ jobs:
run: pip install -r requirements.txt
- name: Install collection dependencies
run: ansible-galaxy install -r requirements.yml
- name: Start VM
run: vagrant up quadlet
- name: Start VMs
run: |
vagrant up quadlet
vagrant up client
Comment on lines +54 to +55
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you intentionally start them up sequentially instead of parallel? Is that the issue with fetching the same box concurrently giving issues?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I honestly haven't tried 😅

- name: Run deployment
run: |
ansible-playbook playbooks/setup.yaml
Expand Down
9 changes: 9 additions & 0 deletions Vagrantfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,13 @@ Vagrant.configure("2") do |config|
provider.vm.box_url = CENTOS_9_BOX_URL
end
end

config.vm.define "client" do |override|
override.vm.hostname = "client.example.com"

override.vm.provider "libvirt" do |libvirt, provider|
libvirt.memory = 1024
provider.vm.box_url = CENTOS_9_BOX_URL
end
end
end
3 changes: 2 additions & 1 deletion playbooks/deploy.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
---
- name: Setup quadlet demo machine
hosts: all
hosts:
- quadlet
become: true
vars:
certificates_hostnames:
Expand Down
9 changes: 8 additions & 1 deletion playbooks/setup.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,14 @@
---
- name: Setup quadlet demo machine
- name: Setup basic stuff
hosts: all
become: true
roles:
- theforeman.forklift.etc_hosts
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎆


- name: Setup quadlet demo machine
hosts:
- quadlet
become: true
pre_tasks:
- name: Upgrade all packages
ansible.builtin.package: # noqa: package-latest
Expand Down
1 change: 1 addition & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
pytest-testinfra
paramiko
apypie>=0.5.0
2 changes: 2 additions & 0 deletions requirements.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ collections:
- name: containers.podman
version: ">=1.14.0"
- name: theforeman.foreman
- name: https://github.com/theforeman/forklift
type: git

roles:
- name: geerlingguy.postgresql
2 changes: 1 addition & 1 deletion roles/foreman/defaults/main.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
---
foreman_container_image: "quay.io/evgeni/foreman-rpm"
foreman_container_tag: "3.12"
foreman_container_tag: "nightly"
10 changes: 10 additions & 0 deletions tests/client_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
def test_foreman_content_view(client_environment, activation_key, organization, foremanapi, client):
client.run('dnf install -y subscription-manager')
rcmd = foremanapi.create('registration_commands', {'organization_id': organization['id'], 'insecure': True, 'activation_keys': [activation_key['name']]})
client.run_test(rcmd['registration_command'])
client.run('subscription-manager repos --enable=*')
client.run_test('dnf install -y bear')
assert client.package('bear').is_installed
client.run('dnf remove -y bear')
client.run('subscription-manager unregister')
client.run('subscription-manager clean')
94 changes: 94 additions & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,101 @@
import uuid

import apypie
import paramiko
import pytest
import testinfra


@pytest.fixture(scope="module")
def server():
yield testinfra.get_host('paramiko://quadlet', sudo=True, ssh_config='./.vagrant/ssh-config')


@pytest.fixture(scope="module")
def client():
yield testinfra.get_host('paramiko://client', sudo=True, ssh_config='./.vagrant/ssh-config')


@pytest.fixture(scope="module")
def ssh_config():
config = paramiko.SSHConfig.from_path('./.vagrant/ssh-config')
return config.lookup('quadlet')


@pytest.fixture(scope="module")
def foremanapi(ssh_config):
return apypie.ForemanApi(
uri=f'https://{ssh_config['hostname']}',
username='admin',
password='changeme',
verify_ssl=False,
)

@pytest.fixture
def organization(foremanapi):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now all these tests have an (implicit) scope=function (https://docs.pytest.org/en/stable/how-to/fixtures.html#fixture-scopes) which means that they are destroyed at the end of every test.

Conceptually, this is good, of course. But on the flip side this means that every repository test also first creates a new org (also in Candlepin), a new product, etc. Those times quickly add up, and make the test tiresome to execute.

We can speed this up with scope=module, but this means that any failure in a test might have cascading effects on later tests (which is the same behaviour as we have today with bats, FWIW).

WDYT?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now I'd lean to the isolation given we still have very few tests, but I agree in the future it will add up. Later we can add a "persistent" fixture that can be reused.

Given we have isolation, it's probably also easy to run things in parallel. Those failures may be harder to diagnose, but it can also uncover real work concurrency issues which large deployments will see at some point.

org = foremanapi.create('organizations', {'name': str(uuid.uuid4())})
yield org
foremanapi.delete('organizations', org)

@pytest.fixture
def product(organization, foremanapi):
prod = foremanapi.create('products', {'name': str(uuid.uuid4()), 'organization_id': organization['id']})
yield prod
foremanapi.delete('products', prod)

@pytest.fixture
def yum_repository(product, organization, foremanapi):
repo = foremanapi.create('repositories', {'name': str(uuid.uuid4()), 'product_id': product['id'], 'content_type': 'yum', 'url': 'https://fixtures.pulpproject.org/rpm-no-comps/'})
Comment on lines +47 to +48
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know how far you want to take this, but I think you can even make the URL a parameter so it can be overridden

Suggested change
def yum_repository(product, organization, foremanapi):
repo = foremanapi.create('repositories', {'name': str(uuid.uuid4()), 'product_id': product['id'], 'content_type': 'yum', 'url': 'https://fixtures.pulpproject.org/rpm-no-comps/'})
def yum_repository(product, organization, foremanapi, url='https://fixtures.pulpproject.org/rpm-no-comps/'):
repo = foremanapi.create('repositories', {'name': str(uuid.uuid4()), 'product_id': product['id'], 'content_type': 'yum', 'url': url})

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think fixtures take regular parameters like this (there are fixture factories, but those are more complicated)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://docs.pytest.org/en/7.1.x/example/parametrize.html#indirect-parametrization says you can do it, but you need to use request.param. Feel free to leave it out for now, but let's keep it in mind for the future.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh via indirect… Mhh, interesting. Maybe. I'll think about it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#60 is to track this, merging as is for now

yield repo
foremanapi.delete('repositories', repo)

@pytest.fixture
def file_repository(product, organization, foremanapi):
repo = foremanapi.create('repositories', {'name': str(uuid.uuid4()), 'product_id': product['id'], 'content_type': 'file', 'url': 'https://fixtures.pulpproject.org/file/'})
yield repo
foremanapi.delete('repositories', repo)

@pytest.fixture
def container_repository(product, organization, foremanapi):
repo = foremanapi.create('repositories', {'name': str(uuid.uuid4()), 'product_id': product['id'], 'content_type': 'docker', 'url': 'https://quay.io/', 'docker_upstream_name': 'foreman/busybox-test'})
yield repo
foremanapi.delete('repositories', repo)

@pytest.fixture
def lifecycle_environment(organization, foremanapi):
library = foremanapi.list('lifecycle_environments', 'name=Library', {'organization_id': organization['id']})[0]
Comment on lines +65 to +66
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here perhaps?

Suggested change
def lifecycle_environment(organization, foremanapi):
library = foremanapi.list('lifecycle_environments', 'name=Library', {'organization_id': organization['id']})[0]
def lifecycle_environment(organization, foremanapi, prior='Library'):
library = foremanapi.list('lifecycle_environments', f'name={prior}', {'organization_id': organization['id']})[0]

Though perhaps it should just accept an instance where None is automatically resolve to the library:

Suggested change
def lifecycle_environment(organization, foremanapi):
library = foremanapi.list('lifecycle_environments', 'name=Library', {'organization_id': organization['id']})[0]
def lifecycle_environment(organization, foremanapi, prior=None):
if not prior:
prior = foremanapi.list('lifecycle_environments', 'name=Library', {'organization_id': organization['id']})[0]

Which also makes me question what the Katello API does if no prior ID is given. It's not listed as required and mandating this, but would it make sense to find the library server side and make it an optional parameter?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant that last part as an RFE.

lce = foremanapi.create('lifecycle_environments', {'name': str(uuid.uuid4()), 'organization_id': organization['id'], 'prior_id': library['id']})
yield lce
foremanapi.delete('lifecycle_environments', lce)

@pytest.fixture
def content_view(organization, foremanapi):
cv = foremanapi.create('content_views', {'name': str(uuid.uuid4()), 'organization_id': organization['id']})
yield cv
foremanapi.delete('content_views', cv)

@pytest.fixture
def activation_key(organization, foremanapi):
ak = foremanapi.create('activation_keys', {'name': str(uuid.uuid4()), 'organization_id': organization['id']})
yield ak
foremanapi.delete('activation_keys', ak)

@pytest.fixture
def client_environment(activation_key, content_view, lifecycle_environment, yum_repository, organization, foremanapi):
foremanapi.resource_action('repositories', 'sync', {'id': yum_repository['id']})
foremanapi.update('content_views', {'id': content_view['id'], 'repository_ids': [yum_repository['id']]})
foremanapi.resource_action('content_views', 'publish', {'id': content_view['id']})

library = foremanapi.list('lifecycle_environments', 'name=Library', {'organization_id': organization['id']})[0]
foremanapi.update('activation_keys', {'id': activation_key['id'], 'organization_id': organization['id'], 'environment_id': library['id'], 'content_view_id': content_view['id']})

yield activation_key

foremanapi.update('activation_keys', {'id': activation_key['id'], 'organization_id': organization['id'], 'environment_id': None, 'content_view_id': None})
evgeni marked this conversation as resolved.
Show resolved Hide resolved

versions = foremanapi.list('content_view_versions', params={'content_view_id': content_view['id']})
for version in versions:
current_environment_ids = {environment['id'] for environment in version['environments']}
for environment_id in current_environment_ids:
foremanapi.resource_action('content_views', 'remove_from_environment', params={'id': content_view['id'], 'environment_id': environment_id})
foremanapi.delete('content_view_versions', version)
50 changes: 50 additions & 0 deletions tests/foreman_api_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
import urllib.parse

import requests


def _repo_url(repo, ssh_config):
return urllib.parse.urlunparse(urllib.parse.urlparse(repo['full_path'])._replace(netloc=ssh_config['hostname']))
ekohl marked this conversation as resolved.
Show resolved Hide resolved


def test_foreman_organization(organization):
assert organization

def test_foreman_product(product):
assert product

def test_foreman_yum_repository(yum_repository, foremanapi, ssh_config):
assert yum_repository
foremanapi.resource_action('repositories', 'sync', {'id': yum_repository['id']})
repo_url = _repo_url(yum_repository, ssh_config)
assert requests.get(f'{repo_url}/repodata/repomd.xml', verify=False)
assert requests.get(f'{repo_url}/Packages/b/bear-4.1-1.noarch.rpm', verify=False)


def test_foreman_file_repository(file_repository, foremanapi, ssh_config):
assert file_repository
foremanapi.resource_action('repositories', 'sync', {'id': file_repository['id']})
repo_url = _repo_url(file_repository, ssh_config)
assert requests.get(f'{repo_url}/1.iso', verify=False)


def test_foreman_container_repository(container_repository, foremanapi, ssh_config):
assert container_repository
foremanapi.resource_action('repositories', 'sync', {'id': container_repository['id']})


def test_foreman_lifecycle_environment(lifecycle_environment):
assert lifecycle_environment


def test_foreman_content_view(content_view, yum_repository, foremanapi):
assert content_view
foremanapi.update('content_views', {'id': content_view['id'], 'repository_ids': [yum_repository['id']]})
foremanapi.resource_action('content_views', 'publish', {'id': content_view['id']})
# do something with the published view
versions = foremanapi.list('content_view_versions', params={'content_view_id': content_view['id']})
for version in versions:
current_environment_ids = {environment['id'] for environment in version['environments']}
for environment_id in current_environment_ids:
foremanapi.resource_action('content_views', 'remove_from_environment', params={'id': content_view['id'], 'environment_id': environment_id})
foremanapi.delete('content_view_versions', version)