Skip to content

Commit

Permalink
test: introduce an integration test suite
Browse files Browse the repository at this point in the history
Since we already use Packit, let's utilize its second part as well and
add a TMT [0] based integration test suite.

This PR add a second job to the existing Packit configuration, which
then sends the just built RPMs to TestingFarm [1] that executes all
selected tests (which currently means all discovered tests).

To demonstrate the functionality a bit, this PR also adds a simple test
case for polkit-org#439.

[0] https://tmt.readthedocs.io/en/stable/overview.html
[1] https://docs.testing-farm.io/Testing%20Farm/0.1/index.html
  • Loading branch information
mrc0mmand committed Jun 6, 2024
1 parent 060b34c commit d570c42
Show file tree
Hide file tree
Showing 8 changed files with 121 additions and 0 deletions.
8 changes: 8 additions & 0 deletions .packit.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,11 @@ jobs:
trigger: pull_request
targets:
- fedora-all

- job: tests
trigger: pull_request
fmf_path: test/integration/
tmt_plan: upstream_ci
targets:
- fedora-all-x86_64
- fedora-all-aarch64
1 change: 1 addition & 0 deletions test/integration/.fmf/version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1
54 changes: 54 additions & 0 deletions test/integration/README
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# polkit integration test suite

polkit's integration test suite uses TMT (Test Management Tool [0]) to organize and run tests. Since TMT
offers a _lot_ of features, this document pinpoints the most _interesting_ ones to get stuff up and running
quickly.

## How to contribute

Creating a new test case is pretty simple:

```
$ cd test/integration
$ tmt test create --template=shell test/name
Test directory '/home/.../polkit/test/integration/test/name' created.
Test metadata '/home/.../polkit/test/integration/test/name/main.fmf' created.
Test script '/home/.../polkit/test/integration/test/name/test.sh' created.
```

The newly created `test.sh` will be the actual test case, and `main.fmf` contains the test metadata, including
test summary & description, test dependencies, runtime, and so on. See [1] for more details.

After tweaking the test metadata it's usually a good idea to run `tmt lint test/name` to make sure that the
configuration is still valid:

```
$ tmt lint test/name
/test/name
pass C000 fmf node passes schema validation
pass C001 summary key is set and is reasonably long
...
```

To check if the test itself works as expected you can use `tmt run`:

```
$ cd test/name
$ tmt run -vvv --all provision --how local tests --name .
...
total: 1 test passed
```

The `tmt run` command is _very_ customizable (as is the rest of `tmt`). In this particular example we tell it
to run all steps (`--all`) and override the `provision` and `tests` steps to run just one particular test on
the local machine. As in previous cases, check the `tmt` documentation [0] and examples [2] for more details.

## Links

[0] https://tmt.readthedocs.io/en/stable/overview.html

[1] https://tmt.readthedocs.io/en/stable/spec/tests.html

[2] https://tmt.readthedocs.io/en/stable/examples.html#run

<!-- vim: set syntax=markdown tw=110 : -->
2 changes: 2 additions & 0 deletions test/integration/main.fmf
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
require:
- polkit
11 changes: 11 additions & 0 deletions test/integration/plans/upstream_ci.fmf
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# vi: set sw=2 ts=2 et ft=yaml tw=80:

# This plan discovers and executes all (enabled) integration tests

summary: Upstream integration test suite
/:
inherit: false
discover:
how: fmf
execute:
how: tmt
2 changes: 2 additions & 0 deletions test/integration/rules/main.fmf
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
summary: Check polkit's behavior with various rules
test: ./test.sh
8 changes: 8 additions & 0 deletions test/integration/rules/test-rules/testuser-start-unit.rules
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
polkit.addRule(function(action, subject) {
if (action.id == "org.freedesktop.systemd1.manage-units") {
if (subject.user == "polkit-testuser" && action.lookup("unit") == "simple-test-unit.service" &&
(action.lookup("verb") == "restart" || action.lookup("verb") == "stop" || action.lookup("verb") == "start")) {
return polkit.Result.YES;
}
}
});
35 changes: 35 additions & 0 deletions test/integration/rules/test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
#!/bin/bash

set -eux
set -o pipefail

TEST_RULES="$PWD/test-rules"
TEST_USER="polkit-testuser"

: "Setup"
mkdir -p /run/systemd/system/
useradd "$TEST_USER"

: "Allow $TEST_USER to start/restart/stop a simple systemd unit"
# Use `systemctl edit --full ...` in the future
cat >/run/systemd/system/simple-test-unit.service <<EOF
[Unit]
Type=oneshot
ExecStart=true
EOF
systemctl daemon-reload
# Copy the test polkit rule in place
cp -fv "$TEST_RULES/testuser-start-unit.rules" /etc/polkit-1/rules.d/99-test.rules
# Following systemctl invocations should not trigger polkit's authentication prompt
timeout 5s sudo -u testuser systemctl start simple-test-unit.service
timeout 5s sudo -u testuser systemctl restart simple-test-unit.service
timeout 5s sudo -u testuser systemctl stop simple-test-unit.service
# But this one should (and should hence timeout with exit code 124)
timeout 5s sudo -u testuser systemctl restart systemd-journald.service && ec=0 || ec=$?
[[ $ec -eq 124 ]]
# Cleanup
rm -f /etc/polkit-1/rules.d/99-test.rules

: "Cleanup"
userdel -rf "$TEST_USER"
rm -f /etc/polkit-1/rules.d/99-test.rules

0 comments on commit d570c42

Please sign in to comment.