Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add build time QEMU/UEFI tests #26

Merged
merged 5 commits into from
May 20, 2024
Merged

Conversation

pbatard
Copy link
Collaborator

@pbatard pbatard commented May 18, 2024

Now that we have GitHub Actions build, we can use them to perform runtime validation, at build time, of the UEFI executables we produce through QEMU, as well as make the build artefacts available for download.

To accomplish that, we first need to fix the MinGW builds, that were producing broken executables (hence the reason why you want to have runtime tests at build) as well as fix the output of the apps we are going to use for testing, that were producing unwanted extra lines. This is accomplished with the first 2 patches.

The next patch then enables the generation of the GitHub Actions artifacts, which we will need for testing, and the last patch adds the actual tests, where you will see, on GitHub Actions output, that an extra task for the tests is now run after a successful build.

The testing is designed to be flexible and easy to modify (to add more tests as needed), as the whole test sequence, expected output, and buildup/teardown is defined in the test_list.txt file. This is based on what I already use in uefi-md5sum.

Basically, a test entry in that file starts with a # line that defines the display name for the test, and bash commands prefixed by > or <, that are used for buildup and teardown respectively, and, without any prefix, the expected text output of the test. Some shell variables ($UEFI_DIR and $UEFI_ARCH) can also be used in the bash commands.

Since we run UEFI applications and not UEFI bootloaders, most of the buildup is designed to create a startup.nsh Shell script that will be automatically executed by the Shell before we shutdown the QEMU UEFI firmware with reset -s (since we need to exit QEMU so that we don't error on a timeout, which is also detected).

This text file can then be converted to a series of bash scripts through gen_tests.sh, which are then run through the run_tests.sh script, which then produces an output similar to:

Test #001: ConOut text output...                                [PASS]
Test #002: Test args...                                         [PASS]
Test #003: Print args...                                        [PASS]

===============================================================================
# Test summary for gnu-efi
===============================================================================
# TOTAL: 3
# PASS:  3
# FAIL:  0
# ERROR: 0
===============================================================================

This is the output you will see, for each arch, in the Run tests step of the tests action.

At this stage, this testing obviously relies on elements that need to be sourced from third-parties, namely the AVMF/OVMF firmwares, a copy of which (along with their provenance) can be found at https://github.com/pbatard/EfiFs/tree/gh-pages/AAVMF and https://github.com/pbatard/EfiFs/tree/gh-pages/OVMF which is the content behind the https://efi.akeo.ie URL and that have been sourced from Debian packages, as well as the UEFI Shell, for which I also provide executables, built directly from stable EDK2 builds through GitHub Actions, at https://github.com/pbatard/UEFI-Shell.

For now, only x64, ia32, aa64 and arm are being tested, but I'll see if I can work on adding riscv64 at a later date.

Hopefully, this will help detect regressions as they arise.

pbatard added 4 commits May 18, 2024 15:37
The EFI subsystem and application entrypoint were not being set for MinGW builds
and we don't need to go through objcopy either, so fix that.
Also make sure that the link options can be understood by the compiler, which
is what we use for linking, and remove the extra GNU_EFI_USE_MS_ABI, that was
already automatically added.

Signed-off-by: Pete Batard <pete@akeo.ie>
LFCR results in unwanted blank lines being added on serial output.
CRLF is what UEFI actually uses.
Also fix unneeded CR added to Print() optput in tcc.c.

These fixes are needed to to use these application for build time tests.

Signed-off-by: Pete Batard <pete@akeo.ie>
This makes the apps/*.efi files available for download and testing after each build.
Also clean up the GitHub Actions workflows a little bit.

Signed-off-by: Pete Batard <pete@akeo.ie>
Use OVMF/AVMF and the UEFI Shell in a QEMU environment to validate that the
UEFI executables we build do run and produce the expected output.

The tests to run are flexibly defined is test_list.txt and can be expanded.

At this stage, only x64, ia32, aa64 and arm are run, which rely on the UEFI
firmwares found at https://github.com/pbatard/EfiFs/tree/gh-pages/AAVMF and
https://github.com/pbatard/EfiFs/tree/gh-pages/OVMF as well as Shell builds
from https://github.com/pbatard/UEFI-Shell.

Signed-off-by: Pete Batard <pete@akeo.ie>
@ncroxon ncroxon merged commit 7440e5e into ncroxon:master May 20, 2024
30 checks passed
@pbatard pbatard deleted the Resolve-Conflicts branch May 21, 2024 17:40
@gmbr3 gmbr3 added the ci/cd Bugs and changes to CI/CD label Nov 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci/cd Bugs and changes to CI/CD
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants