Reboot check test python version (New)

[tomli380576]

Description

This PR has the 2 following major changes:

Migrating the original reboot_check_test.sh to python

The new script splits the original script into 2 tester classes and 1 test function

• DeviceInfoCollector class compares device info gathered after reboots against the ground truth collected at the beginning. For each reboot, it collects output from iw, lsub, lspci, and ls /sys/class/drm.

• FwtsTester class simply calls fwts and runs the sleep_test_log_check.py script that lives in the same directory

• get_failed_services returns a list of failed system services

This migration does not introduce any changes to the CLI flags except the new -g flag for the renderer test.

The corresponding test plan, boot.pxu, is updated in #1339 with the new file name and the new
-g flag.

New hardware renderer test

A check for hardware renderer in warm/cold boot tests is implemented. This checks if the DUT is always using hardware rendering across reboots.

• HardwareRendererTester contains the test implementation. This test only runs if:
  • Checkbox is running on a desktop image
  • A display is connected (or a display emulator)
• This test is only applicable to machines running the ubuntu desktop image. We will automatically skip this test if server/ubuntu core image is detected.

Resolved issues

Launchpad 2051627

Documentation

python3 reboot_check_test.py -h shows all the flags and what they mean

usage: Reboot tests [-h] [-d OUTPUT_DIRECTORY] [-c COMPARISON_DIRECTORY] [-s] [-f] [-g]

Collects device info and compares them across reboots

options:
  -h, --help            show this help message and exit
  -d OUTPUT_DIRECTORY, --dump-to OUTPUT_DIRECTORY
                        Device info-dumps will be written here
  -c COMPARISON_DIRECTORY, --compare-to COMPARISON_DIRECTORY
                        Directory of ground-truth for device info comparison
  -s, --service-check   If specified, check if all system services are running
  -f, --fwts-check      If specified, look for fwts log errors
  -g, --graphics        If specified, check if hardware rendering is being used

Tests

On 22.04 server image: https://certification.canonical.com/hardware/202405-34022/submission/381261/

On a laptop with 22.04 desktop image: https://certification.canonical.com/hardware/202406-34135/submission/381283/

Run unit tests: checkbox/providers/base/manage.py test -u

[codecov]

Codecov Report

Attention: Patch coverage is 92.15686% with 12 lines in your changes missing coverage. Please review.

Project coverage is 46.97%. Comparing base (6f10cea) to head (d636da3).
Report is 116 commits behind head on main.

Files with missing lines	Patch %	Lines
providers/base/bin/reboot_check_test.py	92.15%	8 Missing and 4 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1338      +/-   ##
==========================================
+ Coverage   44.82%   46.97%   +2.15%     
==========================================
  Files         359      361       +2     
  Lines       38834    39334     +500     
  Branches     6585     6663      +78     
==========================================
+ Hits        17409    18479    +1070     
+ Misses      20756    20157     -599     
- Partials      669      698      +29     

Flag	Coverage Δ
provider-base	24.70% <92.15%> (+6.13%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Hook25]

Very good, please strip out that custom command runner, I think it is hurting the whole file a bit if you dont have a very good reason to use it.

(Also, sorry this took me so much time to review, hope we can land this soon)

[Hook25]

This looks ok, please remember that instead of using check=True you can almost always use check_(output/call) calls. Also, you don't need .stdout.decode(), just use the universal_newlines=True. Either way, feel free to land this