CRAYSAT-1937: Add the first functional test for SAT

[haasken-hpe]

Summary and Scope

Add the first functional test for the System Admin Toolkit (SAT). This is a proof of concept for future tests for SAT.

This test simply executes the sat --version command on the NCN and verifies that the output matches the value in the file used to set the version.

SAT tests are implemented in Python using the unittest standard library package. This provides an easy way to structure tests, add set up and tear down logic for each test, and run the tests from a script. The Python code for the SAT tests is all under a sat_functional subpackage, and individual subpackages and modules beneath that subpackage will implement tests for specific SAT functionality. Tests will be organized by the sat command they are testing.

The __main__.py in the sat_testing module simply passes along the arguments, which should be a list of relative test paths, to the unittest.TextTestRunner, which then runs the tests.

SAT tests are added in Gossfiles. Currently a single test file for SAT tests, goss-sat-version.yaml, is created. A test suite is created in the goss file ncn-sat-functional-tests.yaml. This test suite is limited to run on only ncn-m001. It will eventually include all SAT functional tests.

Issues and Related PRs

• Resolves CRAYSAT-1937

Testing

Tested on:

• drax

Test description:

Downloaded the RPM built on Jenkins to drax. Extracted the RPM using rpm2cpio $RPM_FILE | cpio -idvmu. Modified the path to the Python virtual environment in the test_sat_functional script in the bin directory of the virtualenv to match the extraction path, set GOSS_BASE to the extracted Goss base directory, and then ran the Goss test file and test suite using goss -g $GOSS_FILE validate.

Risks and Mitigations

Pretty low risk. It's possible the test could not work as I'm expecting in the vshasta jenkins jobs, but if that's the case, we can fix it. I'm not sure how to test beyond what I described above.

Pull Request Checklist

• Version number(s) incremented, if applicable
• Copyrights updated
• License file intact
• Target branch correct
• CHANGELOG.md updated
• Testing is appropriate and complete, if applicable
• HPC Product Announcement prepared, if applicable

[haasken-hpe]

Test output:

ncn-m001:~/haasken/opt/cray/tests/install/ncn/suites # goss -g ncn-sat-functional-tests.yaml validate
..

Total Duration: 1.796s
Count: 2, Failed: 0, Skipped: 0

And the detailed logs for this test show the individual Python test method being run via unittest:

ncn-m001:/opt/cray/tests/install/logs/sat_version # cat 20241212_222505_096447746_3924799.log
log_run argument(s): -l sat_version
Executable: '/root/haasken/opt/cray/tests/install/ncn/scripts/python/sat_functional'
1 argument(s) to executable: 'version.test_version'

## 20241212_222505_109987914 ##                      executable starting ##
###########################################################################
test_version_command (csm_testing.tests.sat_functional.version.test_version.TestVersion)
Test that `sat --version` returns the semantic version. ... ok

----------------------------------------------------------------------
Ran 1 test in 1.689s

OK
###########################################################################
## 20241212_222506_875399675 ##       executable completed (exit code=0) ##

[haasken-hpe]

@rajeshranganathan-hpe, @Pankhuri-Rajesh, @Srinivas-Anand-HPE, I added you all for awareness per our meeting today. We'll keep you in the loop on further SAT automated testing PRs as well.

[haasken-hpe]

/backport release/1.7

[github-actions]

Backporting into branch release/1.7 was successful. New PR: #654