Include test scenario for multiple partitions

[marcusburghardt]

Description:

An issue in the Ansible remediation was fixed by #11174 and later improved by #11263 but no test scenario was included to test this condition where there are privileged commands in different partitions.

Rationale:

• Better test scenarios coverage
• Related to RHEL-25828

Review Hints:

automatus tests should be enough.
It could also be checked the reports generated by automatus tests in the reports folder.

[github-actions]

Start a new ephemeral environment with changes proposed in this pull request:

rhel8 (from CTF) Environment (using Fedora as testing environment)

Fedora Testing Environment

Oracle Linux 8 Environment

[github-actions]

🤖 A k8s content image for this PR is available at:
ghcr.io/complianceascode/k8scontent:11950
This image was built from commit: e3370aa

Click here to see how to deploy it

If you alread have Compliance Operator deployed:
utils/build_ds_container.py -i ghcr.io/complianceascode/k8scontent:11950

Otherwise deploy the content and operator together by checking out ComplianceAsCode/compliance-operator and:
CONTENT_IMAGE=ghcr.io/complianceascode/k8scontent:11950 make deploy-local

[codeclimate]

Code Climate has analyzed commit e3370aa and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 59.4% (0.0% change).

View more on Code Climate.

[Mab879]

This new scenario passes (see below) but the existing ones don't seem to. Is this what are you seeing as well?

$ ./automatus.py rule --libvirt qemu:///system automatus_rhel9_4 --product rhel9 --datastream ../build/ssg-rhel9-ds.xml --scenarios augenrules_rules_multiple_partitions --remediate-using bash audit_rules_privileged_commands 
Setting console output to log level INFO
INFO - The base image option has not been specified, choosing libvirt-based test environment.
INFO - Logging into /home/mburket/Developer/ComplianceAsCode/content/tests/logs/rule-custom-2024-05-06-1306/test_suite.log
INFO - xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands
INFO - Script augenrules_rules_multiple_partitions.fail.sh using profile (all) OK

[marcusburghardt]

This new scenario passes (see below) but the existing ones don't seem to. Is this what are you seeing as well?

$ ./automatus.py rule --libvirt qemu:///system automatus_rhel9_4 --product rhel9 --datastream ../build/ssg-rhel9-ds.xml --scenarios augenrules_rules_multiple_partitions --remediate-using bash audit_rules_privileged_commands 
Setting console output to log level INFO
INFO - The base image option has not been specified, choosing libvirt-based test environment.
INFO - Logging into /home/mburket/Developer/ComplianceAsCode/content/tests/logs/rule-custom-2024-05-06-1306/test_suite.log
INFO - xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands
INFO - Script augenrules_rules_multiple_partitions.fail.sh using profile (all) OK

Here they passed fine:

Mon May  6 06:12:02 PM CEST 2024 - rhel9 - audit_rules_privileged_commands - bash VM
./tests/automatus.py rule --libvirt qemu:///session rhel9 --datastream build/ssg-rhel9-ds.xml --dontclean --remediate-using bash audit_rules_privileged_commands
Setting console output to log level INFO
INFO - The base image option has not been specified, choosing libvirt-based test environment.
INFO - xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands
INFO - Script auditctl_default.fail.sh using profile (all) OK
INFO - Script auditctl_missing_rule.fail.sh using profile (all) OK
INFO - Script auditctl_one_rule.fail.sh using profile (all) OK
INFO - Script auditctl_rules_configured.pass.sh using profile (all) OK
INFO - Script auditctl_rules_without_perm_x.pass.sh using profile (all) OK
INFO - Script augenrules_default.fail.sh using profile (all) OK
INFO - Script augenrules_duplicated.fail.sh using profile (all) OK
INFO - Script augenrules_extra_rules_configured.pass.sh using profile (all) OK
INFO - Script augenrules_missing_rule.fail.sh using profile (all) OK
INFO - Script augenrules_one_rule.fail.sh using profile (all) OK
INFO - Script augenrules_rules_configured.pass.sh using profile (all) OK
INFO - Script augenrules_rules_configured_mixed_keys.pass.sh using profile (all) OK
INFO - Script augenrules_rules_ignore_dracut_tmp.pass.sh using profile (all) OK
INFO - Script augenrules_rules_without_perm_x.pass.sh using profile (all) OK
INFO - Script augenrules_two_rules_mixed_keys.fail.sh using profile (all) OK
INFO - Script augenrules_two_rules_sep_files.fail.sh using profile (all) OK
INFO - Script rules_with_own_key.pass.sh using profile (all) OK
INFO - Script augenrules_rules_multiple_partitions.fail.sh using profile (all) OK

real	9m8.102s
user	0m43.467s
sys	0m6.657s

Mon May  6 06:21:10 PM CEST 2024 - rhel9 - audit_rules_privileged_commands - ansible VM
./tests/automatus.py rule --libvirt qemu:///session rhel9 --datastream build/ssg-rhel9-ds.xml --dontclean --remediate-using ansible audit_rules_privileged_commands
Setting console output to log level INFO
INFO - The base image option has not been specified, choosing libvirt-based test environment.
INFO - xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands
INFO - Script auditctl_default.fail.sh using profile (all) OK
INFO - Script auditctl_missing_rule.fail.sh using profile (all) OK
INFO - Script auditctl_one_rule.fail.sh using profile (all) OK
INFO - Script auditctl_rules_configured.pass.sh using profile (all) OK
INFO - Script auditctl_rules_without_perm_x.pass.sh using profile (all) OK
INFO - Script augenrules_default.fail.sh using profile (all) OK
INFO - Script augenrules_duplicated.fail.sh using profile (all) OK
INFO - Script augenrules_extra_rules_configured.pass.sh using profile (all) OK
INFO - Script augenrules_missing_rule.fail.sh using profile (all) OK
INFO - Script augenrules_one_rule.fail.sh using profile (all) OK
INFO - Script augenrules_rules_configured.pass.sh using profile (all) OK
INFO - Script augenrules_rules_configured_mixed_keys.pass.sh using profile (all) OK
INFO - Script augenrules_rules_ignore_dracut_tmp.pass.sh using profile (all) OK
INFO - Script augenrules_rules_without_perm_x.pass.sh using profile (all) OK
INFO - Script augenrules_two_rules_mixed_keys.fail.sh using profile (all) OK
INFO - Script augenrules_two_rules_sep_files.fail.sh using profile (all) OK
INFO - Script rules_with_own_key.pass.sh using profile (all) OK
INFO - Script augenrules_rules_multiple_partitions.fail.sh using profile (all) OK

real	12m24.319s
user	1m25.253s
sys	0m15.055s

[Mab879]

The tests pass on my RHEL 8 box.

$ ./automatus.py rule --datastream ../build/ssg-rhel8-ds.xml --remediate-using oscap  --libvirt qemu:///system automatus_rhel_8_6 audit_rules_privileged_commands
Setting console output to log level INFO
INFO - The base image option has not been specified, choosing libvirt-based test environment.
INFO - Logging into /home/mburket/Developer/ComplianceAsCode/content/tests/logs/rule-custom-2024-05-06-1643/test_suite.log
INFO - xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands
INFO - Script auditctl_default.fail.sh using profile (all) OK
INFO - Script auditctl_missing_rule.fail.sh using profile (all) OK
INFO - Script auditctl_one_rule.fail.sh using profile (all) OK
INFO - Script auditctl_rules_configured.pass.sh using profile (all) OK
INFO - Script auditctl_rules_without_perm_x.pass.sh using profile (all) OK
INFO - Script augenrules_default.fail.sh using profile (all) OK
INFO - Script augenrules_duplicated.fail.sh using profile (all) OK
INFO - Script augenrules_extra_rules_configured.pass.sh using profile (all) OK
INFO - Script augenrules_missing_rule.fail.sh using profile (all) OK
INFO - Script augenrules_one_rule.fail.sh using profile (all) OK

INFO - Script augenrules_rules_configured.pass.sh using profile (all) OK
INFO - Script augenrules_rules_configured_mixed_keys.pass.sh using profile (all) OK
INFO - Script augenrules_rules_ignore_dracut_tmp.pass.sh using profile (all) OK
INFO - Script augenrules_rules_without_perm_x.pass.sh using profile (all) OK
INFO - Script augenrules_two_rules_mixed_keys.fail.sh using profile (all) OK
INFO - Script augenrules_two_rules_sep_files.fail.sh using profile (all) OK
INFO - Script rules_with_own_key.pass.sh using profile (all) OK
INFO - Script augenrules_rules_multiple_partitions.fail.sh using profile (all) OK

But on my 9.4 box (similar issue with Ansible).

$ ./automatus.py rule --datastream ../build/ssg-rhel9-ds.xml --libvirt qemu:///system automatus_rhel9_4 audit_rules_privileged_commands
Setting console output to log level INFO
INFO - The base image option has not been specified, choosing libvirt-based test environment.
INFO - Logging into /home/mburket/Developer/ComplianceAsCode/content/tests/logs/rule-custom-2024-05-06-1648/test_suite.log
INFO - xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands
INFO - Script auditctl_default.fail.sh using profile (all) OK
INFO - Script auditctl_missing_rule.fail.sh using profile (all) OK
ERROR - Rule evaluation resulted in error, instead of expected fixed during remediation stage 
ERROR - The remediation failed for rule 'xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands'.
INFO - Script auditctl_one_rule.fail.sh using profile (all) OK
ERROR - Script auditctl_rules_configured.pass.sh using profile (all) found issue:
ERROR - Rule evaluation resulted in fail, instead of expected pass during initial stage 
ERROR - The initial scan failed for rule 'xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands'.
ERROR - Script auditctl_rules_without_perm_x.pass.sh using profile (all) found issue:
ERROR - Rule evaluation resulted in fail, instead of expected pass during initial stage 
ERROR - The initial scan failed for rule 'xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands'.
INFO - Script augenrules_default.fail.sh using profile (all) OK
INFO - Script augenrules_duplicated.fail.sh using profile (all) OK
ERROR - Script augenrules_extra_rules_configured.pass.sh using profile (all) found issue:
ERROR - Rule evaluation resulted in fail, instead of expected pass during initial stage 
ERROR - The initial scan failed for rule 'xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands'.
INFO - Script augenrules_missing_rule.fail.sh using profile (all) OK
ERROR - Rule evaluation resulted in error, instead of expected fixed during remediation stage 
ERROR - The remediation failed for rule 'xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands'.
INFO - Script augenrules_one_rule.fail.sh using profile (all) OK
ERROR - Script augenrules_rules_configured.pass.sh using profile (all) found issue:
ERROR - Rule evaluation resulted in fail, instead of expected pass during initial stage 
ERROR - The initial scan failed for rule 'xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands'.
ERROR - Script augenrules_rules_configured_mixed_keys.pass.sh using profile (all) found issue:
ERROR - Rule evaluation resulted in fail, instead of expected pass during initial stage 
ERROR - The initial scan failed for rule 'xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands'.
ERROR - Script augenrules_rules_ignore_dracut_tmp.pass.sh using profile (all) found issue:
ERROR - Rule evaluation resulted in fail, instead of expected pass during initial stage 
ERROR - The initial scan failed for rule 'xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands'.
ERROR - Script augenrules_rules_without_perm_x.pass.sh using profile (all) found issue:
ERROR - Rule evaluation resulted in fail, instead of expected pass during initial stage 
ERROR - The initial scan failed for rule 'xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands'.
INFO - Script augenrules_two_rules_mixed_keys.fail.sh using profile (all) OK
ERROR - Rule evaluation resulted in error, instead of expected fixed during remediation stage 
ERROR - The remediation failed for rule 'xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands'.
INFO - Script augenrules_two_rules_sep_files.fail.sh using profile (all) OK
ERROR - Rule evaluation resulted in error, instead of expected fixed during remediation stage 
ERROR - The remediation failed for rule 'xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands'.
ERROR - Script rules_with_own_key.pass.sh using profile (all) found issue:
ERROR - Rule evaluation resulted in fail, instead of expected pass during initial stage 
ERROR - The initial scan failed for rule 'xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands'.
INFO - Script augenrules_rules_multiple_partitions.fail.sh using profile (all) OK

So I updated another RHEL 9.4 box and everything seems fine there:

$ ./automatus.py rule --datastream ../build/ssg-rhel9-ds.xml --libvirt qemu:///system automatus_rhel9_4_1 --remediate-using ansible audit_rules_privileged_commands
Setting console output to log level INFO
INFO - The base image option has not been specified, choosing libvirt-based test environment.
INFO - Logging into /home/mburket/Developer/ComplianceAsCode/content/tests/logs/rule-custom-2024-05-06-1710/test_suite.log
INFO - xccdf_org.ssgproject.content_rule_audit_rules_privileged_commands
INFO - Script auditctl_default.fail.sh using profile (all) OK
INFO - Script auditctl_missing_rule.fail.sh using profile (all) OK
INFO - Script auditctl_one_rule.fail.sh using profile (all) OK
INFO - Script auditctl_rules_configured.pass.sh using profile (all) OK
INFO - Script auditctl_rules_without_perm_x.pass.sh using profile (all) OK
INFO - Script augenrules_default.fail.sh using profile (all) OK
INFO - Script augenrules_duplicated.fail.sh using profile (all) OK
INFO - Script augenrules_extra_rules_configured.pass.sh using profile (all) OK
INFO - Script augenrules_missing_rule.fail.sh using profile (all) OK
INFO - Script augenrules_one_rule.fail.sh using profile (all) OK
INFO - Script augenrules_rules_configured.pass.sh using profile (all) OK
INFO - Script augenrules_rules_configured_mixed_keys.pass.sh using profile (all) OK
INFO - Script augenrules_rules_ignore_dracut_tmp.pass.sh using profile (all) OK
INFO - Script augenrules_rules_without_perm_x.pass.sh using profile (all) OK
INFO - Script augenrules_two_rules_mixed_keys.fail.sh using profile (all) OK
INFO - Script augenrules_two_rules_sep_files.fail.sh using profile (all) OK
INFO - Script rules_with_own_key.pass.sh using profile (all) OK

So there seems to be something up that one box. I'm willing to merge this.

[Mab879]

LGTM.

Thanks!