Store rendered control files

[ggbecker]

Description:

• Store rendered (resolved) control files when building the content.
• The files are stored under build/$product/controls/
• This PR also adds support for reusing controls across different policies.

Rationale:

• Files can be used to display a policy and the relationship of controls with rules in the project.

Note: Seeking feedback and help on how to implement this properly.

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[github-actions]

Start a new ephemeral environment with changes proposed in this pull request:

Fedora Environment

Oracle Linux 8 Environment

[jan-cerny]

Files can be used to display a policy and the relationship of controls with rules in the project.

Questions: What is the use-case for this? In what way do you want to show the relationship of controls and rules? What is the benefit of viewing "resolved controls" instead of the original source or the HTML output online?

[ggbecker]

Files can be used to display a policy and the relationship of controls with rules in the project.

Questions: What is the use-case for this? In what way do you want to show the relationship of controls and rules? What is the benefit of viewing "resolved controls" instead of the original source or the HTML output online?

This is a requirement we have been discussing for OpenShift products. They want to have an ability to ship the contents of a resolved control to customers. And the controls file resolved will contain only things that are applicable to a given product and any jinja macros (although there is only one in the project in controls file) will also be resolved and the file can be interpreted as pure yaml file.

These files will be bundled whenever the content is built by OpenShift for example and they need to be "versioned" opposed the the HTML output online which represents latest upstream code always.

[jan-cerny]

@ggbecker Thanks for explanation! If it needs to be in a YAML format I would prefer a "nice" YAML, with no build system internal details and no empty fields, so that it would be better readable.

[ggbecker]

@ggbecker Thanks for explanation! If it needs to be in a YAML format I would prefer a "nice" YAML, with no build system internal details and no empty fields, so that it would be better readable.

Agree with the part that no build system internal details, but the empty fields might be useful in the future depending on the needs of a policy. This PR definitely needs more iterations.

It doesn't need to be a "nice" YAML since this will be consumed most likely by Compliance Operator which will parse the file and produce something else for example. So the empty fields are not really a problem.

[ggbecker]

patch.txt

[matejak]

It turned out that this feature interferes with the nested controls feature, when we define (sub)controls inside another controls, and the information is then discarded, so the compiled controls don't have it - they just have the expected rule selections without knowing how those got there.
I have handled that, so subcontrols are formally recorded, and I have extended the mechanics to allow not only definition of subcontrols, but also reference of controls from other policies, all handled the same way.
At this time, the PR contains "ugly" code, but it also contains tests of this functionality.

[ggbecker]

/packit retest-failed

[ggbecker]

It looks good to me, but I think we should provide some examples of how to use the nested controls feature and how people can benefit from it to avoid code duplication.

[ggbecker]

LGTM, thanks for the follow up work @matejak .

[jan-cerny]

@ggbecker What about the CodeClimate problems? And do you need a 3rd person to a review or do you plan to be the assignee?

[matejak]

So I believe that as Becker and me got our hands dirty with this one, we need e.g. you, @jan-cerny as the 3rd person.

Anyway, I wouldn't document the feature further than it is proposed in the README, because we don't have a meaningful and illustrative use case. Moreover, there are open questions about inheritance of attributes that will influence the usefulness of sub-controls in practice.
Right now, the PR is about policy being able to save its compiled version into a file, and about controls remembering their sub-controls, and being able to load themselves consistently, taking those sub-controls into the account.
Ability to include control references also in control definitions is a side-effect of compilation, and I propose to explore this new capability in subsequent PRs, along with documentation.

Regarding Code Climate, I think that findings are false positives, because the explicit pass makes the code more understandable, so I propose to waive it.

[jan-cerny]

I have tried to reference some controls from other controls and I have watched that the rule selections propageted.

[jan-cerny]

CI fail on Rawhide is caused by OpenSCAP and isn't related to the contents of this PR, see OpenSCAP/openscap#1995.

[codeclimate]

Code Climate has analyzed commit f0506b9 and detected 1 issue on this pull request.

Here's the issue category breakdown:

Category	Count
Bug Risk	1

The test coverage on the diff in this pull request is 97.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 53.4% (0.5% change).

View more on Code Climate.