Test and use rules to components mapping #10693

jan-cerny · 2023-06-07T09:41:37Z

Description:

This PR is a follow up on PR #10609 and has been initially created by extracting some commits from there.

The PR introduces a new module ssg.components which is a basis for future use of the component data in our project. A simple unit test for this module is added as well.

The PR introduces the test that performs some basic checks that will help keep the component data consistent. This test tests the following facts:

presence of rule in a group that is known to map to a component
usage of templates in the rule that allow us to easily infer the component, eg. package_installed, package_removed, service_enabled
usage of platforms in the given rule that allow us to easily infer the component, eg. package platform.

The PR enforces component assignment for all rules in the linux_os/guide benchmark. That will prevent people from introducing a new rule without mapping it to a component.

For more details, please read the commit messages of every commit.

Rationale:

These changes will ensure the component data consistency. Mandatory component assignment will help content authors think about component outreach of new rules. The tests will help people assign the new rules to correct components.

Review Hints:

Remove some rules from some component, eg. remove the rule package_fapolicyd_installed from components/fapolicyd.yml and run ctest --verbose -R components to see how the test reacts. Also, try to build the content to see an error.

github-actions · 2023-06-07T09:51:12Z

Start a new ephemeral environment with changes proposed in this pull request:

Fedora Environment

Oracle Linux 8 Environment

Mab879

Thanks for the PR!. Please see my comments.

docs/manual/developer/03_creating_content.md

Mab879 · 2023-06-07T20:48:53Z

ssg/build_yaml.py


    def _process_values(self):
        for value_yaml in self.value_files:
            value = Value.from_yaml(value_yaml, self.env_yaml)
            self.all_values[value.id_] = value
            self.loaded_group.add_value(value)

+    def __process_rule(self, rule):


Any reason for the double underscore?

tests/CMakeLists.txt

Mab879 · 2023-06-07T20:57:50Z

tests/test_components.py

+    if template_name in template_to_component:
+        component = template_to_component[template_name]
+        reason = (
+            "all rules using template '%s' should be assigned to component "


Suggested change

"all rules using template '%s' should be assigned to component "

"all rules using template '%s' must be assigned to component "

I think that if this this behavior cannot be overridden using the world "must" or "shall" would be better. This applies to all the reasons in this file.

matejak · 2023-06-08T09:50:42Z

ssg/build_yaml.py


    def _process_values(self):
        for value_yaml in self.value_files:
            value = Value.from_yaml(value_yaml, self.env_yaml)
            self.all_values[value.id_] = value
            self.loaded_group.add_value(value)

+    def __process_rule(self, rule):
+        if self.rule_to_components is not None and rule.id_ not in self.rule_to_components:


This is motivated mainly by feedback to contributors that forget to assign components, and this case is not caught by tests. A build failure provides an early feedback, and the contributor will have to think of components early on.

However, there are disadvantages as well:

We may end up with an "other" component, but its rules would not share reasons to change, which is the main driving force behind components.

Although there is an existing division by package related to files that the check/remediation operates on, this structure again may not be beneficial.

If a rule should be assigned to more components, this error alone won't assure that.

We may end up with an "other" component, but its rules would not share reasons to change, which is the main driving force behind components.

We can do it a more clever way, for rules that don't belong to any component we can put them to some predefined categories that aren't neccesary a RHEL component but make sense for the assessment, for example "filesystem", "users", "crypto".

Although there is an existing division by package related to files that the check/remediation operates on, this structure again may not be beneficial.

It's not a problem of this check, it's the problem of the way we did the initial assignment that is already merged to master.

If a rule should be assigned to more components, this error alone won't assure that.

Yes, but it would at least assure that the rule has at least one component. Once the reviewer of the PR will see that a component is modified, he might think about possible other components.

If a rule isn't part of any component, the build will fail. The enforcement will be efficient for the `linux_os` benchmark.

Also introduces a `ssg.components` module to the project which can be used later in other tools and build system to work with component files data. This commit adds a simple test case `test_components.py` which tests that - components don't map to rules that don't exist - all rules are mapped to at least 1 component - the rule is assigned to component according to templates or template parameters

This test checks if a rule is mapped to component if it uses a package platform or other platforms that can be easily used to determine to which component the rule should belong to.

This commit extracts method `__process_rule` to reduce the code complexity of the original method, as suggested by CodeClimate.

This aims to address the CodeClimate code complexity problem in the `main()`.

CodeClimate emits a warning if there is too many return statements in a single function.

This change reduces code duplication.

Products can decide to use components or they can define their own set of components. Products specify a path to the directory with component files by the `components_root` key in the `product.yml`. This allows products like OCP to use components.

Extract function get_rule_to_components_mapping and move it to the ssg.components module in order to simplify the BuildLoader class.

The reason is to avoid code duplication.

This change will add the mapping of a rule to component to the resolved rules. Developers that build the content will be able to see all components that the given rule is assigned to in the resolved rule yamls.

This will prevent everyone from running the script without its parameters.

This will allow developers to add additional tests for component mapping when a component assignement will be depending on the template used in the rule.

If this this behavior cannot be overridden using the world "must" or "shall" would be better. This applies to all the reasons in this file.

jan-cerny · 2023-06-13T09:45:11Z

I have rebased this PR on the top of the latest upstream master branch. I have updated example product.yml in Developer's guide. I have replaced double underscore by a single underscore. I have add components to resolved rules. I have made the test_components.py paremeters required and I have created testing plugins for the templates. I have also replaced must by should.

Addressing: loader.process_directory_tree(subdir) File "/__w/content/content/ssg/build_yaml.py", line 1265, in process_directory_tree self._load_group_process_and_recurse(start_dir) File "/__w/content/content/ssg/build_yaml.py", line 1259, in _load_group_process_and_recurse self._process_rules() File "/__w/content/content/ssg/build_yaml.py", line 1383, in _process_rules if not self._process_rule(rule): File "/__w/content/content/ssg/build_yaml.py", line 1372, in _process_rule rule.components = self.rule_to_components[rule.id_] TypeError: 'NoneType' object has no attribute '__getitem__'

jan-cerny · 2023-06-13T10:39:11Z

I have fixed a traceback a traceback and I have removed a space.

Avoid code duplication in ssg/components.py. The mapping functions are similar, they differ only in attribute and the type of the dicitionary values. By unifying the code into a single private function we can avoid this duplication.

jan-cerny · 2023-06-13T15:21:31Z

I have fixed the CodeClimate issue.

Mab879

Thanks for the PR! One minor nitpick.

Mab879 · 2023-06-13T15:03:03Z

ssg/components.py

+    return components
+
+
+def __reverse_mapping(components, attribute):


Double underscore here as well.

Mab879 · 2023-06-14T01:40:57Z

/packit retest-failed

jan-cerny · 2023-06-14T12:19:24Z

I have changed the underscore

codeclimate · 2023-06-14T12:58:09Z

Code Climate has analyzed commit 5d20bde and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 59.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 52.9% (0.1% change).

View more on Code Climate.

Mab879 · 2023-06-15T13:26:34Z

/packit build

Mab879

Thanks for the PR! This will be great addition to the project!

jan-cerny added enhancement General enhancements to the project. Highlight This PR/Issue should make it to the featured changelog. labels Jun 7, 2023

jan-cerny added this to the 0.1.69 milestone Jun 7, 2023

jan-cerny requested a review from a team as a code owner June 7, 2023 09:41

jan-cerny mentioned this pull request Jun 7, 2023

Test and use rules to components mapping #10638

Closed

jan-cerny force-pushed the components3 branch 2 times, most recently from 21aa810 to 5ca7909 Compare June 7, 2023 12:04

Mab879 self-assigned this Jun 7, 2023

Mab879 requested changes Jun 7, 2023

View reviewed changes

matejak reviewed Jun 8, 2023

View reviewed changes

jan-cerny mentioned this pull request Jun 12, 2023

Render components to HTML #10709

Merged

jan-cerny added 18 commits June 13, 2023 08:51

Assign network_implement_access_control to firewalld component

766503d

Enforce component mapping

4ef6a60

If a rule isn't part of any component, the build will fail. The enforcement will be efficient for the `linux_os` benchmark.

Add test for component based on platforms

06c2448

This test checks if a rule is mapped to component if it uses a package platform or other platforms that can be easily used to determine to which component the rule should belong to.

Add a test for components based on groups

699a453

remove env_yaml

e2fcfbe

Add a unit test for the ssg.components module

5569fc7

Extract method

32aa53b

This commit extracts method `__process_rule` to reduce the code complexity of the original method, as suggested by CodeClimate.

Fix CodeClimate PEP8 problem

06726fb

Reduce complexity of main function

1487959

This aims to address the CodeClimate code complexity problem in the `main()`.

Reduce amount of parameters

1c21ea0

Avoid many return statements

caeb48d

CodeClimate emits a warning if there is too many return statements in a single function.

Use ssg.components module in _load_components

f442dd4

This change reduces code duplication.

Extract function

3a0b1eb

Extract function get_rule_to_components_mapping and move it to the ssg.components module in order to simplify the BuildLoader class.

Simplify function test_platform

a6c8c87

The reason is to avoid code duplication.

Assign rule package_iptables-services_removed to a component

6211e68

Update example product.yml in Developer's guide

aef1955

jan-cerny added 4 commits June 13, 2023 09:11

Replace double underscore by single underscore

daa860f

Add components to resolved rules

4c97b55

This change will add the mapping of a rule to component to the resolved rules. Developers that build the content will be able to see all components that the given rule is assigned to in the resolved rule yamls.

Make the parameters required

9649fc9

This will prevent everyone from running the script without its parameters.

Create template testing plugins

052e0c1

This will allow developers to add additional tests for component mapping when a component assignement will be depending on the template used in the rule.

jan-cerny force-pushed the components3 branch from 5ca7909 to 052e0c1 Compare June 13, 2023 09:37

Replace must by should

c7fbf85

If this this behavior cannot be overridden using the world "must" or "shall" would be better. This applies to all the reasons in this file.

jan-cerny added 2 commits June 13, 2023 11:48

Remove a space

0156763

Fix CodeClimate problem

4cdee0c

Avoid code duplication in ssg/components.py. The mapping functions are similar, they differ only in attribute and the type of the dicitionary values. By unifying the code into a single private function we can avoid this duplication.

Mab879 requested changes Jun 14, 2023

View reviewed changes

ssg/components.py Outdated

return components

def __reverse_mapping(components, attribute):

Copy link

Member

Mab879 Jun 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Double underscore here as well.

Change double underscore to a single underscore

5d20bde

Mab879 approved these changes Jun 15, 2023

View reviewed changes

Mab879 merged commit 8e35aea into ComplianceAsCode:master Jun 15, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test and use rules to components mapping #10693

Test and use rules to components mapping #10693

jan-cerny commented Jun 7, 2023

github-actions bot commented Jun 7, 2023

Mab879 left a comment

Mab879 Jun 7, 2023

Mab879 Jun 7, 2023

matejak Jun 8, 2023

jan-cerny Jun 14, 2023

jan-cerny commented Jun 13, 2023

jan-cerny commented Jun 13, 2023

jan-cerny commented Jun 13, 2023

Mab879 left a comment

Mab879 Jun 13, 2023

Mab879 commented Jun 14, 2023

jan-cerny commented Jun 14, 2023

codeclimate bot commented Jun 14, 2023

Mab879 commented Jun 15, 2023

Mab879 left a comment

	"all rules using template '%s' should be assigned to component "
	"all rules using template '%s' must be assigned to component "

		return components


		def __reverse_mapping(components, attribute):

Test and use rules to components mapping #10693

Test and use rules to components mapping #10693

Conversation

jan-cerny commented Jun 7, 2023

Description:

Rationale:

Review Hints:

github-actions bot commented Jun 7, 2023

Mab879 left a comment

Choose a reason for hiding this comment

Mab879 Jun 7, 2023

Choose a reason for hiding this comment

Mab879 Jun 7, 2023

Choose a reason for hiding this comment

matejak Jun 8, 2023

Choose a reason for hiding this comment

jan-cerny Jun 14, 2023

Choose a reason for hiding this comment

jan-cerny commented Jun 13, 2023

jan-cerny commented Jun 13, 2023

jan-cerny commented Jun 13, 2023

Mab879 left a comment

Choose a reason for hiding this comment

Mab879 Jun 13, 2023

Choose a reason for hiding this comment

Mab879 commented Jun 14, 2023

jan-cerny commented Jun 14, 2023

codeclimate bot commented Jun 14, 2023

Mab879 commented Jun 15, 2023

Mab879 left a comment

Choose a reason for hiding this comment