Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Include metrics for rules and variables selected in Controls #11128

Merged
merged 4 commits into from
Sep 21, 2023
Merged

Include metrics for rules and variables selected in Controls #11128

merged 4 commits into from
Sep 21, 2023

Conversation

marcusburghardt
Copy link
Member

Description:

Besides the status of each Policy, now controleval_metrics.py is also exporting metrics about rules and variables selected in each Policy.

For this, the ssg/controls.py was slightly incremented and the new property was used in two new functions of controleval.py. These functions are finally consumed by controleval_metrics.py.

Rationale:

Better tracking of rules and variables included in control files.

Review Hints:

e.g.:
utils/controleval_metrics.py prometheus -p rhel9
Then check the new metrics in the output while the former metrics are intact.

You can also check the output of controleval.py stats. e.g.:
utils/controleval.py stats -i cis_rhel9 -p rhel9 -l l1_server

Then observe the last lines of the output. e.g.:

Rules and Variables in cis_rhel9 - l1_server:
279 rules are selected
53 variables are explicitly defined

@marcusburghardt
Include rules list for each control
25ad09c 
The controls.py module was updated to also initialize a list of rules
included in each control. This make it easier to other tools analyse
rules included in controls.
@marcusburghardt
Extend controleval.py to show rules and variables
4d0ba70 
Now the controleval.py stats will also show the number of rules and
variables included in each control, under "rules" property.
@marcusburghardt
Export controls content metrics to Prometheus
bc9dbf3 
Besides the status of each Policy it is now also exporting metrics about
rules and variables selected in each Policy.
@marcusburghardt marcusburghardt added enhancement General enhancements to the project. Documentation Update in project documentation. labels Sep 20, 2023
@marcusburghardt marcusburghardt added this to the 0.1.70 milestone Sep 20, 2023
@marcusburghardt marcusburghardt changed the title Include metric for list of rules and variables selected in Controls Include metric for rules and variables selected in Controls Sep 20, 2023
@marcusburghardt marcusburghardt changed the title Include metric for rules and variables selected in Controls Include metrics for rules and variables selected in Controls Sep 20, 2023
@github-actions
Copy link

Start a new ephemeral environment with changes proposed in this pull request:

Fedora Environment
Open in Gitpod

Oracle Linux 8 Environment
Open in Gitpod

@jan-cerny jan-cerny self-assigned this Sep 20, 2023
jan-cerny
jan-cerny reviewed Sep 20, 2023
View reviewed changes
Copy link
Collaborator

@jan-cerny jan-cerny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this PR. I have generated the outputs suggested in the Review hints section and I have reviewed these outputs.

utils/controleval.py Outdated
Comment on lines 192 to 193
rules_count = sum(count_rules_and_vars_in_control(ctrl)[0] for ctrl in ctrls)
vars_count = sum(count_rules_and_vars_in_control(ctrl)[1] for ctrl in ctrls)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better if count_rules_and_vars_in_control returned a namedtuple, it would allow you to avoid using the indices. Another possible approach would be to split count_rules_and_vars to two separate functions count_rules and count_vars.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thoughts @jan-cerny . I tried to make it a little more readable using namedtuple in the last commit. Could you take a look, please?

@marcusburghardt
Make new functions more readable
c27923d 
The count_rules_and_vars and count_rules_and_vars_in_control got some
additional lines in total but became more readable by using namedtupple
instead of going to index and eliminates the need for additional list
comprehensions.
@codeclimate
Copy link

codeclimate bot commented Sep 20, 2023

Code Climate has analyzed commit c27923d and detected 2 issues on this pull request.

Here's the issue category breakdown:

Category Count
Complexity 2

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 56.8% (0.0% change).

View more on Code Climate.

jan-cerny
jan-cerny approved these changes Sep 21, 2023
View reviewed changes
Copy link
Collaborator

@jan-cerny jan-cerny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have generated the outputs suggested in the Review hints section and I have reviewed these outputs.

@jan-cerny jan-cerny merged commit 490d800 into ComplianceAsCode:master Sep 21, 2023
34 checks passed
@marcusburghardt marcusburghardt deleted the controleval_rules branch September 21, 2023 08:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jan-cerny jan-cerny jan-cerny approved these changes

@Mab879 Mab879 Awaiting requested review from Mab879

Assignees

@jan-cerny jan-cerny

Labels
Documentation Update in project documentation. enhancement General enhancements to the project.
Projects
None yet
Milestone
0.1.70
Development

Successfully merging this pull request may close these issues.
2 participants
@marcusburghardt @jan-cerny