Brainstorming OVAL 6.0 and some type of conditional (if/else..) etc...

[vanderpol]

@OVAL-Community/oval-board-members

We've had on occasion the need to exclude performing a certain test, because it may error out on that system (or for various other reasons). Something like

If
Run some cmdlet test that we know is installed on domain controllers
else
Run some other test that we know works for non-domain controllers

We currently have this 'working' by other means with all criteria AND/OR, but we have had to change some error/warnings to just debug when we come across a cmdlet that isn't installed and returns error, as all tests are run on all systems regardless.

This could be implemented like the XCCDF platform specification, or a true IF/ELSEIF/ELSE scenario.

I'll admit I haven't put enough thought into this to make a full proposal and realize the consequences, but I suspect that I can't be the only person to dream of being able to do an IF in OVAL?

IF there was a time (pun intended), OVAL 6 is it. This is our one and only chance.

IF you agree, and would like to see it happen, share your thoughts, proposals, etc...

[solind]

We've previously had discussions around making the applicability_check attribute of criteria/criterion/extend_definition consequential to the evaluation. For the use-case you're describing, however, I've thought the XCCDF platform checks should suffice, since they can be used to gate whether OVAL checks should be evaluated on a target machine.

[vanderpol]

In our use case, this is all under a single DISA STIG XCCDF rule, which has two different methods for checking, dependent on the target. As this has to tie back to a single XCCDF rule in the STIG manual, the logic of how to check falls to OVAL. If we had 2 different rules, one for DC and one for Member Servers, XCCDF applicability would suffice. We have used it in many other scenarios.

If this is the only scenario, then we can just push DISA for a STIG update in the future, but I suspected that having some conditional checks in OVAL would be useful overall.

[solind]

My recollection of what's technically allowed is fuzzy, but, couldn't you have multiple oval checks with different platforms? Then the evaluation phase could pick the right one to run. Or... is the platform only specified at the rule/group level?

[vanderpol]

the xccdf platform is currently at the benchmark, group/rule level. I think we would need to add it inside of the "check" for it to work the way we would need. We can pursue that as well if we don't think that OVAL needs this capability.

[vojtapolasek]

In our use case, this is all under a single DISA STIG XCCDF rule, which has two different methods for checking, dependent on the target. As this has to tie back to a single XCCDF rule in the STIG manual, the logic of how to check falls to OVAL. If we had 2 different rules, one for DC and one for Member Servers, XCCDF applicability would suffice. We have used it in many other scenarios.

If this is the only scenario, then we can just push DISA for a STIG update in the future, but I suspected that having some conditional checks in OVAL would be useful overall.

I think this could be expressed with XCCDF complex checks. I think you could check the XCCDF specification (attaching) section 6.4.4.4.
xccdf.pdf

[vanderpol]

I might be mis-interpretting the XCCDF spec, but in my reading the XCCDF complex check still performs all of the checks called, it just allows the AND/OR to be calculated in XCCDF vs down in OVAL, primarily for mixing OVAL/OCIL etc.. From section 6.4.4.4 of the XCCDF spec:

"In place of an xccdf:check element, XCCDF allows an xccdf:complex-check element. An xccdf:complex-check is a boolean logical expression whose individual terms are xccdf:check and/or xccdf:complex-check elements."

[evgenyz]

Well, the state is definitely not ideal: OpenSCAP/openscap#273.

I would prefer a better not applicable/not implemented/not available states bubbling up from OVAL into XCCDF and better handling of these instead of having the same logic duplicated in OVAL.

[vanderpol]

And what we're trying to accomplish is to figure out how to make OVAL easier to use, as SCE (not even sure what is is) isn't part of SCAP. Now is the time to make OVAL better.

[solind]

@vanderpol - SCE (which stands for Script Check Engine) is just a way to embed a shell script in an SCAP benchmark (in ZIP bundle or XML datastream form), and map the exit code of a script execution to an XCCDF rule result. It was invented by RedHat, and has been used pretty extensively by CIS. And, since it can be embedded in a datastream, it can also be digitally signed.

The shell_command test is nice enough from a capabilities perspective, but sometimes you need a more complicated script to express the logic of a check.

If there's going to be a post-v1.3 SCAP, it could make sense to include it.

[vanderpol]

@solind, interesting to learn about SCE, especially after being shared a CIS benchmark that had their own commandshell OVAL test in it, which was also digitally signed. The fact that we independently came up with essentially the same thing means SCAP/OVAL definitely needs something like it, one way or another.

I was hoping to have more of an official proposal, with more sample content, results and a demo build of SCC to share this week, but should have something by next week. As FYI, the shellcommand test actually supports more than just a single command, anything you could pass into the shell interpreter, and we are expecting essentially shell scripts quite frequently, along with single commands, depending on what is needed. We bounced around several ideas on what the OVAL test name should be, and are open to suggestions.

[vanderpol]

platform_without_foo

And what I'd like to be able to do is to not process any specific test, based on the outcome of some other test.

If Test1 = True
Process test 2
Else
Process test 3

Tests 2 and 3 'may' or may not be applicable to the target, but I'd like to be able to not process ALL tests when I know that only a subset of them are needed to get the answer I'm looking for. In my specific scenario, Test 2 happens to cause an error to be logged, because the cmdlet in question is not installed on the target, but the ability to do and IF/ELSE could be for any various reason.

[solind]

You know, you can determine if a cmdlet exists using the Get-Command cmdlet, passing in arguments -errorAction SilentlyContinue to avoid an error. I bet you could write a cmdlet_test to check for the existence of your other cmdlet, and then structure the definition accordingly.

[wmunyan]

I know I'm not at CIS anymore and they've pretty much abandoned any new implementation of SCAP/OVAL development, but wanted to offer my 2 cents of historical context. Back in the day, CIS-CAT implemented and proposed an OVAL extension for "shellcommand", but the proposal to add that to the official OVAL language was denied. We moved forward with implementing SCE to get around that, although probably left in the shellcommand stuff because it was already there. Unfortunately that opened Pandora's box for CIS to abandon OVAL, drop all the people that had any OVAL experience or advocated for it, and implement benchmarks using more and more SCE-based checks. I am sure there's probably some mailing list archives *somewhere* where the discussions around the shellcommand_test could be found. As a side note, I'd like to try and stay involved in the community here, but obviously don't have a tool I can test any of these new proposals on, but am really happy to see all the activity and renewed interest in the next iterations of SCAP/XCCDF/OVAL. Thanks for not purging me from the mailing list :) Cheers, -Bill M.

…

On Thu, Oct 3, 2024 at 8:38 PM Jack Vander Pol ***@***.***> wrote: @solind <https://github.com/solind>, interesting to learn about SCE, especially after being shared a CIS benchmark that had their own commandshell OVAL test in it, which was also digitally signed. The fact that we independently came up with essentially the same thing means SCAP/OVAL definitely needs something like it, one way or another. I was hoping to have more of an official proposal, with more sample content, results and a demo build of SCC to share this week, but should have something by next week. As FYI, the shellcommand test actually supports more than just a single command, anything you could pass into the shell interpreter, and we are expecting essentially shell scripts quite frequently, along with single commands, depending on what is needed. We bounced around several ideas on what the OVAL test name should be, and are open to suggestions. — Reply to this email directly, view it on GitHub <#170 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB7K53CD3C72RE237O4WFZ3ZZXPRXAVCNFSM6AAAAABPJT6UYGVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAOBTHA3DAMY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.*** com>

[vanderpol]

Thanks for your background on this Bill, and I wish I had been more involved (or more open minded if by chance I was involved back when you proposed the OVAL shellcommand test. You are very welcome to stay as part of this community, and we appreciate your insight.

[solind]

@wmunyan good to see you chiming in! Hope all's well!

[vanderpol]

So my takeaway so far from this discussion is that OVAL lacks the ability to have conditional tests, but the general consensus seems to be that they are not needed, and there are ways to cobble around this limitation. If there isn't enough interest in improving OVAL with this feature, we can call this discussion closed. If I have misinterpreted the general consensus, please correct me, or add a proposal on how to improve OVAL.

[solind]

I believe that what I'm saying is that what OVAL "needs" is not so much conditional evaluation, what it needs is a more meaningful/useful way to specify applicability. These are related, but not identical concepts. And, I think your use-case would be better addressed by an improved applicability mechanism, than by the addition of a new conditional evaluation concept.

One of the familiar, even beneficial characteristics of OVAL is its blind obedience to collecting all the objects and evaluating all the tests, even when the outcome of a test is already knowable with what's been collected so far. IMHO

[vanderpol]

I'm game for anything that gives us more granularity on what should be checked. Any chance you have time to mock up what you think would scratch this itch?

[solind]

I previously had proposed a new "AND_APPLICABLE" operator that would have a new truth table, in which "not applicable" had weight.

But, like @wmunyan, I don't have access to an interpreter anymore.

[vanderpol]

If you still have a copy of the proposal, or where it might be (what year, etc..) to MITRE, or CIS? I've searched online, and not finding it (which most of the MITRE managed proposals are easy to find online). I've searched through the CIS email inbox archives, and didn't see anything with "AND_APPLICABLE". If you have it or can point me to it, maybe someone on our team can prototype it

[vanderpol]

Closing this discussion for OVAL 6.0, we can revisit if/when OVAL 7.0 is created.