Discuss concept of an encapsulated OVAL definition for OVAL 6.0

[vanderpol]

@OVAL-Community/oval-board-members
During our meeting on 09-26-2024, the board was not able to come to a consensus regarding redesigning OVAL definitions.
Pros:

1. Easier content editing and creation
2. Easier content debugging
3. Easier content maintenance
4. Easier to copy definition to more than one benchmark

Cons:

1. Will make files somewhat larger (maybe 10-20% depending on content)
2. Makes extend_defintion obsolete
3. Will require existing content authoring tools to be updated

Existing 5.12 format with several silos of information

Proposed 6.0 format with encapsulated definitions

Discussion will end on 10-04-2024 with voting to occur on 10-07-2024

[evgenyz]

So, as we discussed on the meeting, there is already a way to organize OVAL documents in this way: use of external references. Can we somehow try and use this feature (improved maybe) to make is less backward-incompatible? I'd really hate to kick out extended definitions.

[vanderpol]

@maxullman as our primary goal for the update is to make content author's lives easier, we are going to have to step back and consider the consequences of having 'quasi-encapsulated' where extend_defintions are still somehow supported. Related to your use case of 'Windows 10 is installed', are there scenarios where you would want the end result to be "not_applicable"? If so, please chime in on the the discussion #170

[maxullman]

@vanderpol , it wouldn't be as encapsulated in the sense that you couldn't blindly copy paste whole definitions, but it seems like you could still keep the extend definitions as is just as references to other definitions and encapsulate the other oval components in the definitions.

A more extreme but more backwards compatible proposal is we could keep the existing structure where the tests/objects/... are grouped spread out but just add the ability to define tests/objects/... encapsulated in the definition. Then in a later phase if people with time actually adopt/like the encapsulation, the current place could be removed.

[evgenyz]

We are hoping to make SCAP easier to write and understand. If you could provide an example of what you are thinking, and demonstrate how this would help OVAL content adoption, ease of understanding and long term maintenance, I'd appreciate it.

Here is a completely legit (it might miss few ids or namespaces, but structurally it is sound) SCAP1.3/OVAL5.11 data stream:
ds.zip

What I'm trying to say is that there is a way to structure OVAL sub-docs the way you want it. And if you don't want to use extended defs in your org, you can just deploy a git hook or something; you definitely don't have to forbid it for the whole world.

[evgenyz]

Meh, forgot to add second Rule referring oval_file.xml check, but I guess you got the idea.

[evgenyz]

This approach actually has another perk. These separate OVAL defs could be physical files and could be referred to like

  <Rule selected="true" id="xccdf_moc.elpmaxe.www_rule_1">
    <check system="http://oval.mitre.org/XMLSchema/oval-definitions-5">
      <check-content-ref href="file.oval.xml" name="oval:x:def:1"/>
    </check>
  </Rule>

Which makes it dramatically easier to navigate, edit and test without changing a single line of the code in these definitions.

[solind]

Two observations:

1. It seems to me like it should be easy to convert a file between the two approaches (encapsulated defs vs. traditional)
2. Does the encapsulation go far enough? Couldn't the object and state(s) declarations be collapsed/encapsulated into each *_test? Of course, this would be an even more radical schema change.

[vanderpol]

I agree on the ability to convert a file, as we are about 90% complete with an application which will do exactly that. We have not pondering going back from 6.0 to 5.12, although that should be doable as well, although by using some metadata breadcrumbs documenting what the original items/states/object/var ID's were.

As for the test level encapsulation, I pondered it a while back as well, as for very large/complex definitions, it would make understanding the content easier, but as with this example, would further bloat the content, as objects could not be reused even across tests of the same definition. At that point, we could just drop object/state/variable ID's entirely and just have them be components of the test (when I tried to prototype up an XML based XCCDF results format, this was essentially what I ended up with. Given the current OVAL board apprehension to change, and our hard deadline with SCAP 3.0, I suspect any change beyond OVAL definition encapsulation would be beyond what we can deal with, but I'd be open to further discussion if we think it could be stable in 2 months.

[solind]

Well, there's a limit to the encapsulation you can do if you still want to do things like use <object_component> ... of course, I suppose you could also just embed objects inside such tags and eliminate the IDs altogether.

Would that work generally?

One technical advantage to this approach is that you could interpret it with a SAX parser, and keep very little of the document in-memory. Of course, the same is true with encapsulated definitions as initially proposed... the whole definition would be needed in-memory, but this is much less than a whole OVAL document.

[DeafBDoor]

Encapsulated definitions look much easier to maintain, but I understand the benefit of allowing references to previously described objects and states. So, how about a hybrid approach, where you can refer to objects and states previously defined in other definitions?

Something like this:

<definition ...>
  ...
  <objects>
    <registry_object import_id="oval:mil.disa.fso.windows.obj:420800" />
  ...

to allow the reuse of object previously defined with that id. This is similar to what happens in the YAML language, for instance.

And a tool would be able to quickly crossreference the id and provide a preview or a link to it.

[DeafBDoor]

Additionally, a tool could be used to actually expand the element, essentially copying it from the original place. In this case, a strategy of automatically assigning id needs to be devised.

[harrisdk]

Please correct me if I am not understanding your idea fully.

You are suggesting that instead of duplicating identical elements (objects, states, tests, variables) the content could include a reference to the shared element instead. While this idea would reduce the file size of the content it would not improve the readability of the content for end-users. An end-user would still need to either run the content they received through a debugging tool or treat the content as a choose-your-own-adventure novel.

While a debugging tool could be developed that is a notable burden for end-users to know such a thing exists as well as to be able to use it effectively. I believe that as specification authors we should move any complexity burden to the content authors, who should then offload that burden to content authoring tools, because they are the most knowledgeable in the chain.

[DeafBDoor]

or treat the content as a choose-your-own-adventure novel

That made me cackle!

Yeah, I understand your point. From my past experience doing DISA-STIG and CIS, the amount of objects, tests and states which were reused was a small fraction of the total. However, I'd like more input into this from the other OVAL writers to confirm.

[dodys]

in compliance as code, we already write oval files not in an encapsulated form, but in a "single view" e.g.:
https://github.com/ComplianceAsCode/content/blob/master/docs/workshop/data/accounts_tmout/oval/shared.xml

The only thing is that the build of a product (generating the output oval, datastream, xccdf) will make sure to put the each tag under its correct place.
Couldn't we just have a tool to show any of the files in either "encapsulated-like" or not encapsulated?

Because if this is to make development easier, the way we do at compliance as code is simple enough.

[vanderpol]

During the 11-07-2024 OVAL board meeting, 2 straw polls were conducted to determine if any change should occur, and if so, what should that change be. A new option of BOTH the existing 5.x format and new encapsulated format with allowing extended definitions was proposed.

Of the 3 options, the straw poll concluded that either the new encapsulated format, allowing extended definitions or the new "BOTH" method should be used. Leaving the format as 5.x was voted out.

[robertgendler]

Apologies for missing the meeting.

If it was the Both option do we expect to support both ways for some defined time and then after version X(to be determined) OVAL would fully switch to the new way and drop the legacy way?

While backwards compatible can be great, it tends to hold back progress if you hang on too long and can just build tech debt and cruft. And if we have both it could be confusing in a way for newer content developers. It's definitely much much easier IMO if there's a very clear cut method. I believe the encapsulated new way is easier for new content developers to understand what's going on.

[vanderpol]

@robertgendler from my view in the meeting the BOTH option would likely remain in the language for the foreseeable future, as there appears a need, primarily with large vulnerability content to use the existing 5.x design, there were concerns of file size and efficiency of processing. If after some time we revisit this and content authors opinions change, we can always drop the legacy 5.x format, but likely that would be an OVAL 7.0 as it would break backward compatibility.

[robertgendler]

Ahhh yes the vulnerability content. I wasn't even thinking of that. Kind of makes sense. But I do think maybe by OVAL 7 it should be dropped (whenever that may be someday)

[vanderpol]

Official vote has been recorded at https://github.com/OVAL-Community/OVAL-Board/blob/main/board_votes/2024-11-07_OVAL_6.0_Definition_Structure_Vote.md

Closing this discussion for OVAL 6.0, we can open a new discussion if further changes are needed for OVAL 7.0