Support for "Rules" in OSCAL Models

[aj-stein-nist]

User Story:

As an OSCAL tool developer, in order to ensure my software can document testing requirements that an information system must implement as one part of cumulative control implementation requirements, I would like enhancement to the OSCAL models to more explicitly define the concept of a rule as a first-class citizen. Modifications and new additions to OSCAL to tool developers to build software for users to give specific criteria to test for a specific kind of implementation implied by control requirements, and have such criteria expressed in OSCAL.

Goals:

• Define notional use cases for the rule, operationally, then in terms of OSCAL mechanics
• Design supporting structure and insertion points for TBD rule assembly
• Design a rule assembly #1339
• Review current approaches to defining rules to confirm minimal data fields in rules-related models #1391
• Prototype Metaschema models and mockup data samples #1364
• Finalize the rule design (in the story above)
• Update the framing document content into Contents page and other relevant pages as needed
• Analyze use cases and determine where rule assembly should and should not be used in relevant OSCAL models
• Obtain NIST team and community buy-in for design that supports continuum of automated to manual rules

Dependencies:

N/A

Acceptance Criteria

• All OSCAL website and readme documentation affected by the changes in this issue have been updated. Changes to the OSCAL website can be made in the docs/content directory of your branch.
• A Pull Request (PR) is submitted that fully addresses the goals of this User Story. This issue is referenced in the PR.
• The CI-CD build process runs without any reported errors on the PR. This can be confirmed by reviewing that all checks have passed in the PR.

[aj-stein-nist]

While reviewing the backlog today, #821 seems related, but specifically focused on component-definitions. This story work is very similarly aligned, but perhaps greater in scope. User feedback this week indicates it might not be ideal to have such checks only or exclusive in a component, but more modeling work is needed to determine that.

[aj-stein-nist]

First things we ought to tackle:

• How do rules fit into each of the OSCAL models conceptually? (Does the concept of "rules" make sense in a catalog's model and document instance, for example? It seemingly makes sense at first glance in component or system-security-plan, right? Analyze this for each and every model.)
• How we will we connect/reference to the check for the rule (from Rule Checks in OSCAL Models #1059)?
• For third parties, how do we allow them to include checks for a model document instance? They might not "own" the document instance of a model (e.g. a SSP, a component) as its original author? They have to shim it in even if they are not the original author(s).

[aj-stein-nist]

Discussed more with Dave today. Will begin by sketching out the "do rules fit into each of the OSCAL models conceptually?" question as well as gathering consensus on what the the relevant data elements are.

[aj-stein-nist]

@david-waltermire-nist, I know we will have another backlog refinement in the near future, but you had asked if I am ready to pull this one into Sprint 49 if you and team are ready.

[aj-stein-nist]

Notes from our last touch-base with interest community members.

My current action item: build information model outline with recommended changes to include the concepts of rules.

[aj-stein-nist]

NOTE: This draft is obsolete, please see the updated draft spec below.

Proposed Information Model for rule

Moving forward, we might suppose a rule can look like this:

• rule [0 to ∞]
  • uuid [1]
  • title [1]
  • description [1]
  • origin [0 to ∞]
  • subject [0 to ∞]
  • props [0 to ∞]
  • link [0 to 1]
    • href
    • rel:
      • satisfies-partially
      • satisfies-completely
      • recommended-with
      • required-with
      • conflicts-with

A rule will be a summary explanation of mechanisms to evaluate and/or test amount of security and/or privacy information in OSCAL, including linkages to relevant controls. I specifically do not call out where rules should be inside a document instance of an OSCAL model yet because that is a separate discussed below.

Because a rule is an explicit mechanism to test or evaluate if a system satisfies a given statement, there are some obvious parts of the information model.

• uuid: uniquely identify the rule in its embedded OSCAL model document instance
• title: a summary title for the rule in question
• description: the brief explanation of the test
• origin: where the rule information originated from, be it a person or an automated assessment tool. We expect there are fully automated, semi-automated, and manual evaluation methods. origin, as it is used now in the assessment-results model
• subject: the subject of assessment, be it a person, process, or system component (service, server, et cetera)
• props: One or more properties with variables and their values to be mapped as inputs to a tool, if the origin is an automated assessment tool.
• link: this linkage is intended to function in two ways: 1) by way of specific @rel values (satisfies-partially; satisfies-completely) denotes if it partially or completely satisfies requirements elsewhere in a OSCAL document instance (e.g. the statement of an implemented-requirement of a control); 2) denotes it's relation to other adjacent rules by way of specific @rel values for the link (recommended-with; required-with; conflicts-with). This can be used in combination to allow specific rules to be annotated in a way for sufficient aggregation of what OSCAL document instance data they satisfy.

[david-waltermire]

A few questions:

• Given the above, if multiple rules are used to "satisfies-partially", how do you define the set of rules that "satisfies-completely"?
• How would you point to the rule or check implementation?

[aj-stein-nist]

A few questions:

* Given the above, if multiple rules are used to "satisfies-partially", how do you define the set of rules that "satisfies-completely"?

I think it will be up operator expertise, unfortunately. I was thinking about this a lot yesterday, and I have no good ideas on how we can determine that without some really complex semantics not in place yet.

But back to the actual question, you could use link/@href to point the implemented-requirement/statement (in the case of a component-definition for example) with a link/@rel bound to satisfies-completely and then set additional link/@hrefs for with the UUID fragements pointing to others independent rules that support that same implemented-requirement/statement and the link/@rel would be set to required-with. I know this leans heavily on link but I didn't know if we wanted new constructs here and additional complexity.

* How would you point to the rule or check implementation?

I think I need to flesh out an example, but that would be the origin construct. Is this a good reuse of such a construct or a bad one?

[iMichaela]

@aj-stein-nist and @david-waltermire-nist Would a numeric value associated with the rel for the satisfaction (aka percentage or score) make sense? In this way, an additive operation can be applied to identify the cumulative effect of partial satisfaction and aggregation towards satisfies-completely. The numeric values would be inputs from authoritative entities that can be identified through their ID, since it would be nice to know who decided this test satisfies 30% and not 80%
Also, a field that would indicate this rule is part of a superset of rules what is the uuid of that superset might be nice.

[aj-stein-nist]

@aj-stein-nist and @david-waltermire-nist Would a numeric value associated with the rel for the satisfaction (aka percentage or score) make sense?

Personally, I think it would be hard to measure and quantify, your question makes me believe maybe the satisfies- rel attributes need more thought. I meant it be qualitative and intended for expert human insight, but as I write this I realize that also might be too bold.

In this way, an additive operation can be applied to identify the cumulative effect of partial satisfaction and aggregation towards satisfies-completely.

I like the idea in principle but I have no idea how we could implement this in practice. Do you have any ideas around this? I too will reflect on it.

The numeric values would be inputs from authoritative entities that can be identified through their ID, since it would be nice to know who decided this test satisfies 30% and not 80% Also, a field that would indicate this rule is part of a superset of rules what is the uuid of that superset might be nice.

My intent with link/@rel="required-with" or link/@rel="recommended-with" was to allow for that through a tagging approach. In most instances, I find this more flexible. Are there downsides to this approach?

[iMichaela]

@aj-stein-nist I agree it allows for more flexibility, but I do not see a mechanism for identifying the completness or coverage of the superset of rules. A superset might still provide only partial coverage. Through the aggregation, the superset comprised of satisfies-partially rules could achieve satisfies-completely or still be satisfies-partially. Who makes the decission (e.g. role-id) and based on what evaluation mechanism to support consistent reproduceability? I am not saying a scoring or any numeric value will not factor in subjectivism, but at least it can provide a more accurate representation of the categorization (partial vs complete). Something a tool could parse, process (calculate), keep track of, and report. I am thinking one might want to include rules to carry on evaluation of the technology, without any level of testing or proof of the control sadisfaction. This scenario will provide 0% satisfaction for 800-53 control per 800-53A, but might support vulnerability scanning or alignment with CIS benchmark.

[wendellpiez]

Personally I think numbers are not a good way to represent such a value space. It is not even one-dimensional. Also, they can't reliably or meaningfully be treated as numbers (does satisfies="50" + satisfies "60" = satisfies "110"?) Simple labels are better in that they do not promise what they cannot deliver. If systems have actual ranked/numeric values in a taxonomy they wish to label things with, the model offers prop.

Noting however that the discussion on how to characterize linking semantics is a little different from the discussion on how to describe rules. The problem of describing the rules vs characterizing the relations between rules. For example, we have silently assumed the target of a link from a rule is another rule. But what if we want a link to point to a control or a statement inside a control, for reference (how do we do that)?

The discussion about the meaning of values of @rel makes sense only for links that are more tightly constrained than links in general (that is, links from one rule to another rule).

[iMichaela]

But what if we want a link to point to a control or a statement inside a control, for reference (how do we do that)?

@wendellpiez - Per our initial discussion, rules are associated with controls or statements of controls. I cannot picture a scenario when such a cyclic approach would be necessary (when a rule associated with a control or statement of control needs to link to another control). Do you have an example in mind?

does satisfies="50" + satisfies "60" = satisfies "110"?

It depends on the numeric value :). If it is 50% added to 60% you can only get 100%. If it is a different numeric value (a score) with a range of 0-200, then 50+60=110. If normalization will be necessary, then that can be done with a simple mathematical operation on the range. Do you have another approach that would support automation for aggregation and quantification of satisfaction degree, something GRC tools can implement? How will a bunch of satisfies-partially will be quantified? How will one assessor provide his perspective on satisfaction to another team member or system owner? I am open to better quantification solutions.

[aj-stein-nist]

I think both of you are making good points, @iMichaela and @wendellpiez, and I will likely adjust my draft recommendation to say the link/@rel can only be used to talk about possible relationships between independent rules (e.g. link/@rel="required-with" or link/@rel="recommended-with" or the others) and remove the idea of satisfaction markings we are discussing for this very reason. There is too much ambiguity.

I thought it might add value, for very specific assertions at the statement level, for a system owner or developer (not an assessor) to make very clear what they are and are not claiming about coverage of something. In addition to customer , I thought there might be value for very specific details of an implementation for such personas to point out "passing this rule check covers this specific detail of a statement on an implemented requirement for control AB-1 in its entirety" versus "for AB-1, passing this rule check is a good start, but we are not claiming that is enough and you have more work to do on your own (whether that is something else with the sub-system I provide or someone else, that is for us to discern somewhere else)."

I do not think there will be satisfactory answers to these questions at this point and it will get more difficult moving forward.

[aj-stein-nist]

I have continued trying to formulate examples, and it seems trying to reuse the origin and subject assemblies from elsewhere might not work so well, structurally. That, in addition to the feedback above, probably warrants an update in the design of the rule object proposed above.

[aj-stein-nist]

Trying to work backwards and come up with some examples now.

• Example of linking an automated OpenSCAP rule test to a statement of an implemented requirement of a control for an OpenShift Container Platform component-definition
• A complex listing of one or more interlinked rules supporting one or more statements of an implemented requirement of a control for an OpenShift Container Platform component-definition
• A representation of semi-automated rule completed by human interpretation of a semi-automated result
• A manual rule completed by human interpretation without immediately viable method for automation

[aj-stein-nist]

NOTE: This draft is obsolete, please see the updated draft spec below.

Proposed Information Model for rule

Moving forward, we might suppose a rule can look like this:

• rule [0 to ∞]
  • uuid [1]
  • name [0 to 1]
    • title [1]
    • description [1]
    • condition [0 to 1]
    • condition-evaluator [0 to ∞]
      • uuid [1]
      • name [0 to 1]
      • type [0 to 1]
        • prop [0 to ∞]
        • link [0 to ∞]
          • href
          • rel: dependency
    • prop [0 to ∞]
      • name: supports

A rule will be a summary explanation of mechanisms to evaluate and/or test amount of security and/or privacy information in OSCAL, including linkages to relevant controls. I specifically do not call out where rules should be inside a document instance of an OSCAL model yet because that is a separate discussed below.

Because a rule is an explicit mechanism to test or evaluate if a system satisfies a given statement, there are some obvious parts of the information model.

• uuid: uniquely identify the rule in its embedded OSCAL model document instance
• title: a summary title for the rule in question
• description: the brief explanation of the test
• condition: the fact used for evaluation of the confirmation or negation of the rule during a check
• condition-evaluator: where the rule information originated from, be it a person or an automated assessment tool. We expect there are fully automated, semi-automated, and manual evaluation methods.
• condition-evaluator/@prop: One or more properties with variables and their values to be mapped as inputs to a tool, if the origin is an automated assessment tool.
• condition-evaluator/@link: One or more links to a back-matter/resource with @rel="dependency" to link to copies of utilities and supporting configuration files (for cases of automated rule evaluation).
• prop: Additional properties for arguments to the rule. At this time, one explicit prop designed for use within the rule is prop[@name]="supports'] which will be used to logically link the rule and its condition (and later checks) as supporting evidence for an element in one or more OSCAL model document instances (e.g. a statement of an implemented requirement of a control in a component-definition, but more to follow). NOTE: This supersedes the link/@rel satisfies-partially and satisfies-completely construct recommend in an earlier draft.

[wendellpiez]

Are we allowed to debate names yet? I like the model so far. I like the migration of the settings from link to prop.

Is condition-target a subject? The idea with task/subject, associated-activity/subject is that these are inline descriptions (which could have links) as opposed to simply links to other (first-class) OSCAL objects. As against that use case ('rule' vs description of an assessment task), obviously requiring them to be links to first-class objects has processing implications perhaps good ones. But the boundary seems a bit blurry. (Is a rule an abstraction of a task?)

https://pages.nist.gov/OSCAL/reference/latest/complete/xml-definitions/#/assembly/oscal-assessment-common/task/subject

[aj-stein-nist]

Are we allowed to debate names yet?

Absolutely!

I like the model so far. I like the migration of the settings from link to prop.

Great, thanks for saying so.

Is condition-target a subject? The idea with task/subject, associated-activity/subject is that these are inline descriptions (which could have links) as opposed to simply links to other (first-class) OSCAL objects. As against that use case ('rule' vs description of an assessment task), obviously requiring them to be links to first-class objects has processing implications perhaps good ones. But the boundary seems a bit blurry. (Is a rule an abstraction of a task?)

https://pages.nist.gov/OSCAL/reference/latest/complete/xml-definitions/#/assembly/oscal-assessment-common/task/subject

The reason I moved away from that, @wendellpiez, is it could or could not be an abstraction over an (assessment task), dependent upon other future design decisions about what elements in what models it can refer to; task/subject, associated-activity/subject are constrained in that way. But rules do behave similar to them insomuch as they are " inline descriptions (which could have links) as opposed to simply links to other (first-class) OSCAL objects" but cannot point to a known predetermined list of ref sites. It was hard to reason about this when pointing rules to an associated-activity/subject, for example. This is why I backed out of using the subject and tasks constructs, because they are dependent on referring specific entities with @uuid identifiers that must precisely exist in specific models. Rules might be more haphazard than that.

(Ironically, this whole paragraph is a summary of yesterday's thought process around the id, uuid, id-ref conversation with you and why I struggled to explain why my previous design was hard to practice with an example.)

[aj-stein-nist]

Updated some more examples above. More examples to come, and maybe gathering up documents on use cases more concretely over the next week.

[aj-stein-nist]

@stephenbanghart, last we chatted you had discussed interest in this draft to add recommended features to one or more OSCAL models. The current draft design, with some examples embedded in a component, are available for review and public comment. Feedback welcome!

[aj-stein-nist]

Added some more examples. I am also tracking component-definitions and other XML samples in the following fork branch of the oscal-content repo.

https://github.com/aj-stein-nist/oscal-content/tree/issue-1058-rules

[aj-stein-nist]

In today's developer meeting, Dave discussed an emerging standard around remote attestation of system facts, RATS. I will look into this for potential overlap or improvements that can inform this work, especially around rules and rule checks regarding automated testing to support assessment objectives in some cases.

[iMichaela]

@aj-stein-nist & @pburkholder -- We might be able to also use one of the OSCAL mini-workshop meetings for this topic. Anca will probably touch on it during her talk on June 15 @ 11:00 AM ET
Presentation: IBM’s Trestle - compliance as code orchestrator and automation workflow
Presenter: Dr. Anca Sailer
Meeting URL: https://bluejeans.com/434440986/9212
For additional information about the OSCAL mini-workshop series, please visit:
https://pages.nist.gov/OSCAL/learn/presentations/oscal-mini-workshop-series/

[degenaro]

AJ, I think your example is good. Here's a slightly different view from my mind's eye (ie Anca my disagree or not), in trying to keep with what we've done with the current OSCAL thru re-purposing of properties. rule-test-model-proposal.json.gz
The idea is that a component implemented-requirement control optionally comprises one or more rules which optionally comprise one or more tests. I could image an optional "test-procedure" that expresses how to do the tests if there is a step-by-step "complex" procedure necessary. For our automations thus far, the tests are simply run. Not shown, but also possible I could image that the rules (and tests) would be directly embedded in implemented requirements rather than referenced by uuids. So far we have 1 rule comprising 1 test, but multiple rules comprising a control. As always, I reserve the right to be wrong, and I may have gone a bit overboard on the optional items comprising rules and tests.

[david-waltermire]

The idea is that a component implemented-requirement control optionally comprises one or more rules which optionally comprise one or more tests.

@degenaro We want to be able to share the same rule implementation (combination or rule and tests) across multiple controls. This is the reason for the testing-scenario. We haven't implemented a construct yet to associate a testing-scenario with a control-implementation.

I could image an optional "test-procedure" that expresses how to do the tests if there is a step-by-step "complex" procedure necessary.

Yes. We have the activity defined in the assessment plan and results. This might be a good way to define a manual test procedure.

For our automations thus far, the tests are simply run. Not shown, but also possible I could image that the rules (and tests) would be directly embedded in implemented requirements rather than referenced by uuids.

I have some concerns with allowing these constructs to be defined both inline and by reference, as this greatly complicates writing code that parses, persists, and processes content. Allowing the content inline makes it marginally easier for content creators at a cost to the tool developer that has to deal with this added complexity. We need to weigh such a decision carefully to make sure we address the equities in the right way.

[degenaro]

Agreed that rule-implementations in AJ's example (or rules my example) can be shared by multiple controls. The point was that testing-scenario may be unnecessary in many cases (in all of our cases so far). Just looking for a way to not impose this extra layer unless wanted. The solution was to 1) add the optional-test-procedure that is the equivalent of AJ's testing-scenario and 2) have rules list the tests that test them. Much less concerned about the inlining, and suggest this because that is how we are doing it today via re-purposed props.

[aj-stein-nist]

@david-waltermire-nist as discussed during our modeling session, I updated the models in aj-stein-nist/OSCAL@8dc7f35 to:

• Re-ordered rules, tests, and testing-scenarios accordingly the component-definition.
• Re-targeted the rule-implementation to either be referenced or bound at the implemented-requirement or its statement in a component's control-implementation.

[aj-stein-nist]

@david-waltermire-nist in advance of follow-on work this week on rules, per your requested, I collapsed #1160 and #1168 into the feature-rules branch for ongoing development.

[aj-stein-nist]

Per request from participants in the Model Review meeting on June 6, 2022, I am attaching the slides from today's presentation on the rules updates.

OSCAL_Model_Meeting_202206_Rules.pdf

Following up the last modeling meeting, here is the slides with updates on rules on today's model meeting.

OSCAL_Model_Meeting_Rules_202209.pdf

[openprivacy]

Just for grins (and possible collaborations though the referenced project seems to have slowed down) here's a project for turning legislation into code and running a rules engine on that: https://www.digital.nsw.gov.au/article/rules-code-test-learn-repeat

[aj-stein-nist]

Just for grins (and possible collaborations though the referenced project seems to have slowed down) here's a project for turning legislation into code and running a rules engine on that: https://www.digital.nsw.gov.au/article/rules-code-test-learn-repeat

Thank you very much for thinking of our work and providing this feedback, Fen! I reviewed the site and will determine how much I can incorporate into the work (if we can beyond looking at the site; more involvement seems to require we email this group and USG employees interacting with foreign officials is always a challenge, fun trivia).

Also, outside of work, I have been following the Linux Foundation/OSSF Alpha Omega project on their approach to how they intend to perform similar work for wrapping different kinds of attestations. But I largely skim and observe the high-level approach. No direct inputs or inspirations yet.

ossf/alpha-omega#28

It is nice to see a growing, vibrant space around this kind of workflow and process.

[degenaro]

Currently we re-purpose properties to hold rules and checks (aka tests), employing namespace and class. We envision the OSCAL "rules" enhancement to allow all that we can express with properties along with the ability to group sets of properties into one rule (or check) and to relate rules and checks to each other. With that in mind...

• Will the new rules support have facility to specify namespace and class? (Not everyone calls a rule a rule and a check a test.)
• Will the new rule support provide linkage capability to evidence reference urls and such?
• How will parameter ids be specified relative to rules and checks, each having different default/selected value and each having different set of possible valid values?

[degenaro]

Metaschema links of interest:

• https://github.com/usnistgov/OSCAL/blob/feature-rules/src/metaschema/oscal_ssp_metaschema.xml
• https://github.com/usnistgov/OSCAL/blob/feature-rules/src/metaschema/oscal_component_metaschema.xml
  Related mapping:
• https://pages.nist.gov/OSCAL/reference/develop/mapping/

[aj-stein-nist]

This work should go back to user research and discovery, so this will be moved back to DEFINE Research Needed. After that, if it is returned to development as-is, we should consider refinement being needed as this epic, as previously used it, is too large re upcoming #1688 reorganization and needs to be broken down into manageable pieces.

[Arminta-Jenkins-NIST]

Reviewed on 11/2 at triage meeting.