Correcting profile resolution spec wrt validation of inputs

[wendellpiez]

Discussed in #1308

^{Originally posted by wendellpiez June 13, 2022}
The current profile resolution spec is somewhat ambiguous and possibly problematic regarding the requirement that inputs be valid OSCAL.

At https://pages.nist.gov/OSCAL/concepts/processing/profile-resolution/#d2e317-head it is stated clearly:

If the object fetched cannot be found or is not a valid OSCAL object, the tool MUST cease processing and provide an error.

Yet later (for example) https://pages.nist.gov/OSCAL/concepts/processing/profile-resolution/#d2e786-head, regarding structuring directives, the spec says "If more than one appears, processors MUST generate an error and cease processing" - which does not happen on valid inputs.

For lightweight implementations of profile resolution as simple transformations (not full-scale OSCAL object marshalling), the first requirement is onerous. It means a lightweight profile resolution processor that works on simple GIGO principles is not actually conformant, at least not delivered and deployed separably from a validator. This could be conformant only if the language is rewritten to the effect that "only on valid OSCAL inputs can results be predictable" rather than that a processor must (a) incur the overhead of validating any resource (including upstream catalogs), and (b) stop with an error when such (in)validity is detected (which could certainly remain an option).

However, if the first requirement is needed (invalid inputs will never be processed), surely the second is superfluous (what to do when they are seen).

Let's discuss whether validation of all inputs is an absolute runtime requirement, or whether a GIGO implementation (that accepts any input and 'does its best') could be conformant even if it does not perform validation or report exceptions on nominally invalid inputs?

If not, let's remove the redundant check for invalid configurations.

Language suggested by @david-waltermire-nist:

If the object fetched cannot be found, or does not parse as an OSCAL catalog or profile, the tool MUST cease processing and provide an error.

For implementation, this implies that formal validation need not always be asserted (and processors are not required to cease for nominal validation errors that do not affect outputs), but that inputs that are clearly not correct (e.g., not OSCAL or not the right kind of OSCAL) can be defended against.

[david-waltermire]

@wendellpiez Would you please make a PR with the following adjusted text?

If the object fetched cannot be found, or does not parse as an OSCAL catalog or profile, the tool MUST cease processing and provide an error.