Extending OSCAL prose

[wendellpiez]

User Story:

As a producer of OSCAL data, I need to be able to drop in chunks of data with richer block-level structure than the prose model allows.

FedRAMP SSPs offer an example of this. Submitters will often put entire mini-documents with headers, or tables, into their implementation description. Currently OSCAL has no direct provision for these. Even if data conversions produce semantic-equivalent structures, users will feel perturbed if their visual representations are missing or distorted. Even if we do not preserve decorative detail such as table border settings, OSCAL at this level should be free-form enough to let users do what they are already doing or something close enough for comfort. This includes "make this a header line" and "display this as a table".

Currently the prose model allows paragraphs, code blocks, bulleted lists and numbered lists (not yet implemented at time of writing) as well as a range of inline elements for bold, italic, underlining etc.

HTML-like elements that could be considered included table (simple) with td and tr along with headers h1-h6. Since they have conventional markdown representations we could permit them without breaking our nominal compatibility with HTML within prose.

If it is judged that these elements should be permitted in some variants of OSCAL and not others (as they probably should not be used in catalogs for example, as they are semantically weak), we can consider a way of making these distinctions (for example, using Schematron to forbid them in catalogs).

Goals:

Permit greater expressiveness in OSCAL documents including implementation-level documents such as SSPs (eventually assessments etc.) by allowing more familiar block-level formatting in prose blocks.

Note that there are disadvantages to permitting users to include free-form elements including headers and tables, when it is easy (as OSCAL arguably makes it) to provide a richer and more rigorous data description by using nested, labelled sections (OSCAL part) instead of "art" (such as tables). Semantics that are only implied by display (such as table layout or typography) is only latent, therefore useless.

The counter argument is that even well structured OSCAL with ad-hoc labelling will be latent, so letting the users have their eye candy is actually good here.

NB: this issue, like #283, picks up from #200.

Dependencies:

• Stabilizing the markdown conversion pathway as described in Completing support for markdown in metaschema-driven conversions #283 including spec
• Collecting some samples or illustrations demonstrating the requirement (@brianrufgsa can help here)

Acceptance Criteria

• We can show OSCAL data including the relevant structures, valid to appropriate schemas and preserved through bidirectional conversion

[brian-ruf]

@wendellpiez thank you for creating this based on our conversation yesterday!
I wanted to add a few clarifying points.

First, this topic applies to three portions of an SSP:

• The system description (typically the least use of rich formatting)
• The description associated with the authorization boundary, network, and data flow diagrams. (typically the most use of rich formatting)
• The control implementation description (typically a moderate use of rich formatting)
  (NOTE: There will be additional use cases for this in the Assessment layer.)

In general, the decisions and actions here can and should apply to all three areas.

The most common formatting elements include:

• bold
• underline
• italics
• ordered lists
• unordered lists

The next most common formatting elements I see are:

• tables
• titles (via Heading4 style, Heading 5 style, or simply with bold text)
• inserted images

In very rare cases have I noticed a change in font face or size. CSPs are discouraged from deviating from the font face/size set in the SSP Template by FedRAMP. Additionally, I am unaware of any example where the use of a different font face or size would result in a loss of information or context.

In short, I believe if we allow the eight formatting capabilities above, we can faithfully represent existing Word-based SSP content in OSCAL without loss of context or fidelity.

[brian-ruf]

We should also address how to handle unsupported formatting. Do we ignore it? Do we convert angle brackets to escape codes (< and >)? Just in XML or also in JSON?

[anweiss]

Keep in mind that most of this is really only applicable in XML and to make it easier for running stylesheets and presenting HTML. With JSON, there are only two options: properly escape the prose as a JSON string ... or base64 encode the prose and store the result as a JSON string. In either case, there's no added benefit to the user because regardless, the JSON has to be deserialized and subsequently processed for whatever presentation mechanism the producer/consumer/tool desires.

[david-waltermire]

#200 provides good sources of documentation for this issue.

[wendellpiez]

Since PR #339 (accepted) the only elements not permitted in prose, which we have discussed, are img and br.

I don't much like br: it is semantically problematic in that it implies a structure that it cannot represent - it essentially says "split this" within undefined scope. However (as @brianrufgsa points out), there is a case to be made that users will demand it.

We are going to need img.

Anything we add needs to be supported in bidirectional conversion (#343).

[wendellpiez]

Work proceeds on bidirectional conversion. Everything should now be convertible except tables, br and img (in development). The Metaschema schema and production tools should be finalized in the context of unit testing these conversions.

Adding this Issue to Sprint 21 Project since completing this is a dependency for M1. The Acceptance Criterion will be satisfied by unit tests showing the range of permitted prose elements, valid (to a Metaschema-derived schema) and converting successfully.

[iMichaela]

5/23/2019

This work is in @wendellpiez 's branch. A PR will be submitted as soon as this issue is ready.

[wendellpiez]

Tables and HTML img are now supported in bidirectional conversion (see PR #379). br is supported in the schemas; finalizing its handling in conversion awaits fuller specification. (Need input from @david-waltermire-nist. There appears to be no community-regular standard for a br-equivalent in Markdown.)

[david-waltermire]

Done.