Schema documentation generation from the Metaschema

[david-waltermire]

User Story:

As an OSCAL developer, I can use the metaschema to post documentation and examples to pages.nist.gov/OSCAL site.

Goals:

• Complete support for tagged samples (example code, both XML and JSON) into documentation produced from metaschema. (@wendellpiez)
• Merge existing schema documentation on pages.nist.gov/OSCAL into the metaschema (@wendellpiez @kscarf1)
• Generate pages.nist.gov JSON and XML schema documentation and examples from the metaschema using a scriptable approach. (@wendellpiez)

Dependencies:

None.

Acceptance Criteria

• All schema documentation on pages.nist.gov/OSCAL can be automatically generated from the metaschema using a script.
• Existing documentation has been merged into the metaschema.
• Existing examples on pages.nist.gov/OSCAL have been merged into the metaschema.

[david-waltermire]

This should also address #139.

[wendellpiez]

Sprint 11 Progress June 7 2018

The metaschema now supports element documentation, and (older) documentation of the catalog model has been merged into the prototype metaschema. The same has to be done for the profile and declarations models. Also, an XSLT can be used to produce HTML documentation from the metaschema, including writing out literal tags for embedded example elements. In turn, this HTML documentation can be post-processed to produce clean markdown.

Progress: modeling 100%; merging legacy content 50%; supporting docs production 50%; editing this document and producing and integrating examples 5% (all together: 40%?)

There is also a dependency on finishing the metaschema support for bidirectional XML-JSON conversion #186, since we intend to process XML examples to produce equivalent JSON examples.

[david-waltermire]

2018-06-07 Meeting Notes

@wendellpiez will focus on completing this early in the week of 6/11, to include production of examples in the documentation.

[wendellpiez]

Now see https://github.com/wendellpiez/OSCAL/blob/feature-metaschema/schema/metaschema/OSCAL-catalog-metaschema-docs.md

[kscarf1]

@wendellpiez the URL in your previous status update for this issue doesn't work for me.

[wendellpiez]

@kscarf1 Sorry about that, the file moved - now at https://github.com/wendellpiez/OSCAL/blob/feature-metaschema/schema/metaschema/oscal-catalog-docs.md

There is also https://github.com/wendellpiez/OSCAL/blob/feature-metaschema/schema/metaschema/oscal-profile-docs.md

Both of these are pulled from metaschema source files, next to them in the same directory.

[wendellpiez]

Sprint 11 Progress

With apologies, there has been further "adjustment" in the locations of things. Indeed final locations are not absolutely fixed.

At the moment, markdown versions of documentation for both XML- and JSON-formatted OSCAL are located next to the (generated) schemas. Functionality is reasonably complete. Data (the documentation itself) is coming along (subject to changes due to #190).

The linkage to Pages remains to be tested.

Also the renditions (both/all) are not absolutely finished and polished at time of writing (80%).

Can't believe this Issue has been marked LoE Medium.

[david-waltermire]

6/21/2018 Status

1. The mechanism for generating XML and JSON documentation is close to done and mostly working.
2. The catalog documentation from pages.nist.gov has been merged into the metaschema, with some enhancements. Profile documentation still needs to be merged.
3. Examples still need to be merged. Many new examples are needed.
4. The documentation framework for pages needs to be updated, and generated files need to be tested within the modified framework.
   • XML and JSON pages need to be split into separate top-level sections, with only the related example tab.
   • Tweaks to the content are needed to make them render properly.

[wendellpiez]

The transformations now target the markdown usage currently on the pages so things are now as good as I can make them without staging and testing them. @anweiss please remind me where I will find hints? You have helped me with this in the past. (But ideally we would do this somewhere visible to everyone.)

FWIW future iterations of this docs pipeline could deliver HTML/CSS, letting us face the browser directly as well as the Pages scripts and filters ... but we have to be happy with the maintenance model first.

[anweiss]

@wendellpiez you can find the current pages.nist.gov/OSCAL schema reference markdown structure in https://github.com/usnistgov/OSCAL/tree/docs/source/includes/schema. There are some nuances to the way that Slate interprets and parses headers, anchor links, and ordering of elements. You can find the official reference at https://github.com/lord/slate/wiki/Markdown-Syntax.

[kscarf1]

June 27, 2018 status: Standing by to help when needed.

[wendellpiez]

Sprint 11 Progress Report: June 28 2018

Finishing this task and actually completing the merge -- moving the Pages site to the new metaschema-based docs production model -- is (I believe) the primary agenda item for tomorrow's meeting.

A second agenda item is to continue improving the documentation itself, cf #190.

[iMichaela]

July 18 Status Update (Sprint 12 Acceptance Meeting)

The issue is waiting for a review from @david-waltermire-nist and/or @iMichaela

[david-waltermire]

Completed by PR #201.