Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide JSON mechanism to represent XML-equivalent prose #64

Closed
david-waltermire opened this issue Oct 31, 2017 · 25 comments
Closed

Provide JSON mechanism to represent XML-equivalent prose #64

david-waltermire opened this issue Oct 31, 2017 · 25 comments

Comments

@david-waltermire
Copy link
Contributor

Goals:

  1. Ensure that the JSON representation of OSCAL is equivalent in all functionality to the XML representation. This means that content can be round-tripped between the XML-JSON-XML representations. Whitespace preservation may not be possible,
  2. Develop a prose markup capability in JSON to support the equivalent features in XML (e.g., parameter insertion, etc.). Consider using markdown to do this, with some extensions.
  3. Answer how namespaces will be handled in JSON. OSCAL currently has a single namespace.

Dependencies:

  1. Examples of OSCAL XML-based catalog and profile content. These examples are SP 800-53 based.
  2. Pull request (Add JSON schema and examples #63) is merged into the sprint-5 branch,

Acceptance Criteria:

  1. Demonstrate that content can be round-tripped. Content can be converted from XML-to-JSON-to-XML without any semantic loss of information. (Addresses JSON data sources as well? #59)
  2. Demonstrate that OSCAL JSON content can be validated to be well-formed. (JSON schema #10)
@anweiss
Copy link
Contributor

anweiss commented Nov 1, 2017

Prose conversion taking shape. You can see how JSON escapes the XML prose in the latest generated JSON here. This allows us to retain the original semantics for round-tripping while still providing well-formed JSON. The same escaping mechanism applies whether or not we leave prose syntax as is, decide to leverage Markdown or utilize some other format for representing prose.

If need be, we can also easily retain both the raw prose as an escaped-JSON string and also extract the prose syntax in to additional JSON properties where it makes sense (i.e. <insert> tags, etc)

@akarmel
Copy link
Contributor

akarmel commented Nov 7, 2017

11/7/2017 - Sprint 5 Progress Notes

@wendellpiez
Copy link
Contributor

@anweiss it'd be great to see this one rendered into JSON: /examples/mini-testing/mini-testing-catalog.xml (in the Sprint 5 branch). It's considerably smaller than the catalogs so far, but shows some inline markup etc. (And eventually, the profiles you see alongside it.)

@anweiss
Copy link
Contributor

anweiss commented Nov 8, 2017

@wendellpiez -> mini-testing-catalog.json

Note that I've retained the line break characters and all prose tags. The original lexical order isn't currently maintained, but that's an easy tweak.

I'm currently putting prose under a literal JSON key named prose, but this is certainly up for discussion.

@wendellpiez
Copy link
Contributor

@anweiss, nice. Neatly illustrates a few issues (some already noted) including:

  • Some information loss in prose, including list contents, class attributes
  • Maintaining lexical order and whether/where this matters
  • Inline markup and sufficiency of current strategy vis-a-vis possible alternatives e.g. Markdown

Note that these are all critical if we aim for round-tripping. (If not perhaps we should focus on application requirements?)

Also there are now example profiles, including a couple with patches as well as parameters:

https://github.com/usnistgov/OSCAL/blob/sprint-5/examples/mini-testing/30_patched-profile.xml
https://github.com/usnistgov/OSCAL/blob/sprint-5/examples/mini-testing/31_patched-messy-profile.xml

@anweiss
Copy link
Contributor

anweiss commented Nov 9, 2017

@wendellpiez thanks, yea making a few tweaks to the tool as the loss of those elements and the class attributes was not intended.

@anweiss
Copy link
Contributor

anweiss commented Nov 9, 2017

@wendellpiez just updated the example -> mini-testing-catalog.json ... retains raw elements and inner XML.

@anweiss
Copy link
Contributor

anweiss commented Nov 9, 2017

Also, regardless of what inline markup we use (XML, Markdown, etc) to represent prose, we'll still have the same requirement of having to escape newlines and any special characters in the JSON string value to retain the originally intended format. So while it may seem like Markdown, or similar might provide for synchronous visibility between XML- and JSON-formatted OSCAL, it may not be sufficient still.

@anweiss
Copy link
Contributor

anweiss commented Nov 10, 2017

@wendellpiez added JSON examples for entirety of contents in mini-testing/ directory -> https://github.com/usnistgov/OSCAL/tree/bee9009a1e35b47c0dc66d067497041fc7f80db6/examples/mini-testing

@wendellpiez
Copy link
Contributor

Excellent point, Andrew.

@wendellpiez
Copy link
Contributor

(I mean, about the mapping of MD into JSON.) Oh thanks! Those folders are still thrashing, of course ...

@anweiss
Copy link
Contributor

anweiss commented Nov 14, 2017

No worries. Updated mini-testing-catalog.json which keeps prose in same lexical order as XML

@anweiss
Copy link
Contributor

anweiss commented Nov 14, 2017

@wendellpiez Added JSON equivalents for all of the FedRAMP, mini-testing and SP800-53 examples ... https://github.com/usnistgov/OSCAL/pull/63/files

Question for you on the following property_decl definition:

value = element value { attribute xml:space { 'preserve' }?, ( \inherit | autonum | text )* }
\inherit = element inherit { attribute from { text }?, text }
autonum = element autonum { text }

Is there a scenario where you envision more than one occurrence of <inherit>, <autonum> and/or text elements within a <value> element? All of the examples I've seen only depict self-closing <inherit> elements with a single <autonum> element. But the RNC schema seems to allow for one or more occurrences via the * denotation. This will impact how JSON is outputted for these particular elements.

@wendellpiez
Copy link
Contributor

Andrew, thanks for the very perspicacious question. :-)

The short answer is that I'm actually not altogether happy with this model currently: it's been on my come-back-to list. However, the problem is also worse than you suggest, as RNC text is an indicator not of an element named 'text', but of text content (#PCDATA in old DTD-speak), i.e. simply <value>TEXT</value> -- and more often than not, in the data, value is even whitespace sensitive, which is why that (xml:space attribute) switch is there to provide a signal warning against ws munging. Since bits of text might occur anywhere within the value (before, after, between any autonum or inherit that happens to be present) -- order becomes important, as well as cardinality/repeatability (of the text if not of the elements).

As for your question as put -- yes, we do envision such scenarios, or at least we haven't ruled them out -- while at the same time we haven't been focused on testing these functionalities. (They have been developed to the point where they are demonstrably useful on actual data "in the lab". However, exactly how they will be useful "in the field", and to whom, are very much still open questions.)

Is there a way we can sequester this until a user story addresses functionality/testing (as well as by implication, modeling) of OSCAL declarations? Their role within the entire JSON processing ecosystem is a lot less clear to me than in XML. (There they serve as control points for Schematron validation of the data with which they are associated. What is the JSON/Javascript analogue?) At the same time, they are not yet fully worked out even on the XML side....

@anweiss
Copy link
Contributor

anweiss commented Nov 15, 2017

Thanks for the explanation @wendellpiez! This helps a ton. Yea, let's push this to a future user story until we flush out the JSON elements.

wendellpiez added a commit that referenced this issue Nov 15, 2017
@wendellpiez
Copy link
Contributor

wendellpiez commented Nov 15, 2017

So I spoke too soon, I wasn't quite able to put this down.

The latest commit changes the model for declarations somewhat, improving and simplifying things. (See https://github.com/usnistgov/OSCAL/blob/sprint-5/schema/xml/RNC/oscal-core.rnc lines 54-59 and 81ff.)

In the new model, the requirement for preservation of order/repeatability of element contents no longer applies to thevalue element (anywhere), which will never be more than a string (in schema-valid OSCAL). Instead, it affects only the (new) calc element (available only as an alternative to value inside declarations and not elsewhere).

This does not solve the problem with representing calc in JSON but at least it no longer applies to value. (And it addresses other issues that were bothering me.)

Note we may be coming back to this when the "OSCAL declarations" mechanism becomes more salient.

@anweiss
Copy link
Contributor

anweiss commented Nov 16, 2017

Sounds good. Will generate it nonetheless and we can revisit when appropriate. Thanks!

@anweiss
Copy link
Contributor

anweiss commented Nov 21, 2017

@wendellpiez squashed PR and updated with equivalent JSON for all XML ... also included a couple of roundtripped examples ... https://github.com/usnistgov/OSCAL/pull/63/files

@wendellpiez
Copy link
Contributor

Awesome @anweiss, looks really nice.

In addition to other issues noted elsewhere, here's one small adjustment to improve the output of the XML serializer -- when attribute values are null (for example, when there is no value for class or control-id), then avoid emitting the attribute. I.e. do not write class="" or control-id="".

This would clean up things in general and avoid a validation error on profiles

<call control-id="ac.7" subcontrol-id=""/>

Where the rule given by the schema is that (on class elements) one only of control-id and subcontrol-id is permitted (and required) -- so this is invalid as given.

I imagine this should not be difficult to accomplish. In general, I should think writing an attribute with an empty string would be a special case, not the ordinary case, and a tool should not do this by default. (Or, the switch to make it happen should be easy, and switched on in these cases.)

@anweiss
Copy link
Contributor

anweiss commented Nov 21, 2017

Yep, agreed. I've actually been doing this already but missed a couple of fields. Will make sure to clean this up in the tool.

@wendellpiez
Copy link
Contributor

NB for future reference: a good example for showing issues/weaknesses in component ordering and arbitrarily ordered (and mixed) content is /mini-testing/mini-testing-catalog-roundtrip.xml.

@wendellpiez
Copy link
Contributor

(Oh we need that round trip I think! 😄 👍 )

@anweiss
Copy link
Contributor

anweiss commented Nov 21, 2017

mini-testing-catalog-roundtripped.xml ... some spacing and line break nuances, but otherwise not bad

@akarmel
Copy link
Contributor

akarmel commented Nov 21, 2017

11/21/2017 - Sprint 5 Progress Notes

  • Keeping up with Wendell on XML/JSON synchronization
  • Couple of tweaks/nuances remaining to be addressed with final code reminiscent of the original XML
    • Comments can be included if desired in JSON which should mirror XML

@akarmel
Copy link
Contributor

akarmel commented Nov 29, 2017

11/28/2017 - Sprint 5 Acceptance Notes

  • We have a solid set of JSON equivalents with some open issues to be addressed in future sprints.
  • JSON could use some design work around what the final form of the output should look like.

@akarmel akarmel closed this as completed Nov 30, 2017
david-waltermire added a commit that referenced this issue Apr 3, 2018
# The first commit's message is:

# This is a combination of 54 commits.
# The first commit's message is:

# This is a combination of 4 commits.
# The first commit's message is:

# This is a combination of 2 commits.
# The first commit's message is:

# This is a combination of 2 commits.
# The first commit's message is:

# This is a combination of 6 commits.
# The first commit's message is:

# This is a combination of 3 commits.
# The first commit's message is:

# This is a combination of 111 commits.
# The first commit's message is:

Initial commit of docs branch.

# This is the commit message #2:

Create CONTRIBUTING.md
# This is the commit message #3:

Create ROADMAP.md
# This is the commit message #4:

Update README.md
# This is the commit message #5:

Create README.md
# This is the commit message #6:

Update README.md
# This is the commit message #7:

Update README.md
# This is the commit message #8:

Create OSCAL-PRODUCERS.md
# This is the commit message #9:

Create OSCAL-CONSUMERS.md
# This is the commit message #10:

Update and rename OSCAL-CONSUMERS.md to USERS.md
# This is the commit message #11:

Update and rename OSCAL-PRODUCERS.md to IMPLEMENTERS.md
# This is the commit message #12:

Rename CONTRIBUTING.md to CONTRIBUTORS.md
# This is the commit message #13:

Update README.md
# This is the commit message #14:

Update README.md
# This is the commit message #15:

Update USERS.md
# This is the commit message #16:

Update README.md
# This is the commit message #17:

Update IMPLEMENTERS.md
# This is the commit message #18:

Update README.md
# This is the commit message #19:

Update ROADMAP.md
# This is the commit message #20:

Update USERS.md
# This is the commit message #21:

Update CONTRIBUTORS.md
# This is the commit message #22:

Update README.md
# This is the commit message #23:

Update README.md
# This is the commit message #24:

Update IMPLEMENTERS.md
# This is the commit message #25:

Update IMPLEMENTERS.md
# This is the commit message #26:

Rename CONTRIBUTORS.md to CONTRIBUTING.md
# This is the commit message #27:

Create control.md
# This is the commit message #28:

Update control.md
# This is the commit message #29:

Update control.md
# This is the commit message #30:

Update control.md
# This is the commit message #31:

Update control.md
# This is the commit message #32:

Add files via upload
# This is the commit message #33:

Update control.md
# This is the commit message #34:

Create temp.md
# This is the commit message #35:

Delete NIST-SP-800-53-Rev4-AC1.png
# This is the commit message #36:

Add files via upload
# This is the commit message #37:

Delete temp.md
# This is the commit message #38:

Add files via upload
# This is the commit message #39:

Update control.md
# This is the commit message #40:

Add files via upload
# This is the commit message #41:

Add files via upload
# This is the commit message #42:

Update control.md
# This is the commit message #43:

Update CONTRIBUTING.md
# This is the commit message #44:

Update CONTRIBUTING.md
# This is the commit message #45:

Update USERS.md
# This is the commit message #46:

Update CONTRIBUTING.md
# This is the commit message #47:

Delete CONTRIBUTING.md
# This is the commit message #48:

Delete USERS.md
# This is the commit message #49:

Add files via upload
# This is the commit message #50:

Delete CSA-CCM-IAM02.png
# This is the commit message #51:

Update control.md
# This is the commit message #52:

Update control.md
# This is the commit message #53:

Update control.md
# This is the commit message #54:

Update control.md
# This is the commit message #55:

Update control.md
# This is the commit message #56:

Update control.md
# This is the commit message #57:

Update control.md
# This is the commit message #58:

Update control.md
# This is the commit message #59:

Update control.md
# This is the commit message #60:

Update control.md
# This is the commit message #61:

Delete NIST-SP-800-53-AC1-in-OSCAL-XML.png
# This is the commit message #62:

Update README.md
# This is the commit message #63:

Update control.md
# This is the commit message #64:

Update control.md
# This is the commit message #65:

Add files via upload
# This is the commit message #66:

Delete ISO-27001-Control-A9.png
# This is the commit message #67:

Update control.md
# This is the commit message #68:

Add files via upload
# This is the commit message #69:

Add files via upload
# This is the commit message #70:

Delete ISO-27002-Control-9.1.1-part1.png
# This is the commit message #71:

Delete ISO-27002-Control-9.1.1-part2.png
# This is the commit message #72:

Update control.md
# This is the commit message #73:

Update control.md
# This is the commit message #74:

Update control.md
# This is the commit message #75:

Update control.md
# This is the commit message #76:

Update control.md
# This is the commit message #77:

Update README.md
# This is the commit message #78:

Update IMPLEMENTERS.md
# This is the commit message #79:

Add files via upload
# This is the commit message #80:

Delete oscal-layers.png
# This is the commit message #81:

Add files via upload
# This is the commit message #82:

Delete oscal-layers.png
# This is the commit message #83:

Add files via upload
# This is the commit message #84:

Update IMPLEMENTERS.md
# This is the commit message #85:

Update control.md
# This is the commit message #86:

Update IMPLEMENTERS.md
# This is the commit message #87:

Update control.md
# This is the commit message #88:

Rename IMPLEMENTERS.md to docs/prose/IMPLEMENTERS.md
# This is the commit message #89:

Rename IMPLEMENTERS.md to implementers.md
# This is the commit message #90:

Rearranged and outlined catalog documentation based on the conversation with karen and Wendell.

# This is the commit message #91:

Create catalog-xml.md
# This is the commit message #92:

Rename control.md to catalog.md
# This is the commit message #93:

Update catalog.md
# This is the commit message #94:

Update catalog.md
# This is the commit message #95:

Update catalog.md
# This is the commit message #96:

Update catalog-xml.md
# This is the commit message #97:

Update catalog-xml.md
# This is the commit message #98:

Update catalog-xml.md
# This is the commit message #99:

Update catalog-xml.md
# This is the commit message #100:

Update catalog-xml.md
# This is the commit message #101:

Update catalog-xml.md
# This is the commit message #102:

Update catalog-xml.md
# This is the commit message #103:

Update catalog-xml.md
# This is the commit message #104:

Update catalog-xml.md
# This is the commit message #105:

Update catalog-xml.md
# This is the commit message #106:

Docset migration to Slate

# This is the commit message #107:

Removing unused file.

# This is the commit message #108:

Update README.md

Corrected a typo
# This is the commit message #109:

Add files via upload

Graphical representation of OSCAL schemas aligned with Risk Management Framework steps and tasks.
# This is the commit message #110:

Create CONTRIBUTING.md
# This is the commit message #111:

Create ROADMAP.md
# This is the commit message #2:

Create README.md
# This is the commit message #3:

Update README.md
# This is the commit message #2:

Create OSCAL-PRODUCERS.md
# This is the commit message #3:

Create OSCAL-CONSUMERS.md
# This is the commit message #4:

Update and rename OSCAL-CONSUMERS.md to USERS.md
# This is the commit message #5:

Update and rename OSCAL-PRODUCERS.md to IMPLEMENTERS.md
# This is the commit message #6:

Rename CONTRIBUTING.md to CONTRIBUTORS.md
# This is the commit message #2:

Update USERS.md
# This is the commit message #2:

Update IMPLEMENTERS.md
# This is the commit message #2:

Update ROADMAP.md
# This is the commit message #3:

Update USERS.md
# This is the commit message #4:

Update CONTRIBUTORS.md
# This is the commit message #2:

Update IMPLEMENTERS.md
# This is the commit message #3:

Update IMPLEMENTERS.md
# This is the commit message #4:

Rename CONTRIBUTORS.md to CONTRIBUTING.md
# This is the commit message #5:

Create control.md
# This is the commit message #6:

Update control.md
# This is the commit message #7:

Update control.md
# This is the commit message #8:

Update control.md
# This is the commit message #9:

Update control.md
# This is the commit message #10:

Add files via upload
# This is the commit message #11:

Update control.md
# This is the commit message #12:

Create temp.md
# This is the commit message #13:

Delete NIST-SP-800-53-Rev4-AC1.png
# This is the commit message #14:

Add files via upload
# This is the commit message #15:

Delete temp.md
# This is the commit message #16:

Add files via upload
# This is the commit message #17:

Update control.md
# This is the commit message #18:

Add files via upload
# This is the commit message #19:

Add files via upload
# This is the commit message #20:

Update control.md
# This is the commit message #21:

Update CONTRIBUTING.md
# This is the commit message #22:

Update CONTRIBUTING.md
# This is the commit message #23:

Update USERS.md
# This is the commit message #24:

Update CONTRIBUTING.md
# This is the commit message #25:

Delete CONTRIBUTING.md
# This is the commit message #26:

Delete USERS.md
# This is the commit message #27:

Add files via upload
# This is the commit message #28:

Delete CSA-CCM-IAM02.png
# This is the commit message #29:

Update control.md
# This is the commit message #30:

Update control.md
# This is the commit message #31:

Update control.md
# This is the commit message #32:

Update control.md
# This is the commit message #33:

Update control.md
# This is the commit message #34:

Update control.md
# This is the commit message #35:

Update control.md
# This is the commit message #36:

Update control.md
# This is the commit message #37:

Update control.md
# This is the commit message #38:

Update control.md
# This is the commit message #39:

Delete NIST-SP-800-53-AC1-in-OSCAL-XML.png
# This is the commit message #40:

Update README.md
# This is the commit message #41:

Update control.md
# This is the commit message #42:

Update control.md
# This is the commit message #43:

Add files via upload
# This is the commit message #44:

Delete ISO-27001-Control-A9.png
# This is the commit message #45:

Update control.md
# This is the commit message #46:

Add files via upload
# This is the commit message #47:

Add files via upload
# This is the commit message #48:

Delete ISO-27002-Control-9.1.1-part1.png
# This is the commit message #49:

Delete ISO-27002-Control-9.1.1-part2.png
# This is the commit message #50:

Update control.md
# This is the commit message #51:

Update control.md
# This is the commit message #52:

Update control.md
# This is the commit message #53:

Update control.md
# This is the commit message #54:

Update control.md
# This is the commit message #2:

Update IMPLEMENTERS.md
# This is the commit message #3:

Add files via upload
# This is the commit message #4:

Delete oscal-layers.png
# This is the commit message #5:

Add files via upload
# This is the commit message #6:

Delete oscal-layers.png
# This is the commit message #7:

Add files via upload
# This is the commit message #8:

Update IMPLEMENTERS.md
# This is the commit message #9:

Update control.md
# This is the commit message #10:

Update IMPLEMENTERS.md
# This is the commit message #11:

Update control.md
# This is the commit message #12:

Rename IMPLEMENTERS.md to docs/prose/IMPLEMENTERS.md
# This is the commit message #13:

Rename IMPLEMENTERS.md to implementers.md
# This is the commit message #14:

Rearranged and outlined catalog documentation based on the conversation with karen and Wendell.

# This is the commit message #15:

Create catalog-xml.md
# This is the commit message #16:

Rename control.md to catalog.md
# This is the commit message #17:

Update catalog.md
# This is the commit message #18:

Update catalog.md
# This is the commit message #19:

Update catalog.md
# This is the commit message #20:

Update catalog-xml.md
# This is the commit message #21:

Update catalog-xml.md
# This is the commit message #22:

Update catalog-xml.md
# This is the commit message #23:

Update catalog-xml.md
# This is the commit message #24:

Update catalog-xml.md
# This is the commit message #25:

Update catalog-xml.md
# This is the commit message #26:

Update catalog-xml.md
# This is the commit message #27:

Update catalog-xml.md
# This is the commit message #28:

Update catalog-xml.md
# This is the commit message #29:

Fixed typos, updated repo documentation, and migrated documentation for use in Slate.

Corrected a typo (+4 squashed commit)

Squashed commit:

[6ada57f] Removing unused file.

[503ad71] Docset migration to Slate

[351257e] Update catalog-xml.md

[aae1e8b] Add files via upload

Graphical representation of OSCAL schemas aligned with Risk Management Framework steps and tasks.
david-waltermire added a commit that referenced this issue Apr 5, 2018
renamed file.

Cleaning up scraps (and checking setup)

Initial work modeling 'framework' for profiling functionality

More improvements supporting framework (profile) example

More adjustments to framework (profile) example

Touchups to framework strawman

Improvements to profiling (design and demo)

Updated OSCAL namespace

Fine adjustments for profiles especially regarding subcontrols

First efforts at FedRAMP mapping

More adjustments

More adjustments to profiling including SP80-53 baselines as profiles (provisional)

Slight improvement in logic just in case

Much development on profiling

More adjustments to profile examples and resolution including preventing infinite recursion in profile resolution

Superseded by profile resolver XSLT

FedRAMP misc

CSF stuff mainly

Touchups mostly

Adjustments in integration and display (CSF example mainly)

Rearranging schema docs

Cleanup

Misc improvements, touchups, reorg

Linking README.md to OSCAL Overview.md

Rename OSCAL Overview.md to OSCAL-Overview.md

Update README.md

Update README.md

Transferred/integrated content from main README.md

Moved content to docs/prose/OSCAL-Overview.md

Added pointer to tag library, resequenced material

Update README.md

Create Profile-Catalog-Relationship.md

Linked to the new Profile-Catalog-Relationship.md

Create Schema-Decisions

Rename Schema-Decisions to Schema-Decisions.md

Update README.md

Update README.md

Update OSCAL-Overview.md

More adjustments to schema docs (#25)

A couple of more or less misleading points now corrected

More updates

Moved documentation inside working, eliminating working/doc branch also adjustments to readmes

Amended and improved some high-level discussion

FedRAMP demo adjustments incl temporary readme

Fixed typo

Added presentation Dave made on 10.17.2017.

Correcting small bug in profile extraction

Many files rearranged also more dev on framework analysis

Touchups

New XSLT now runs assessment of a nominal 'worksheet' framework document producing a copy with links and annotations

Revs to high level docs and oscal-oscal.xml (issue #25)

Create README.md

Removed quotes

Mainly demo files for profiling implementation

Renaming in FedRAMP example to reduce confusion

More improvements and clarifications in demo

More adjustments incl filenames; new XSLT producing profile from linked worksheet

Schematron supporting worksheet editing

More adjustments to FedRAMP, schemas

More readme adjustments

More slight adjustments to schema setup

Added rev5 draft controls and moved rev4 to a new directory for better organization.

Added 800-53 schema version 2.0.

Wrapup on Sprint 4

Now UTF-8

Added feed container schema.

Tweaked to validate

Created schema directories for OSCAL XML and JSON schema.

Restructuring SP800-53 working dir

Updates to SP800-53 stuff for rev5

More refinements

Adjustments for SP800-53 rev5 OSCAL conversion

More Fedramp analysis including restructuring

Mini testing catalog

JSON under XSLT3 tinkering

Adjustments for capturing rev5

Fedramp mapping updates

Removing process litter

Schema and schema process updates

Now presenting profile demos

Rearranging schema files

Updated readme

Minor adjustments

Many improvements and adjustments including an implementation of patching in profiles

Improvements in resolution and rendering

Edits to samples and readmes

More unit tests

Another example

Schema adjustments; updated schema docs now describes profiling elements

Preliminary FedRAMP analysis

Standalone OSCAL SP800-53 rev4 and its baselines now in /examples

Refinements

SP800-53 updates

Improvements to profiling Schematron w/ mini-testing

Production pipeline adjustments

Ran fresh for some catalog adjustments

More detail on examples/mini-testing readme

Mini testing refresh

FEDRamp HIGHT baseline analysis with XSLTs

Refinements and reorganization to FedRAMP (preliminary) profile extraction

More refinements and polishing of FedRAMP examples

Further adjustments to profile examples

Further improvements to readme

Refresh

Adjustments in view of #64

Keeping profile schema in line w/ changes in core

Starting on hand adjustments to FedRAMP

Extending profiles Schematron to detect when profiles can/should use exclude instead of include

More updates and name changes

More minor adjustments (more of them and more minor)

Cleanup mostly

Improvements to tag library docs

Updating docs

Trying again (testing Github md alas)

More FedRAMP readme

More small edits

Touches
@david-waltermire david-waltermire added this to the OSCAL 1.0 M1 milestone Apr 6, 2018
david-waltermire added a commit that referenced this issue Apr 13, 2018
Moved everything into 'old' and added a new subdirectory for new
experiments.

More early iteration over names

More adjustments in names mostly

HTML page for Word import testing

Just touchups this time

Updates and misc improvements

Changed document element to 'catalog'

Improved and renamed mapping transformation

Adjustments to models, schema, Schematron

Managing gitignore

Same

Minor adjustments

First cut at 27K mapping into OSCAL

Renamed directories for consistency

Cleanup

Bit more cleanup

Adjustments to mapping

Same

Numerous enhancements, some reorg

Planning notes

Added starter FO XSLT

First cut at FO XSL

Minor improvements

Small adjustments

Many/much schema enhancements; proto XSD version

First cut at inline declarations with live validation

Implementing declarations; styling

Refinements and extensions; new assign, select, withdrawn elements

More support for parameters etc.

Now implementing parameters etc.

More touches

Misc improvements

Sundry adjustments

Many more small improvements

Much documentation

More touchups

Updates

Misc adjustments and notes

Significant rework following design session esp declarations

More improvements incl extending declarations to groups

More adjustments; now mapping SP800-53A also.

More polishing and maintenance

Much more including hierarchical number checking

Adjustments, extensions, corrections

Name changes in the model

More adjustments

Cleanup of obsolete artifacts

Minor alignment

Mostly CSS, also Schematron improvements

ISO27002 mapping document

Many enhancements; new COBIT 5 mappings

More on COBIT

Improvements to mapping docs

Adjustments to mappings incl HTML versions

Moved screenshot for documentation

More adjustments and docs

Mostly improvements to documentation

Refreshed 800-53 data with adjustments

More general improvements to models and mappings

Continuing refinements to validations in samples

Update README.md

Update README.md

Update README.md

Update README.md

Minor tweaks

Including initial cut at profiling (1st, rule-based control extraction) among other refinements and improvements

No longer extant

Added new control to COBIT5 example

Misc improvements including more COBIT5 support

File rename and further adjustments

Misc adjustment; SaxonJS demo files

Misc adjustments

Updated readme contents to reflect merge.

Fixed links.

Changed text to better reflect current state and to add more links.

Rearrangement and cleanup

Schematron and schema adjustment and refactoring

SP800-53 extraction improvement, enhancement, cleanup

Improvements and enhancements to ISO27002 extraction

OSCAL Documentation

COBIT5 example tweakage

Organizational notes

Adjustments

Removed old files directory.
Moved files from draft to root.

Updated README.md to reflect new pathing.

Updated readme to contain more documentation of the OSCAL layers.

Added documentation for the working directory.

Update README.md

Edited the text.

Update README.md

Edited the text.

Added Aug 10 tiger team slides with notes

Create a prose overview of OSCAL

Update OSCAL Overview.md

Update OSCAL Overview.md

Profile and catalog mapping: a trivial example

Replacing graphic with rescaled version

Update OSCAL Overview.md

Rescaled image

Update OSCAL Overview.md

Mods to oXygen project file

Removing 'demo' (till later)

Moved proprietary file

Refinements to ISO27002 mappings (new model)

Refining SP800-53 conversion/mappings

Updating lib support for new model

Top-level org

Draft readme docs

Further adjustment to directory readme.md

More adjustments to ISO 27002 and SP800-53 (new model etc.)

More cleanup to main sample subdirectories + readme tweakage

Mostly restructuring

More adjustments to extraction, parameter support

Includes more model refinements; draft implementation of parameter resolution

Adjustments for revised models

Adjustments to demos and conversion pipelines

Experimenting with profiles

Infrastructure adjustments

Minor correction to schema docs merge

Updating XSD w/ docs

More element renaming/adjustment

Now a stable schema again? With SP800-53 extraction adjustments

Profile-related mods

Org stuff

Moved mappind docs; added readme

schema documentation

Schema production and documentation pipeline

More reorg; tweaked schema; more docs

Improvements to tag library

Another attempt at internal links in md

trying again

Gonna get this eventually

More updates and tuneups

More updates to schema and docs

Lighter adjustments

Similarly subtle adjustments

Adjustments supporting parameter assignments

More docs

Enhancements to demo

Moving distractions out of the way

Slight readme tweakage

Tag library tweaks

More tag library adjustments

New readme for Schematrons

Adjustments to readme

Adjustments to mapping documents

Adjustments to readme

More adjustments to mapping documents

Updated punchlist

Updates to mapping docs

CSS for local editing of OSCAL OSCAL

Slight adjustments

Light editing

Light editing

Light edits

Light edits

Light edits

Light edits

More work on profiles

Revised directory descriptions

Added an explanation of the demo directories

More on profiling

Editorial tweaks to oscal-oscal.xml

Editorial tweaks to oscal-oscal.xml

Edits to schema documentation

More enhancements to tag library and its presentation

Internal links?

Slight adjustments

Further touchups to schema docs and their HTML rendering

Support for 'pre'

Rearranged and touched up profile experiments

More touchups to tag library (linking)

More adjustments to copy

More adjustments

Correcting display bug failing to render chars in CSS

Added a PDF with the notes.

Decided to risk it

One more readme

Added timestamping to extractors

Light edits to oscal-oscal.xml

More edits to oscal-oscal.xml with regenerated materials

Refining markdown export

More edits to docs and rendering

This time smaller adjustments

More improvements to docs and rendering

Slight enhancement to schema mitigating whitespace issue in validation

Revert "Working"

Added comment template.

renamed file.

Cleaning up scraps (and checking setup)

Initial work modeling 'framework' for profiling functionality

More improvements supporting framework (profile) example

More adjustments to framework (profile) example

Touchups to framework strawman

Improvements to profiling (design and demo)

Updated OSCAL namespace

Fine adjustments for profiles especially regarding subcontrols

First efforts at FedRAMP mapping

More adjustments

More adjustments to profiling including SP80-53 baselines as profiles (provisional)

Slight improvement in logic just in case

Much development on profiling

More adjustments to profile examples and resolution including preventing infinite recursion in profile resolution

Superseded by profile resolver XSLT

FedRAMP misc

CSF stuff mainly

Touchups mostly

Adjustments in integration and display (CSF example mainly)

Rearranging schema docs

Cleanup

Misc improvements, touchups, reorg

Linking README.md to OSCAL Overview.md

Rename OSCAL Overview.md to OSCAL-Overview.md

Update README.md

Update README.md

Transferred/integrated content from main README.md

Moved content to docs/prose/OSCAL-Overview.md

Added pointer to tag library, resequenced material

Update README.md

Create Profile-Catalog-Relationship.md

Linked to the new Profile-Catalog-Relationship.md

Create Schema-Decisions

Rename Schema-Decisions to Schema-Decisions.md

Update README.md

Update README.md

Update OSCAL-Overview.md

More adjustments to schema docs (#25)

A couple of more or less misleading points now corrected

More updates

Moved documentation inside working, eliminating working/doc branch also adjustments to readmes

Amended and improved some high-level discussion

FedRAMP demo adjustments incl temporary readme

Fixed typo

Added presentation Dave made on 10.17.2017.

Correcting small bug in profile extraction

Many files rearranged also more dev on framework analysis

Touchups

New XSLT now runs assessment of a nominal 'worksheet' framework document producing a copy with links and annotations

Revs to high level docs and oscal-oscal.xml (issue #25)

Create README.md

Removed quotes

Mainly demo files for profiling implementation

Renaming in FedRAMP example to reduce confusion

More improvements and clarifications in demo

More adjustments incl filenames; new XSLT producing profile from linked worksheet

Schematron supporting worksheet editing

More adjustments to FedRAMP, schemas

More readme adjustments

More slight adjustments to schema setup

Added rev5 draft controls and moved rev4 to a new directory for better organization.

Added 800-53 schema version 2.0.

Wrapup on Sprint 4

Now UTF-8

Added feed container schema.

Tweaked to validate

Created schema directories for OSCAL XML and JSON schema.

Restructuring SP800-53 working dir

Updates to SP800-53 stuff for rev5

More refinements

Adjustments for SP800-53 rev5 OSCAL conversion

More Fedramp analysis including restructuring

Mini testing catalog

JSON under XSLT3 tinkering

Adjustments for capturing rev5

Fedramp mapping updates

Removing process litter

Schema and schema process updates

Now presenting profile demos

Rearranging schema files

Updated readme

Minor adjustments

Many improvements and adjustments including an implementation of patching in profiles

Improvements in resolution and rendering

Edits to samples and readmes

More unit tests

Another example

Schema adjustments; updated schema docs now describes profiling elements

Preliminary FedRAMP analysis

Standalone OSCAL SP800-53 rev4 and its baselines now in /examples

Refinements

SP800-53 updates

Improvements to profiling Schematron w/ mini-testing

Production pipeline adjustments

Ran fresh for some catalog adjustments

More detail on examples/mini-testing readme

Mini testing refresh

FEDRamp HIGHT baseline analysis with XSLTs

Refinements and reorganization to FedRAMP (preliminary) profile extraction

More refinements and polishing of FedRAMP examples

Further adjustments to profile examples

Further improvements to readme

Refresh

Adjustments in view of #64

Keeping profile schema in line w/ changes in core

Starting on hand adjustments to FedRAMP

Extending profiles Schematron to detect when profiles can/should use exclude instead of include

More updates and name changes

More minor adjustments (more of them and more minor)

Cleanup mostly

Improvements to tag library docs

Updating docs

Trying again (testing Github md alas)

More FedRAMP readme

More small edits

Touches

add json schema and examples

Signed-off-by: Andrew Weiss <anweiss@docker.com>

Update README.md

Update README.md

Update README.md

Update README.md

Added an issue template for our sprint-related issues.

Moved the issue template to the right location.

Added a number of issues templates to support project tracking and new
issue creation.

Initial cut and placeholder for Profile Semantics docs

Minor revs

More various edits

More adjustments

More revisions to profile semantics docs

Mostly more diagrams

More touchups to docs

More touchups to docs

Correction to top-level readme

Sketching implementation layer in XML

oXygen schema documentation

Touches

New page suggesting regrouping logic

Tweaking

More touchups and corrections

Had to rename the spec document

Now indexing to SP800-53 controls ... and params too, almost

Now correcting target type

New examples conforming to almost-ready new profile model

Updating terminology

More corrections

Towards new profile model

Merging now working

More adjustments to merge

Merge now includes provenance tracer

Updates to 'implementation layer' XML mockups

Updated examples to new profile schema

Updating HTML production pipeline

Updating tools, Schematrons

Rearranging to collapse paranoid temp directory

Correction to recursive import logic

Adjustments and updates

More corrections and improvements to profile/resolution/validation

Finishing modify implementation under new design

Updates to Merge-Diagrams.html

Adjustments to profiling spec docs

Refinements of enhancing "implementation" XML including RNC schema

Docs; cleanup

SVGs for all mini-testing examples (testing batch production)

Updated working/FedRAMP with some refinements

More misc

Create CONTRIBUTING.md

Create USERS.md

Minor adjustments prior to more major adjustments

Putting up obfuscated and unobfuscated ISO 27002

Putting up obfuscated and unobfuscated ISO 27002

Putting up obfuscated and unobfuscated ISO 27002

Putting up obfuscated and unobfuscated ISO 27002

Next-next gen profile resolver, who default results are much simpler --

More adjustments to profile semantics

Added a link to the current active project

Add files via upload

Initial cut and placeholder for Profile Semantics docs

Minor revs

More various edits

More adjustments

More revisions to profile semantics docs

Mostly more diagrams

More touchups to docs

More touchups to docs

Correction to top-level readme

Sketching implementation layer in XML

oXygen schema documentation

Touches

New page suggesting regrouping logic

Tweaking

More touchups and corrections

Had to rename the spec document

New examples conforming to almost-ready new profile model

Updating terminology

More corrections

Towards new profile model

Merging now working

More adjustments to merge

Merge now includes provenance tracer

Updated examples to new profile schema

Updating HTML production pipeline

Updating tools, Schematrons

Fixed examples.

Adjustments and updates

Updates to Merge-Diagrams.html

Adjustments to profiling spec docs

Refinements of enhancing "implementation" XML including RNC schema

Updated working/FedRAMP with some refinements

More misc

Minor adjustments prior to more major adjustments

Aligning sprint-6

Initial cut and placeholder for Profile Semantics docs

Minor revs

More various edits

More adjustments

More revisions to profile semantics docs

Mostly more diagrams

More touchups to docs

More touchups to docs

Correction to top-level readme

Sketching implementation layer in XML

oXygen schema documentation

Touches

New page suggesting regrouping logic

Tweaking

More touchups and corrections

Now indexing to SP800-53 controls ... and params too, almost

Now correcting target type

New examples conforming to almost-ready new profile model

Updating terminology

More corrections

Towards new profile model

Merging now working

More adjustments to merge

Merge now includes provenance tracer

Updates to 'implementation layer' XML mockups

Updated examples to new profile schema

Updating HTML production pipeline

Updating tools, Schematrons

Rearranging to collapse paranoid temp directory

Correction to recursive import logic

Adjustments and updates

More corrections and improvements to profile/resolution/validation

Finishing modify implementation under new design

Updates to Merge-Diagrams.html

Adjustments to profiling spec docs

Refinements of enhancing "implementation" XML including RNC schema

Docs; cleanup

SVGs for all mini-testing examples (testing batch production)

Updated working/FedRAMP with some refinements

More misc

Create CONTRIBUTING.md

Create USERS.md

Minor adjustments prior to more major adjustments

Putting up obfuscated and unobfuscated ISO 27002

Putting up obfuscated and unobfuscated ISO 27002

Putting up obfuscated and unobfuscated ISO 27002

Putting up obfuscated and unobfuscated ISO 27002

Next-next gen profile resolver, who default results are much simpler --

More adjustments to profile semantics

Aligning sprint-6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants