Metaschema changes from #191 #195
Assignees
Labels
enhancement
New feature or request
XSLT Implementation
Issue relates to the XSLT implementation of Metaschema.
Milestone
Comments
david-waltermire
added
enhancement
wendellpiez
added a commit
to wendellpiez/metaschema
that referenced
this issue
Mar 22, 2022
wendellpiez
added a commit
to wendellpiez/metaschema
that referenced
this issue
Mar 23, 2022
8 tasks
8 tasks
This was referenced Apr 12, 2022
Open
david-waltermire
pushed a commit
that referenced
this issue
Apr 15, 2022
* Redefined lexical constraint on 'token' datatype - #181 - removing references to upper-Unicode characters that break at least one processor - this is not actually relaxed, since the exclusion tests out as inoperative (the offending characters are not in the sets from which they are being excluded) * Rolling back #165 addressing usnistgov/OSCAL#956. We now have no upper-Unicode characters in the schema to be concerned about. * Adding schema-generation unit tests for 'token' datatype * Adjusted JSON Schema generation to capture latest datatype definitions #195 * Debugging path in datatype integration * Adding XSD datatype production logic - reads and rewrites JSON definitions in XSD syntax. * Adjustment to JSON -> XSD rough casting * XSD datatypes adjusted to align with JSON #195 * Adding new datatypes as aliases for old names #1186 * Touchups to inline documentation (for propagation to tools) * Replaced with updated mapping plus adjustments to XSD production * Now doing a better job excluding unneeded datatype (simpleType) definitions * Reconciled datatype merge to emit functional JSON schemas * Removing safety backups; small correction to Metaschema Schematron to avoid a runtime error.
9 tasks
|
See #214 and its working branch for updates. Remaining on this Issue:
|
|
Yes we SHOULD restore older datatype names, at least until OSCAL metaschemas are all valid and checked (see #215). The |
|
@david-waltermire-nist apparently I have no permissions to check boxes but all the boxes above can be checked - everything here has been done and can be tested in the branch behind PR #214. |
wendellpiez
added a commit
to wendellpiez/metaschema
that referenced
this issue
Jul 11, 2022
Closed
3 tasks
david-waltermire
pushed a commit
to wendellpiez/metaschema
that referenced
this issue
Jul 14, 2022
david-waltermire
pushed a commit
that referenced
this issue
Jul 14, 2022
* Composition unit tests now valid to updated schemas; added (missing) tests.
* Adding XSpec for schema generation; nominal schema target examples
* Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing
* Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts
* More details in json-value-key readme; updated top-level JSON schema generation xspec
* Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic
* Cleaned up extra JSON Schema file
* Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups
* Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed.
* Updated anthology ('menagerie') metaschema example, with cleanup
* In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema
* Updating schema TODO notes on synching unit tests; bit of cleanup
* Addressing #212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups)
* Updating top-level composition unit tests
* .gitignore covering HTML reports from test runs
* Removing outdated testing results
* Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252)
* Restoring deprecated datatypes per #195
* Bit of cleanup; utility maintenance
* Removing 'INFO' level comment in XSD
* Tweakage to align XSD out (dropping comment; cleaning up namespaces)
* Patching hole in JSON datatype assignment also
* Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema
* Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types
* Extending atomic data type acquisition to collect one level of indirect references
* Cleaning up obsolete datatype support
* Bit more cleanup; updated readme
* Adjusting handling of warnings and exception messages in pipelines
* Removing outdated artifacts from schema generation unit testing.md
david-waltermire
added a commit
that referenced
this issue
Jul 14, 2022
* Relocate schema resources (#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing #212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per #195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
Repository owner
moved this from In Progress
to Done
in NIST OSCAL Work Board
Jul 14, 2022
david-waltermire
added a commit
that referenced
this issue
Dec 7, 2022
* Relocate schema resources (#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing #212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per #195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
aj-stein-nist
pushed a commit
to aj-stein-nist/metaschema
that referenced
this issue
Jan 9, 2023
* Relocate schema resources (usnistgov#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (usnistgov#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing usnistgov#212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per usnistgov#195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
aj-stein-nist
pushed a commit
to aj-stein-nist/metaschema
that referenced
this issue
Jan 10, 2023
* Relocate schema resources (usnistgov#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (usnistgov#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing usnistgov#212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per usnistgov#195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
aj-stein-nist
pushed a commit
to aj-stein-nist/metaschema
that referenced
this issue
Jan 10, 2023
* Relocate schema resources (usnistgov#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (usnistgov#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing usnistgov#212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per usnistgov#195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
aj-stein-nist
pushed a commit
to aj-stein-nist/metaschema
that referenced
this issue
Jan 10, 2023
* Relocate schema resources (usnistgov#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (usnistgov#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing usnistgov#212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per usnistgov#195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
aj-stein-nist
pushed a commit
to aj-stein-nist/metaschema
that referenced
this issue
Jan 10, 2023
* Relocate schema resources (usnistgov#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (usnistgov#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing usnistgov#212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per usnistgov#195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
aj-stein-nist
pushed a commit
to aj-stein-nist/metaschema
that referenced
this issue
Jan 10, 2023
* Relocate schema resources (usnistgov#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (usnistgov#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing usnistgov#212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per usnistgov#195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
david-waltermire
added a commit
that referenced
this issue
Mar 9, 2023
* Relocate schema resources (#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing #212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per #195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
david-waltermire
added a commit
that referenced
this issue
Mar 29, 2023
* Relocate schema resources (#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing #212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per #195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
david-waltermire
added a commit
that referenced
this issue
Mar 29, 2023
* Relocate schema resources (#191) - Moved schema resources out of XSLT implementation - Relocated XSD and datatype XML schemas. Removed junk and generated files. Many datatype adjustments - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956. - Adjusted type names of Metaschema types to be purely hyphenated. - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127. - Adjusted URI data type names to be more consistent between XML and JSON. Many metaschema.xsd adjustments - Alignment of data types used with new Metschema datatype names. - Additional repairs of tests, including naming consistency. - adding new unit test for a valid json-value key using a label - many fixes to broken tests - Removed duplicate uuid test scenario. - Many fixes to make unit test Metaschemas valid. - Added JSON schema schema. - Some schema generation refactoring to support new data-driven test harness. * Fixed URI of choice unit test. * Update metaschema-datatypes.xsd Adjusting whitespace handling for URI types. * adding leading and trailing whitespace checks to all XML schema datatype derived types * Adjustments to debug charstrings. * Update metaschema-datatypes.json Minor adjustments to data types for comments on #1260. * refactored metaschema schema to support external constraint definitions * Added constraint extensibility configuration. * added formal-name and description to definition references * XSLT M4 Metaschema pipeline updates (#214) * Composition unit tests now valid to updated schemas; added (missing) tests. * Adding XSpec for schema generation; nominal schema target examples * Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing * Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts * More details in json-value-key readme; updated top-level JSON schema generation xspec * Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic * Cleaned up extra JSON Schema file * Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups * Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed. * Updated anthology ('menagerie') metaschema example, with cleanup * In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema * Updating schema TODO notes on synching unit tests; bit of cleanup * Addressing #212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups) * Updating top-level composition unit tests * .gitignore covering HTML reports from test runs * Removing outdated testing results * Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252) * Restoring deprecated datatypes per #195 * Bit of cleanup; utility maintenance * Removing 'INFO' level comment in XSD * Tweakage to align XSD out (dropping comment; cleaning up namespaces) * Patching hole in JSON datatype assignment also * Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema * Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types * Extending atomic data type acquisition to collect one level of indirect references * Cleaning up obsolete datatype support * Bit more cleanup; updated readme * Adjusting handling of warnings and exception messages in pipelines * Removing outdated artifacts from schema generation unit testing.md Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
User Story:
As a user of the XSLT-based Metaschema toolchain, I want support for the latest Metaschema changes.
The following changes require updates to the XSLT-based toolchain.
schema/xml. These should be dynamically injected as part of XML schema generation.metaschema.xsdis the new metaschema XSDmetaschema-datatypes.xsdcontains all non-prose datatypesmetaschema-prose-base.xsdcontains all prose support types needed by line and multiline. These types are not duplicated in the two schemas below.metaschema-markup-line.xsdcontains only the XML types for markup linemetaschema-markup-multiline.xsdcontains only the XML types for markup multilineschema/json. These should be dynamically injected as part of JSON schema generation.metaschema-datatypes.jsoncontain all JSON data typesjson-value-key#169. They are split between:<json-value-key>flag-name</json-value-key><json-value-key-flag flag-ref="flag-name"/>**note the change inflag-nametoflag-ref.Goals:
Update the XSLT-based tooling to address these changes.
Dependencies:
None.
Acceptance Criteria
The text was updated successfully, but these errors were encountered: