Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Relocate schema resources #191

Merged
merged 2 commits into from
Mar 20, 2022
Merged

Relocate schema resources #191

merged 2 commits into from
Mar 20, 2022

Conversation

david-waltermire
Copy link
Collaborator

@david-waltermire david-waltermire commented Mar 5, 2022 •
edited
Loading

Committer Notes

Relocating Metaschema resources to the schema folder, moving them out of the XSLT implementation.

  • Fixed test suite
  • Normalized Metaschema type names to be full-hyphenated.
  • Cleanup of junk files

All Submissions:

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?
  • Have you squashed any non-relevant commits and commit messages? [instructions]
  • Do all automated CI/CD checks pass?

Changes to Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you included examples of how to use your new feature(s)?
  • Have you updated all website](https://pages.nist.gov/metaschema) and readme documentation affected by the changes you made? Changes to the website can be made in the website/content directory of your branch.

@david-waltermire david-waltermire changed the base branch from master to develop March 5, 2022 22:50
aj-stein-nist added a commit to aj-stein-nist/Issue1127Example that referenced this pull request Mar 8, 2022
@aj-stein-nist
Update testing from morning session re usnistgov/metaschema#191.
f8b5d4d
This was referenced Mar 9, 2022
Closed
Closed
Closed
This was linked to issues Mar 10, 2022
Roll back token datatype to avoid problematic character references #181
Closed
Double-escaped RegEx patterns in output JSON schema causing issues with some RegEx flavors #182
Closed
@david-waltermire david-waltermire mentioned this pull request Mar 10, 2022
Closed
@david-waltermire david-waltermire linked an issue Mar 10, 2022 that may be closed by this pull request
Whitespace collapse on datatypes in XSD foils bi-directional conversion (Metaschema M4) #68
Closed
@david-waltermire david-waltermire mentioned this pull request Mar 10, 2022
Closed
@david-waltermire david-waltermire mentioned this pull request Mar 15, 2022
Merged
8 tasks
@wendellpiez wendellpiez mentioned this pull request Mar 17, 2022
Closed
3 tasks
@david-waltermire david-waltermire force-pushed the relocate-schema-resources branch from 84bf205 to b831755 Compare March 17, 2022 20:57
@david-waltermire
Copy link
Collaborator Author

#195 contains a summary of the changes to the metaschema.xsd in this PR.

@david-waltermire david-waltermire marked this pull request as ready for review March 17, 2022 21:25
@david-waltermire
Moved schema resources out of XSLT implementation
ecc2051
@david-waltermire
Relocated XSD and datatype XML schemas.
4bf9ddb 
Removed junk and generated files.
Many datatype adjustments
- Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956.
- Adjusted type names of Metaschema types to be purely hyphenated.
- Removed extra character ranges that are causing C# problems.
- Adjusted URI data type names to be more consistent between XML and JSON.
Many metaschema.xsd adjustments
- Alignment of data types used with new Metschema datatype names.
Additional repairs of tests, including naming consistency.
- adding new unit test for a valid json-value key using a label
- many fixes to broken tests
- Removed duplicate uuid test scenario.
- Many fixes to make unit test Metaschemas valid.
Added JSON schema schema.
Some schema generation refactoring to support new data-driven test harness.
@david-waltermire david-waltermire force-pushed the relocate-schema-resources branch from b831755 to 4bf9ddb Compare March 20, 2022 19:06
@david-waltermire david-waltermire changed the base branch from develop to feature-metaschema-relocation-plus-enhancements March 20, 2022 19:13
@david-waltermire david-waltermire merged commit be3a452 into usnistgov:feature-metaschema-relocation-plus-enhancements Mar 20, 2022
@david-waltermire david-waltermire deleted the relocate-schema-resources branch March 20, 2022 19:17
@david-waltermire david-waltermire mentioned this pull request Mar 29, 2022
Merged
8 tasks
@david-waltermire david-waltermire linked an issue Mar 29, 2022 that may be closed by this pull request
Glitch in (unused) feature on json-value-key #169
Closed
david-waltermire added a commit that referenced this pull request May 2, 2022
@david-waltermire
Relocate schema resources (#191)
d5ba5c0 
- Moved schema resources out of XSLT implementation
- Relocated XSD and datatype XML schemas.
Removed junk and generated files.
Many datatype adjustments
   - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956.
   - Adjusted type names of Metaschema types to be purely hyphenated.
   - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127.
- Adjusted URI data type names to be more consistent between XML and JSON.
Many metaschema.xsd adjustments
- Alignment of data types used with new Metschema datatype names.
- Additional repairs of tests, including naming consistency.
   - adding new unit test for a valid json-value key using a label
   - many fixes to broken tests
   - Removed duplicate uuid test scenario.
   - Many fixes to make unit test Metaschemas valid.
- Added JSON schema schema.
- Some schema generation refactoring to support new data-driven test harness.
david-waltermire added a commit that referenced this pull request Jun 21, 2022
@david-waltermire
Relocate schema resources (#191)
d851905 
- Moved schema resources out of XSLT implementation
- Relocated XSD and datatype XML schemas.
Removed junk and generated files.
Many datatype adjustments
   - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956.
   - Adjusted type names of Metaschema types to be purely hyphenated.
   - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127.
- Adjusted URI data type names to be more consistent between XML and JSON.
Many metaschema.xsd adjustments
- Alignment of data types used with new Metschema datatype names.
- Additional repairs of tests, including naming consistency.
   - adding new unit test for a valid json-value key using a label
   - many fixes to broken tests
   - Removed duplicate uuid test scenario.
   - Many fixes to make unit test Metaschemas valid.
- Added JSON schema schema.
- Some schema generation refactoring to support new data-driven test harness.
This was referenced Jul 6, 2022
Merged
Closed
david-waltermire added a commit that referenced this pull request Jul 14, 2022
@david-waltermire @wendellpiez
Metaschema / XSLT implementation alignment (#197)
1c9ed1f 
* Relocate schema resources (#191)

- Moved schema resources out of XSLT implementation
- Relocated XSD and datatype XML schemas.
Removed junk and generated files.
Many datatype adjustments
   - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956.
   - Adjusted type names of Metaschema types to be purely hyphenated.
   - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127.
- Adjusted URI data type names to be more consistent between XML and JSON.
Many metaschema.xsd adjustments
- Alignment of data types used with new Metschema datatype names.
- Additional repairs of tests, including naming consistency.
   - adding new unit test for a valid json-value key using a label
   - many fixes to broken tests
   - Removed duplicate uuid test scenario.
   - Many fixes to make unit test Metaschemas valid.
- Added JSON schema schema.
- Some schema generation refactoring to support new data-driven test harness.

* Fixed URI of choice unit test.

* Update metaschema-datatypes.xsd

Adjusting whitespace handling for URI types.

* adding leading and trailing whitespace checks to all XML schema datatype derived types

* Adjustments to debug charstrings.

* Update metaschema-datatypes.json

Minor adjustments to data types for comments on #1260.

* refactored metaschema schema to support external constraint definitions

* Added constraint extensibility configuration.

* added formal-name and description to definition references

* XSLT M4 Metaschema pipeline updates (#214)

* Composition unit tests now valid to updated schemas; added (missing) tests.

* Adding XSpec for schema generation; nominal schema target examples

* Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing

* Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts
* More details in json-value-key readme; updated top-level JSON schema generation xspec
* Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic
* Cleaned up extra JSON Schema file
* Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups
* Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed.
* Updated anthology ('menagerie') metaschema example, with cleanup
* In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema
* Updating schema TODO notes on synching unit tests; bit of cleanup
* Addressing #212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups)
* Updating top-level composition unit tests
* .gitignore covering HTML reports from test runs
* Removing outdated testing results
* Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252)
* Restoring deprecated datatypes per #195
* Bit of cleanup; utility maintenance
* Removing 'INFO' level comment in XSD
* Tweakage to align XSD out (dropping comment; cleaning up namespaces)
* Patching hole in JSON datatype assignment also
* Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema
* Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types
* Extending atomic data type acquisition to collect one level of indirect references
* Cleaning up obsolete datatype support
* Bit more cleanup; updated readme
* Adjusting handling of warnings and exception messages in pipelines
* Removing outdated artifacts from schema generation unit testing.md

Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
david-waltermire added a commit that referenced this pull request Dec 7, 2022
@david-waltermire @wendellpiez
Metaschema / XSLT implementation alignment (#197)
abe0ad2 
* Relocate schema resources (#191)

- Moved schema resources out of XSLT implementation
- Relocated XSD and datatype XML schemas.
Removed junk and generated files.
Many datatype adjustments
   - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956.
   - Adjusted type names of Metaschema types to be purely hyphenated.
   - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127.
- Adjusted URI data type names to be more consistent between XML and JSON.
Many metaschema.xsd adjustments
- Alignment of data types used with new Metschema datatype names.
- Additional repairs of tests, including naming consistency.
   - adding new unit test for a valid json-value key using a label
   - many fixes to broken tests
   - Removed duplicate uuid test scenario.
   - Many fixes to make unit test Metaschemas valid.
- Added JSON schema schema.
- Some schema generation refactoring to support new data-driven test harness.

* Fixed URI of choice unit test.

* Update metaschema-datatypes.xsd

Adjusting whitespace handling for URI types.

* adding leading and trailing whitespace checks to all XML schema datatype derived types

* Adjustments to debug charstrings.

* Update metaschema-datatypes.json

Minor adjustments to data types for comments on #1260.

* refactored metaschema schema to support external constraint definitions

* Added constraint extensibility configuration.

* added formal-name and description to definition references

* XSLT M4 Metaschema pipeline updates (#214)

* Composition unit tests now valid to updated schemas; added (missing) tests.

* Adding XSpec for schema generation; nominal schema target examples

* Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing

* Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts
* More details in json-value-key readme; updated top-level JSON schema generation xspec
* Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic
* Cleaned up extra JSON Schema file
* Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups
* Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed.
* Updated anthology ('menagerie') metaschema example, with cleanup
* In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema
* Updating schema TODO notes on synching unit tests; bit of cleanup
* Addressing #212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups)
* Updating top-level composition unit tests
* .gitignore covering HTML reports from test runs
* Removing outdated testing results
* Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252)
* Restoring deprecated datatypes per #195
* Bit of cleanup; utility maintenance
* Removing 'INFO' level comment in XSD
* Tweakage to align XSD out (dropping comment; cleaning up namespaces)
* Patching hole in JSON datatype assignment also
* Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema
* Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types
* Extending atomic data type acquisition to collect one level of indirect references
* Cleaning up obsolete datatype support
* Bit more cleanup; updated readme
* Adjusting handling of warnings and exception messages in pipelines
* Removing outdated artifacts from schema generation unit testing.md

Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
aj-stein-nist pushed a commit to aj-stein-nist/metaschema that referenced this pull request Jan 9, 2023
@david-waltermire @wendellpiez @aj-stein-nist
Metaschema / XSLT implementation alignment (usnistgov#197)
6a534ac 
* Relocate schema resources (usnistgov#191)

- Moved schema resources out of XSLT implementation
- Relocated XSD and datatype XML schemas.
Removed junk and generated files.
Many datatype adjustments
   - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956.
   - Adjusted type names of Metaschema types to be purely hyphenated.
   - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127.
- Adjusted URI data type names to be more consistent between XML and JSON.
Many metaschema.xsd adjustments
- Alignment of data types used with new Metschema datatype names.
- Additional repairs of tests, including naming consistency.
   - adding new unit test for a valid json-value key using a label
   - many fixes to broken tests
   - Removed duplicate uuid test scenario.
   - Many fixes to make unit test Metaschemas valid.
- Added JSON schema schema.
- Some schema generation refactoring to support new data-driven test harness.

* Fixed URI of choice unit test.

* Update metaschema-datatypes.xsd

Adjusting whitespace handling for URI types.

* adding leading and trailing whitespace checks to all XML schema datatype derived types

* Adjustments to debug charstrings.

* Update metaschema-datatypes.json

Minor adjustments to data types for comments on #1260.

* refactored metaschema schema to support external constraint definitions

* Added constraint extensibility configuration.

* added formal-name and description to definition references

* XSLT M4 Metaschema pipeline updates (usnistgov#214)

* Composition unit tests now valid to updated schemas; added (missing) tests.

* Adding XSpec for schema generation; nominal schema target examples

* Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing

* Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts
* More details in json-value-key readme; updated top-level JSON schema generation xspec
* Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic
* Cleaned up extra JSON Schema file
* Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups
* Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed.
* Updated anthology ('menagerie') metaschema example, with cleanup
* In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema
* Updating schema TODO notes on synching unit tests; bit of cleanup
* Addressing usnistgov#212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups)
* Updating top-level composition unit tests
* .gitignore covering HTML reports from test runs
* Removing outdated testing results
* Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252)
* Restoring deprecated datatypes per usnistgov#195
* Bit of cleanup; utility maintenance
* Removing 'INFO' level comment in XSD
* Tweakage to align XSD out (dropping comment; cleaning up namespaces)
* Patching hole in JSON datatype assignment also
* Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema
* Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types
* Extending atomic data type acquisition to collect one level of indirect references
* Cleaning up obsolete datatype support
* Bit more cleanup; updated readme
* Adjusting handling of warnings and exception messages in pipelines
* Removing outdated artifacts from schema generation unit testing.md

Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
aj-stein-nist pushed a commit to aj-stein-nist/metaschema that referenced this pull request Jan 10, 2023
@david-waltermire @wendellpiez @aj-stein-nist
Metaschema / XSLT implementation alignment (usnistgov#197)
749fb19 
* Relocate schema resources (usnistgov#191)

- Moved schema resources out of XSLT implementation
- Relocated XSD and datatype XML schemas.
Removed junk and generated files.
Many datatype adjustments
   - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956.
   - Adjusted type names of Metaschema types to be purely hyphenated.
   - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127.
- Adjusted URI data type names to be more consistent between XML and JSON.
Many metaschema.xsd adjustments
- Alignment of data types used with new Metschema datatype names.
- Additional repairs of tests, including naming consistency.
   - adding new unit test for a valid json-value key using a label
   - many fixes to broken tests
   - Removed duplicate uuid test scenario.
   - Many fixes to make unit test Metaschemas valid.
- Added JSON schema schema.
- Some schema generation refactoring to support new data-driven test harness.

* Fixed URI of choice unit test.

* Update metaschema-datatypes.xsd

Adjusting whitespace handling for URI types.

* adding leading and trailing whitespace checks to all XML schema datatype derived types

* Adjustments to debug charstrings.

* Update metaschema-datatypes.json

Minor adjustments to data types for comments on #1260.

* refactored metaschema schema to support external constraint definitions

* Added constraint extensibility configuration.

* added formal-name and description to definition references

* XSLT M4 Metaschema pipeline updates (usnistgov#214)

* Composition unit tests now valid to updated schemas; added (missing) tests.

* Adding XSpec for schema generation; nominal schema target examples

* Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing

* Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts
* More details in json-value-key readme; updated top-level JSON schema generation xspec
* Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic
* Cleaned up extra JSON Schema file
* Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups
* Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed.
* Updated anthology ('menagerie') metaschema example, with cleanup
* In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema
* Updating schema TODO notes on synching unit tests; bit of cleanup
* Addressing usnistgov#212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups)
* Updating top-level composition unit tests
* .gitignore covering HTML reports from test runs
* Removing outdated testing results
* Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252)
* Restoring deprecated datatypes per usnistgov#195
* Bit of cleanup; utility maintenance
* Removing 'INFO' level comment in XSD
* Tweakage to align XSD out (dropping comment; cleaning up namespaces)
* Patching hole in JSON datatype assignment also
* Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema
* Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types
* Extending atomic data type acquisition to collect one level of indirect references
* Cleaning up obsolete datatype support
* Bit more cleanup; updated readme
* Adjusting handling of warnings and exception messages in pipelines
* Removing outdated artifacts from schema generation unit testing.md

Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
aj-stein-nist pushed a commit to aj-stein-nist/metaschema that referenced this pull request Jan 10, 2023
@david-waltermire @wendellpiez @aj-stein-nist
Metaschema / XSLT implementation alignment (usnistgov#197)
3f12dab 
* Relocate schema resources (usnistgov#191)

- Moved schema resources out of XSLT implementation
- Relocated XSD and datatype XML schemas.
Removed junk and generated files.
Many datatype adjustments
   - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956.
   - Adjusted type names of Metaschema types to be purely hyphenated.
   - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127.
- Adjusted URI data type names to be more consistent between XML and JSON.
Many metaschema.xsd adjustments
- Alignment of data types used with new Metschema datatype names.
- Additional repairs of tests, including naming consistency.
   - adding new unit test for a valid json-value key using a label
   - many fixes to broken tests
   - Removed duplicate uuid test scenario.
   - Many fixes to make unit test Metaschemas valid.
- Added JSON schema schema.
- Some schema generation refactoring to support new data-driven test harness.

* Fixed URI of choice unit test.

* Update metaschema-datatypes.xsd

Adjusting whitespace handling for URI types.

* adding leading and trailing whitespace checks to all XML schema datatype derived types

* Adjustments to debug charstrings.

* Update metaschema-datatypes.json

Minor adjustments to data types for comments on #1260.

* refactored metaschema schema to support external constraint definitions

* Added constraint extensibility configuration.

* added formal-name and description to definition references

* XSLT M4 Metaschema pipeline updates (usnistgov#214)

* Composition unit tests now valid to updated schemas; added (missing) tests.

* Adding XSpec for schema generation; nominal schema target examples

* Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing

* Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts
* More details in json-value-key readme; updated top-level JSON schema generation xspec
* Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic
* Cleaned up extra JSON Schema file
* Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups
* Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed.
* Updated anthology ('menagerie') metaschema example, with cleanup
* In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema
* Updating schema TODO notes on synching unit tests; bit of cleanup
* Addressing usnistgov#212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups)
* Updating top-level composition unit tests
* .gitignore covering HTML reports from test runs
* Removing outdated testing results
* Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252)
* Restoring deprecated datatypes per usnistgov#195
* Bit of cleanup; utility maintenance
* Removing 'INFO' level comment in XSD
* Tweakage to align XSD out (dropping comment; cleaning up namespaces)
* Patching hole in JSON datatype assignment also
* Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema
* Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types
* Extending atomic data type acquisition to collect one level of indirect references
* Cleaning up obsolete datatype support
* Bit more cleanup; updated readme
* Adjusting handling of warnings and exception messages in pipelines
* Removing outdated artifacts from schema generation unit testing.md

Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
aj-stein-nist pushed a commit to aj-stein-nist/metaschema that referenced this pull request Jan 10, 2023
@david-waltermire @wendellpiez @aj-stein-nist
Metaschema / XSLT implementation alignment (usnistgov#197)
cb04e9d 
* Relocate schema resources (usnistgov#191)

- Moved schema resources out of XSLT implementation
- Relocated XSD and datatype XML schemas.
Removed junk and generated files.
Many datatype adjustments
   - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956.
   - Adjusted type names of Metaschema types to be purely hyphenated.
   - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127.
- Adjusted URI data type names to be more consistent between XML and JSON.
Many metaschema.xsd adjustments
- Alignment of data types used with new Metschema datatype names.
- Additional repairs of tests, including naming consistency.
   - adding new unit test for a valid json-value key using a label
   - many fixes to broken tests
   - Removed duplicate uuid test scenario.
   - Many fixes to make unit test Metaschemas valid.
- Added JSON schema schema.
- Some schema generation refactoring to support new data-driven test harness.

* Fixed URI of choice unit test.

* Update metaschema-datatypes.xsd

Adjusting whitespace handling for URI types.

* adding leading and trailing whitespace checks to all XML schema datatype derived types

* Adjustments to debug charstrings.

* Update metaschema-datatypes.json

Minor adjustments to data types for comments on #1260.

* refactored metaschema schema to support external constraint definitions

* Added constraint extensibility configuration.

* added formal-name and description to definition references

* XSLT M4 Metaschema pipeline updates (usnistgov#214)

* Composition unit tests now valid to updated schemas; added (missing) tests.

* Adding XSpec for schema generation; nominal schema target examples

* Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing

* Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts
* More details in json-value-key readme; updated top-level JSON schema generation xspec
* Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic
* Cleaned up extra JSON Schema file
* Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups
* Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed.
* Updated anthology ('menagerie') metaschema example, with cleanup
* In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema
* Updating schema TODO notes on synching unit tests; bit of cleanup
* Addressing usnistgov#212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups)
* Updating top-level composition unit tests
* .gitignore covering HTML reports from test runs
* Removing outdated testing results
* Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252)
* Restoring deprecated datatypes per usnistgov#195
* Bit of cleanup; utility maintenance
* Removing 'INFO' level comment in XSD
* Tweakage to align XSD out (dropping comment; cleaning up namespaces)
* Patching hole in JSON datatype assignment also
* Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema
* Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types
* Extending atomic data type acquisition to collect one level of indirect references
* Cleaning up obsolete datatype support
* Bit more cleanup; updated readme
* Adjusting handling of warnings and exception messages in pipelines
* Removing outdated artifacts from schema generation unit testing.md

Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
aj-stein-nist pushed a commit to aj-stein-nist/metaschema that referenced this pull request Jan 10, 2023
@david-waltermire @wendellpiez @aj-stein-nist
Metaschema / XSLT implementation alignment (usnistgov#197)
106d463 
* Relocate schema resources (usnistgov#191)

- Moved schema resources out of XSLT implementation
- Relocated XSD and datatype XML schemas.
Removed junk and generated files.
Many datatype adjustments
   - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956.
   - Adjusted type names of Metaschema types to be purely hyphenated.
   - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127.
- Adjusted URI data type names to be more consistent between XML and JSON.
Many metaschema.xsd adjustments
- Alignment of data types used with new Metschema datatype names.
- Additional repairs of tests, including naming consistency.
   - adding new unit test for a valid json-value key using a label
   - many fixes to broken tests
   - Removed duplicate uuid test scenario.
   - Many fixes to make unit test Metaschemas valid.
- Added JSON schema schema.
- Some schema generation refactoring to support new data-driven test harness.

* Fixed URI of choice unit test.

* Update metaschema-datatypes.xsd

Adjusting whitespace handling for URI types.

* adding leading and trailing whitespace checks to all XML schema datatype derived types

* Adjustments to debug charstrings.

* Update metaschema-datatypes.json

Minor adjustments to data types for comments on #1260.

* refactored metaschema schema to support external constraint definitions

* Added constraint extensibility configuration.

* added formal-name and description to definition references

* XSLT M4 Metaschema pipeline updates (usnistgov#214)

* Composition unit tests now valid to updated schemas; added (missing) tests.

* Adding XSpec for schema generation; nominal schema target examples

* Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing

* Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts
* More details in json-value-key readme; updated top-level JSON schema generation xspec
* Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic
* Cleaned up extra JSON Schema file
* Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups
* Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed.
* Updated anthology ('menagerie') metaschema example, with cleanup
* In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema
* Updating schema TODO notes on synching unit tests; bit of cleanup
* Addressing usnistgov#212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups)
* Updating top-level composition unit tests
* .gitignore covering HTML reports from test runs
* Removing outdated testing results
* Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252)
* Restoring deprecated datatypes per usnistgov#195
* Bit of cleanup; utility maintenance
* Removing 'INFO' level comment in XSD
* Tweakage to align XSD out (dropping comment; cleaning up namespaces)
* Patching hole in JSON datatype assignment also
* Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema
* Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types
* Extending atomic data type acquisition to collect one level of indirect references
* Cleaning up obsolete datatype support
* Bit more cleanup; updated readme
* Adjusting handling of warnings and exception messages in pipelines
* Removing outdated artifacts from schema generation unit testing.md

Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
aj-stein-nist pushed a commit to aj-stein-nist/metaschema that referenced this pull request Jan 10, 2023
@david-waltermire @wendellpiez @aj-stein-nist
Metaschema / XSLT implementation alignment (usnistgov#197)
a2b511c 
* Relocate schema resources (usnistgov#191)

- Moved schema resources out of XSLT implementation
- Relocated XSD and datatype XML schemas.
Removed junk and generated files.
Many datatype adjustments
   - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956.
   - Adjusted type names of Metaschema types to be purely hyphenated.
   - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127.
- Adjusted URI data type names to be more consistent between XML and JSON.
Many metaschema.xsd adjustments
- Alignment of data types used with new Metschema datatype names.
- Additional repairs of tests, including naming consistency.
   - adding new unit test for a valid json-value key using a label
   - many fixes to broken tests
   - Removed duplicate uuid test scenario.
   - Many fixes to make unit test Metaschemas valid.
- Added JSON schema schema.
- Some schema generation refactoring to support new data-driven test harness.

* Fixed URI of choice unit test.

* Update metaschema-datatypes.xsd

Adjusting whitespace handling for URI types.

* adding leading and trailing whitespace checks to all XML schema datatype derived types

* Adjustments to debug charstrings.

* Update metaschema-datatypes.json

Minor adjustments to data types for comments on #1260.

* refactored metaschema schema to support external constraint definitions

* Added constraint extensibility configuration.

* added formal-name and description to definition references

* XSLT M4 Metaschema pipeline updates (usnistgov#214)

* Composition unit tests now valid to updated schemas; added (missing) tests.

* Adding XSpec for schema generation; nominal schema target examples

* Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing

* Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts
* More details in json-value-key readme; updated top-level JSON schema generation xspec
* Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic
* Cleaned up extra JSON Schema file
* Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups
* Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed.
* Updated anthology ('menagerie') metaschema example, with cleanup
* In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema
* Updating schema TODO notes on synching unit tests; bit of cleanup
* Addressing usnistgov#212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups)
* Updating top-level composition unit tests
* .gitignore covering HTML reports from test runs
* Removing outdated testing results
* Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252)
* Restoring deprecated datatypes per usnistgov#195
* Bit of cleanup; utility maintenance
* Removing 'INFO' level comment in XSD
* Tweakage to align XSD out (dropping comment; cleaning up namespaces)
* Patching hole in JSON datatype assignment also
* Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema
* Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types
* Extending atomic data type acquisition to collect one level of indirect references
* Cleaning up obsolete datatype support
* Bit more cleanup; updated readme
* Adjusting handling of warnings and exception messages in pipelines
* Removing outdated artifacts from schema generation unit testing.md

Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
david-waltermire added a commit that referenced this pull request Mar 9, 2023
@david-waltermire @wendellpiez
Metaschema / XSLT implementation alignment (#197)
c3bbfc2 
* Relocate schema resources (#191)

- Moved schema resources out of XSLT implementation
- Relocated XSD and datatype XML schemas.
Removed junk and generated files.
Many datatype adjustments
   - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956.
   - Adjusted type names of Metaschema types to be purely hyphenated.
   - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127.
- Adjusted URI data type names to be more consistent between XML and JSON.
Many metaschema.xsd adjustments
- Alignment of data types used with new Metschema datatype names.
- Additional repairs of tests, including naming consistency.
   - adding new unit test for a valid json-value key using a label
   - many fixes to broken tests
   - Removed duplicate uuid test scenario.
   - Many fixes to make unit test Metaschemas valid.
- Added JSON schema schema.
- Some schema generation refactoring to support new data-driven test harness.

* Fixed URI of choice unit test.

* Update metaschema-datatypes.xsd

Adjusting whitespace handling for URI types.

* adding leading and trailing whitespace checks to all XML schema datatype derived types

* Adjustments to debug charstrings.

* Update metaschema-datatypes.json

Minor adjustments to data types for comments on #1260.

* refactored metaschema schema to support external constraint definitions

* Added constraint extensibility configuration.

* added formal-name and description to definition references

* XSLT M4 Metaschema pipeline updates (#214)

* Composition unit tests now valid to updated schemas; added (missing) tests.

* Adding XSpec for schema generation; nominal schema target examples

* Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing

* Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts
* More details in json-value-key readme; updated top-level JSON schema generation xspec
* Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic
* Cleaned up extra JSON Schema file
* Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups
* Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed.
* Updated anthology ('menagerie') metaschema example, with cleanup
* In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema
* Updating schema TODO notes on synching unit tests; bit of cleanup
* Addressing #212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups)
* Updating top-level composition unit tests
* .gitignore covering HTML reports from test runs
* Removing outdated testing results
* Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252)
* Restoring deprecated datatypes per #195
* Bit of cleanup; utility maintenance
* Removing 'INFO' level comment in XSD
* Tweakage to align XSD out (dropping comment; cleaning up namespaces)
* Patching hole in JSON datatype assignment also
* Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema
* Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types
* Extending atomic data type acquisition to collect one level of indirect references
* Cleaning up obsolete datatype support
* Bit more cleanup; updated readme
* Adjusting handling of warnings and exception messages in pipelines
* Removing outdated artifacts from schema generation unit testing.md

Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
david-waltermire added a commit that referenced this pull request Mar 29, 2023
@david-waltermire @wendellpiez
Metaschema / XSLT implementation alignment (#197)
7e03773 
* Relocate schema resources (#191)

- Moved schema resources out of XSLT implementation
- Relocated XSD and datatype XML schemas.
Removed junk and generated files.
Many datatype adjustments
   - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956.
   - Adjusted type names of Metaschema types to be purely hyphenated.
   - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127.
- Adjusted URI data type names to be more consistent between XML and JSON.
Many metaschema.xsd adjustments
- Alignment of data types used with new Metschema datatype names.
- Additional repairs of tests, including naming consistency.
   - adding new unit test for a valid json-value key using a label
   - many fixes to broken tests
   - Removed duplicate uuid test scenario.
   - Many fixes to make unit test Metaschemas valid.
- Added JSON schema schema.
- Some schema generation refactoring to support new data-driven test harness.

* Fixed URI of choice unit test.

* Update metaschema-datatypes.xsd

Adjusting whitespace handling for URI types.

* adding leading and trailing whitespace checks to all XML schema datatype derived types

* Adjustments to debug charstrings.

* Update metaschema-datatypes.json

Minor adjustments to data types for comments on #1260.

* refactored metaschema schema to support external constraint definitions

* Added constraint extensibility configuration.

* added formal-name and description to definition references

* XSLT M4 Metaschema pipeline updates (#214)

* Composition unit tests now valid to updated schemas; added (missing) tests.

* Adding XSpec for schema generation; nominal schema target examples

* Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing

* Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts
* More details in json-value-key readme; updated top-level JSON schema generation xspec
* Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic
* Cleaned up extra JSON Schema file
* Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups
* Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed.
* Updated anthology ('menagerie') metaschema example, with cleanup
* In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema
* Updating schema TODO notes on synching unit tests; bit of cleanup
* Addressing #212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups)
* Updating top-level composition unit tests
* .gitignore covering HTML reports from test runs
* Removing outdated testing results
* Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252)
* Restoring deprecated datatypes per #195
* Bit of cleanup; utility maintenance
* Removing 'INFO' level comment in XSD
* Tweakage to align XSD out (dropping comment; cleaning up namespaces)
* Patching hole in JSON datatype assignment also
* Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema
* Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types
* Extending atomic data type acquisition to collect one level of indirect references
* Cleaning up obsolete datatype support
* Bit more cleanup; updated readme
* Adjusting handling of warnings and exception messages in pipelines
* Removing outdated artifacts from schema generation unit testing.md

Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
david-waltermire added a commit that referenced this pull request Mar 29, 2023
@david-waltermire @wendellpiez
Metaschema / XSLT implementation alignment (#197)
4a8ffc4 
* Relocate schema resources (#191)

- Moved schema resources out of XSLT implementation
- Relocated XSD and datatype XML schemas.
Removed junk and generated files.
Many datatype adjustments
   - Tweeks to schema data types to address unicode issues. Resolves usnistgov/OSCAL#1127. Resolves usnistgov/OSCAL#956.
   - Adjusted type names of Metaschema types to be purely hyphenated.
   - Removed extra character ranges that are causing C# problems. Resolves usnistgov/OSCAL#1127.
- Adjusted URI data type names to be more consistent between XML and JSON.
Many metaschema.xsd adjustments
- Alignment of data types used with new Metschema datatype names.
- Additional repairs of tests, including naming consistency.
   - adding new unit test for a valid json-value key using a label
   - many fixes to broken tests
   - Removed duplicate uuid test scenario.
   - Many fixes to make unit test Metaschemas valid.
- Added JSON schema schema.
- Some schema generation refactoring to support new data-driven test harness.

* Fixed URI of choice unit test.

* Update metaschema-datatypes.xsd

Adjusting whitespace handling for URI types.

* adding leading and trailing whitespace checks to all XML schema datatype derived types

* Adjustments to debug charstrings.

* Update metaschema-datatypes.json

Minor adjustments to data types for comments on #1260.

* refactored metaschema schema to support external constraint definitions

* Added constraint extensibility configuration.

* added formal-name and description to definition references

* XSLT M4 Metaschema pipeline updates (#214)

* Composition unit tests now valid to updated schemas; added (missing) tests.

* Adding XSpec for schema generation; nominal schema target examples

* Adding XSpec tests (testing schema generation as transforms) and initial set of targets for testing

* Major reorganization and archiving (some temporary, prior to defenestration); new tests; readme documents in each folder for schema-generation unit tests to help trace efforts
* More details in json-value-key readme; updated top-level JSON schema generation xspec
* Metaschema Schematron rule intercepting a json-key setting with no BY_KEY in the grouping logic
* Cleaned up extra JSON Schema file
* Updated metaschema model wrt json-key, json-value-key flag-ref (no longer flag-name); other Metaschema touchups
* Rewiring and simplifying XSD production pipeline with datatypes now acquired from the normative metaschema /schema subdirectory - removed the namespace fixup step, no longer needed.
* Updated anthology ('menagerie') metaschema example, with cleanup
* In XSD generation, now emitting datatype definitions only for datatypes actually needed by a schema
* Updating schema TODO notes on synching unit tests; bit of cleanup
* Addressing #212 - XML to JSON converter no longer chokes on edge cases (array items of SINGLETON_OR_ARRAY groups)
* Updating top-level composition unit tests
* .gitignore covering HTML reports from test runs
* Removing outdated testing results
* Adding back support for old datatype names as indicated in metaschema.xsd (cf line 1252)
* Restoring deprecated datatypes per #195
* Bit of cleanup; utility maintenance
* Removing 'INFO' level comment in XSD
* Tweakage to align XSD out (dropping comment; cleaning up namespaces)
* Patching hole in JSON datatype assignment also
* Emitting cleaner namespaces in XSD; slight refactoring of JSON Schema
* Adjusting JSON Schema type definitions to produce valid constraints over Metaschema atomic types
* Extending atomic data type acquisition to collect one level of indirect references
* Cleaning up obsolete datatype support
* Bit more cleanup; updated readme
* Adjusting handling of warnings and exception messages in pipelines
* Removing outdated artifacts from schema generation unit testing.md

Co-authored-by: Wendell Piez <wendell.piez@nist.gov>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wendellpiez wendellpiez Awaiting requested review from wendellpiez

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
1 participant
@david-waltermire