Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Round trip fixes (WIP) #413

Closed
wants to merge 8 commits into from

Conversation

wendellpiez
Copy link
Contributor

@wendellpiez wendellpiez commented Jun 7, 2019

Committer Notes

Addressing remaining issues in XML/JSON conversion. See #343, #410.

With this first PR, the SP800-53 catalog converts both ways with what appear to be whitespace discrepancies. The profiles still show lapses.

All Submissions:

  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?
  • Have you squashed any non-relevant commits and commit messages? [instructions]

Changes to Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you included examples of how to use your new feature(s)?

howieavp76 and others added 8 commits June 6, 2019 07:27
Remove some JSON testing custom code and pip installs for Python in favor of AJV-CLI.  Added XPath to the XML comparison code. Testing for XMLDIFF.

Remove Unnecessary Python Libraries

No longer using several external libraries

Minor Updates

FIxed git ignores
Changed from -v check to something more compatible

Updates to Testing

Remove some JSON testing custom code and pip installs for Python in favor of AJV-CLI.  Added XPath to the XML comparison code. Testing for XMLDIFF.

Minor Updates

FIxed git ignores
Changed from -v check to something more compatible

Bash Generalization

Fixed the -v statements to use a more portable -z statement.

Added Exit Code Logic to XML Round Trip Comparison

Refactor XML Comparison

Removes XMLDiff

Adds exit code to the Python code, does error count tracking

Test Content Converer Fix

XML RoundTrip in Run All Tests

Round Trip XML->JSON->XML

Working for all NIST 800-53 checks (FedRAMP currently bombing)

Color Output

Printf messages are now color coded for improved readability in large log files

Temp Directory

Round trip files now written to the Temp directory, this directory added to build artifacts in CI/CD

Rename Temporal Files

Build files for roundtrip now carry their base naming conventions for troubleshooting

Add Roundtrip Check to CI/CD

Initial Attempt

Fix CI/CD Bug

Typo on base image

Local JAR

Trying to get it working end to end with local file

Dynamic Paths

Fix CI/CD bug finding Python/Saxon scripts

PIP Installs

Python Import Error Fix

UTF Encoding

Refactor Python

Get etree.parse to work in CI/CD

Shorten FileNames

Build Artifacts

Use the supplied build directory

Troubleshoot Artifacts

Dynamic JAR

Dynamic JAR 2

Maven Install

Adds Maven to the Circle CI image

Maven Attempt #2

Class Path Refactor

Refactor CI/CD

Faster build time

CI/CD Bug Fix

Fix Path Error

Copy Build Artifacts

Copy Artifacts Attempt #2

Artifact Try #3

Artifact Attempt #4

Copy Step

Artifacts #5

Fix Tab

Artifact #6

Artifact #7

Artifact #8

Artifact #9

Initial JSON Roundtrip

Remove Diff

Updates to Testing

Remove some JSON testing custom code and pip installs for Python in favor of AJV-CLI.  Added XPath to the XML comparison code. Testing for XMLDIFF.

Minor Updates

FIxed git ignores
Changed from -v check to something more compatible

Updates to Testing

Remove some JSON testing custom code and pip installs for Python in favor of AJV-CLI.  Added XPath to the XML comparison code. Testing for XMLDIFF.

Bash Generalization

Fixed the -v statements to use a more portable -z statement.

Added Exit Code Logic to XML Round Trip Comparison

Refactor XML Comparison

Removes XMLDiff

Adds exit code to the Python code, does error count tracking

Test Content Converer Fix

XML RoundTrip in Run All Tests

Round Trip XML->JSON->XML

Working for all NIST 800-53 checks (FedRAMP currently bombing)

Color Output

Printf messages are now color coded for improved readability in large log files

Temp Directory

Round trip files now written to the Temp directory, this directory added to build artifacts in CI/CD

Rename Temporal Files

Build files for roundtrip now carry their base naming conventions for troubleshooting

Add Roundtrip Check to CI/CD

Initial Attempt

Fix CI/CD Bug

Typo on base image

Local JAR

Trying to get it working end to end with local file

Dynamic Paths

Fix CI/CD bug finding Python/Saxon scripts

PIP Installs

Python Import Error Fix

UTF Encoding

Refactor Python

Get etree.parse to work in CI/CD

Shorten FileNames

Build Artifacts

Use the supplied build directory

Troubleshoot Artifacts

Dynamic JAR

Dynamic JAR 2

Maven Install

Adds Maven to the Circle CI image

Maven Attempt #2

Class Path Refactor

CI/CD Bug Fix

Fix Path Error

Copy Build Artifacts

Copy Artifacts Attempt #2

Artifact Try #3

Artifact Attempt #4

Copy Step

Artifact #7

Artifact #8

Artifact #9

Initial JSON Roundtrip

Remove Diff

Merge with Master

CI/CD Fix #2

CI/CD Fix #4

CI/CD Fix #5

CI/CD Fix #6
- Added file naming conventions for XSL conversion templates
- Renamed schema files to follow the naming guidance
- Reorganized metaschema docuemntation
- Refactored scripts to produce files using correct names
- Fixed broken links in repo and website
- Fixed spelling errors and broken links
- Added markdown-link-checker to CI process to check PRs for broken links
- Added colorization to many build messages
- Added support for JSON min and YAML content generation from JSON files (using prettyprint)
… names.

Added a validation step for the round-tripped file.
@david-waltermire
Copy link
Contributor

I am managing our fixes through my branch. Please make your PRs against that. Closing this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants