Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/enforce help props #2

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
13e07fd
Add props to each constraint that has sufficient existing documentati…
Gabeblis Oct 18, 2024
1b69dca
Create style guide for FedRAMP OSCAL Constraints (#760)
aj-stein-gsa Oct 23, 2024
b44d5f0
introduce coverage for help text
wandmagic Oct 4, 2024
9614cd5
only need the help uri
wandmagic Oct 4, 2024
2e59e39
add help uri
wandmagic Oct 17, 2024
d719b94
introduce style guide constraint
wandmagic Oct 18, 2024
801420d
Update fedramp_extensions.feature
wandmagic Oct 18, 2024
682a134
Update fedramp_extensions.feature
wandmagic Oct 18, 2024
cd4b309
remove jsdom filtering
wandmagic Oct 18, 2024
c0c3c2f
improve constraint style checking
wandmagic Oct 18, 2024
f5ea95f
remove orphaned test
wandmagic Oct 21, 2024
383daf2
re introduce missing content
wandmagic Oct 21, 2024
9cdfb6d
require message for expect constraints
wandmagic Oct 21, 2024
0d04085
follow style guide more closely
wandmagic Oct 21, 2024
983bc2b
only validate fedramp constraints with the style guide
wandmagic Oct 21, 2024
6eb8a57
Add props to each constraint that has sufficient existing documentati…
Gabeblis Oct 18, 2024
078507e
re-introduce information-type-800-60-v2r1 & scaffold alphabetical con…
wandmagic Oct 22, 2024
da6262c
a fedramp help message MUST be all caps
wandmagic Oct 28, 2024
02e0066
rebase
wandmagic Oct 28, 2024
f8b658c
Merge pull request #16 from wandmagic/feature-constraints-on-constraints
wandmagic Oct 28, 2024
07d4084
Merge branch 'feature-constraints-style-guide' into feature/enforce-h…
wandmagic Oct 28, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions features/fedramp_extensions.feature
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
Feature: OSCAL Document Constraints

@full-coverage
Scenario Outline: Validating OSCAL constraints with metaschema constraints
Then I should verify that all constraints follow the style guide constraint

@constraints
Scenario Outline: Validating OSCAL documents with metaschema constraints
Given I have Metaschema extensions documents
Expand Down Expand Up @@ -109,8 +113,8 @@ Examples:
| has-system-id-PASS.yaml |
| has-user-guide-FAIL.yaml |
| has-user-guide-PASS.yaml |
| information-type-id-FAIL.yaml |
| information-type-id-PASS.yaml |
| information-type-800-60-v2r1-FAIL.yaml |
| information-type-800-60-v2r1-PASS.yaml |
| information-type-system-FAIL.yaml |
| information-type-system-PASS.yaml |
| interconnection-direction-FAIL.yaml |
Expand Down Expand Up @@ -160,6 +164,7 @@ And I analyze the YAML test files for each constraint ID

@full-coverage
Scenario Outline: Ensuring full test coverage for "<constraint_id>"
Given I have loaded all Metaschema extensions documents
Then I should have both FAIL and PASS tests for constraint ID "<constraint_id>"
Examples:
| constraint_id |
Expand Down
51 changes: 50 additions & 1 deletion features/steps/fedramp_extensions_steps.ts
Original file line number Diff line number Diff line change
Expand Up @@ -641,4 +641,53 @@ Then("I should have both FAIL and PASS tests for constraint ID {string}", functi
constraintId,
`Constraint ${constraintId} is not in the extracted constraints list`
);
});
});
Then('I should verify that all constraints follow the style guide constraint', async function () {
const baseDir = join(__dirname, '..', '..');
const constraintDir = join(baseDir, 'src', 'validations', 'constraints');
const styleGuidePath = join(baseDir, 'src', 'validations', 'styleguides', 'fedramp_constraint_style.xml');

const constraint_files = Array.from(new Set(getConstraintFiles().split("|").join("").split(" ").join("").trim().split("\n")));
const errors = [];

function filterOutBrackets(input) {
return input.replace(/\[.*?\]/g, '');
}

for (const file_name of constraint_files.filter(x=>!x.includes("oscal"))) {
const filePath = join(constraintDir, file_name.trim());
try {
const [result, error] = await executeOscalCliCommand('metaschema', [
'validate',
filePath,
'-c',
styleGuidePath,
'--disable-schema-validation'
]);

console.log(`Validation result for ${file_name}:`, result);
if (error) {
console.error(`Validation error for ${file_name}:`, error);
}

const filteredError = filterOutBrackets(error);
if (filteredError) {
errors.push(`Style guide validation failed for ${file_name}: ${filteredError}`);
}
if (result.includes("ERROR")) {
errors.push(`Style guide validation found errors in ${file_name}: ${result}`);
}
} catch (error) {
errors.push(`Error processing ${file_name}: ${error}`);
}
}

// Display all errors at the end
if (errors.length > 0) {
console.error("Validation errors found:");

throw new Error("Style guide validation failed."+errors.join("\n"));
}

expect(errors, "No style guide validation errors should be found").to.be.empty;
});
Loading
Loading