Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/all valid test #896

Open
wants to merge 27 commits into
base: develop
Choose a base branch
from
Open

Feature/all valid test #896

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
1e19a2a
all valid test
wandmagic Nov 13, 2024
f382bd5
make ssp all valid
wandmagic Nov 13, 2024
e91eed1
update oscal version
wandmagic Nov 13, 2024
56464d5
Update fedramp_extensions.feature
wandmagic Nov 13, 2024
6211011
New oscal-js oscal-cli versions in OCI image spec
aj-stein-gsa Nov 13, 2024
e3b9676
Add ADR to formally deprecate Spock versioning, formalize Semantic Ve…
aj-stein-gsa Nov 14, 2024
03e79c2
Update fedramp_extensions.feature
wandmagic Nov 14, 2024
98fcff0
Update module.mk
wandmagic Nov 14, 2024
b1fdefb
add integration npm commands
wandmagic Nov 14, 2024
9652ef6
Add `Prepared For` Constraints (#869)
Gabeblis Nov 14, 2024
49cc0bf
Add system characteristics href-target constraints
Gabeblis Nov 5, 2024
2d4e11a
delete file
Gabeblis Nov 5, 2024
5bc7dc6
Add validation check for diagram document directly in @href
Gabeblis Nov 7, 2024
c6da668
rephrase messages
Gabeblis Nov 7, 2024
753b400
Update fedramp-external-constraints.xml
wandmagic Nov 14, 2024
ec9273b
get latest ssp-all valid
wandmagic Nov 14, 2024
deb1fd4
all valid test
wandmagic Nov 13, 2024
b67d4c7
make ssp all valid
wandmagic Nov 13, 2024
2df64a4
update oscal version
wandmagic Nov 13, 2024
646eb73
Update fedramp_extensions.feature
wandmagic Nov 13, 2024
bd4ee17
Update fedramp_extensions.feature
wandmagic Nov 14, 2024
4283e07
Update module.mk
wandmagic Nov 14, 2024
184b625
add integration npm commands
wandmagic Nov 14, 2024
4f93113
Update fedramp-external-constraints.xml
wandmagic Nov 14, 2024
f7ad94e
get latest ssp-all valid
wandmagic Nov 14, 2024
e80d90d
Update fedramp_extensions.feature
wandmagic Nov 14, 2024
1a646c5
Merge branch 'feature/all-valid-test' of https://github.com/wandmagic…
wandmagic Nov 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
234 changes: 122 additions & 112 deletions features/fedramp_extensions.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,128 @@ Feature: OSCAL Document Constraints
@style-guide
Scenario Outline: Validating OSCAL constraints with metaschema constraints
Then I should verify that all constraints follow the style guide constraint


@integration
Scenario Outline: Documents that should be valid are pass
Then I should have valid results "<valid_file>"
Examples:
| valid_file |
| ssp-all-VALID.xml |
# | ../../../content/awesome-cloud/xml/AwesomeCloudSSP1.xml |
# | ../../../content/awesome-cloud/xml/AwesomeCloudSSP2.xml |

@full-coverage
Scenario: Preparing constraint coverage analysis
Given I have loaded all Metaschema extensions documents
And I have collected all YAML test files in the test directory
When I extract all constraint IDs from the Metaschema extensions
And I analyze the YAML test files for each constraint ID

@full-coverage
Scenario Outline: Ensuring full test coverage for "<constraint_id>"
Then I should have both FAIL and PASS tests for constraint ID "<constraint_id>"
Examples:
| constraint_id |
#BEGIN_DYNAMIC_CONSTRAINT_IDS
| address-type |
| attachment-type |
| authorization-type |
| categorization-has-correct-system-attribute |
| categorization-has-information-type-id |
| cia-impact-has-adjustment-justification |
| cia-impact-has-selected |
| cloud-service-model |
| component-type |
| control-implementation-status |
| data-center-alternate |
| data-center-count |
| data-center-country-code |
| data-center-primary |
| data-center-us |
| deployment-model |
| fedramp-version |
| fully-operational-date-is-valid |
| fully-operational-date-type |
| has-authenticator-assurance-level |
| has-authorization-boundary-diagram |
| has-authorization-boundary-diagram-caption |
| has-authorization-boundary-diagram-description |
| has-authorization-boundary-diagram-link |
| has-authorization-boundary-diagram-link-rel |
| has-authorization-boundary-diagram-link-rel-allowed-value |
| has-cloud-deployment-model |
| has-cloud-deployment-model-remarks |
| has-cloud-service-model |
| has-cloud-service-model-remarks |
| has-configuration-management-plan |
| has-data-flow |
| has-data-flow-description |
| has-data-flow-diagram |
| has-data-flow-diagram-caption |
| has-data-flow-diagram-description |
| has-data-flow-diagram-link |
| has-data-flow-diagram-link-rel |
| has-data-flow-diagram-link-rel-allowed-value |
| has-data-flow-diagram-uuid |
| has-federation-assurance-level |
| has-fully-operational-date |
| has-identity-assurance-level |
| has-incident-response-plan |
| has-information-system-contingency-plan |
| has-network-architecture |
| has-network-architecture-diagram |
| has-network-architecture-diagram-caption |
| has-network-architecture-diagram-description |
| has-network-architecture-diagram-link |
| has-network-architecture-diagram-link-rel |
| has-network-architecture-diagram-link-rel-allowed-value |
| has-published-date |
| has-rules-of-behavior |
| has-security-impact-level |
| has-security-sensitivity-level |
| has-separation-of-duties-matrix |
| has-system-id |
| has-system-name-short |
| has-user-guide |
| import-profile-has-available-document |
| import-profile-resolves-to-fedramp-content |
| information-type-800-60-v2r1 |
| information-type-has-availability-impact |
| information-type-has-confidentiality-impact |
| information-type-has-integrity-impact |
| information-type-system |
| interconnection-direction |
| interconnection-security |
| inventory-item-allows-authenticated-scan |
| inventory-item-public |
| inventory-item-virtual |
| marking |
| missing-response-components |
| party-has-name |
| privilege-level |
| prop-response-point-has-cardinality-one |
| resource-has-base64-or-rlink |
| resource-has-title |
| responsible-party-is-person |
| responsible-party-prepared-by |
| responsible-party-prepared-by-location-valid |
| role-defined-authorizing-official-poc |
| role-defined-information-system-security-officer |
| role-defined-prepared-by |
| role-defined-system-owner |
| scan-type |
| security-level |
| security-sensitivity-level-matches-security-impact-level |
| user-has-authorized-privilege |
| user-has-privilege-level |
| user-has-role-id |
| user-has-sensitivity-level |
| user-has-user-type |
| user-privilege-level |
| user-sensitivity-level |
| user-type |
#END_DYNAMIC_CONSTRAINT_IDS

@constraints
Scenario Outline: Validating OSCAL documents with metaschema constraints
Given I have Metaschema extensions documents
Expand Down Expand Up @@ -215,114 +336,3 @@ Examples:
| user-type-PASS.yaml |
#END_DYNAMIC_TEST_CASES

@full-coverage
Scenario: Preparing constraint coverage analysis
Given I have loaded all Metaschema extensions documents
And I have collected all YAML test files in the test directory
When I extract all constraint IDs from the Metaschema extensions
And I analyze the YAML test files for each constraint ID

@full-coverage
Scenario Outline: Ensuring full test coverage for "<constraint_id>"
Then I should have both FAIL and PASS tests for constraint ID "<constraint_id>"
Examples:
| constraint_id |
#BEGIN_DYNAMIC_CONSTRAINT_IDS
| address-type |
| attachment-type |
| authorization-type |
| categorization-has-correct-system-attribute |
| categorization-has-information-type-id |
| cia-impact-has-adjustment-justification |
| cia-impact-has-selected |
| cloud-service-model |
| component-type |
| control-implementation-status |
| data-center-alternate |
| data-center-count |
| data-center-country-code |
| data-center-primary |
| data-center-us |
| deployment-model |
| fedramp-version |
| fully-operational-date-is-valid |
| fully-operational-date-type |
| has-authenticator-assurance-level |
| has-authorization-boundary-diagram |
| has-authorization-boundary-diagram-caption |
| has-authorization-boundary-diagram-description |
| has-authorization-boundary-diagram-link |
| has-authorization-boundary-diagram-link-rel |
| has-authorization-boundary-diagram-link-rel-allowed-value |
| has-cloud-deployment-model |
| has-cloud-deployment-model-remarks |
| has-cloud-service-model |
| has-cloud-service-model-remarks |
| has-configuration-management-plan |
| has-data-flow |
| has-data-flow-description |
| has-data-flow-diagram |
| has-data-flow-diagram-caption |
| has-data-flow-diagram-description |
| has-data-flow-diagram-link |
| has-data-flow-diagram-link-rel |
| has-data-flow-diagram-link-rel-allowed-value |
| has-data-flow-diagram-uuid |
| has-federation-assurance-level |
| has-fully-operational-date |
| has-identity-assurance-level |
| has-incident-response-plan |
| has-information-system-contingency-plan |
| has-network-architecture |
| has-network-architecture-diagram |
| has-network-architecture-diagram-caption |
| has-network-architecture-diagram-description |
| has-network-architecture-diagram-link |
| has-network-architecture-diagram-link-rel |
| has-network-architecture-diagram-link-rel-allowed-value |
| has-published-date |
| has-rules-of-behavior |
| has-security-impact-level |
| has-security-sensitivity-level |
| has-separation-of-duties-matrix |
| has-system-id |
| has-system-name-short |
| has-user-guide |
| import-profile-has-available-document |
| import-profile-resolves-to-fedramp-content |
| information-type-800-60-v2r1 |
| information-type-has-availability-impact |
| information-type-has-confidentiality-impact |
| information-type-has-integrity-impact |
| information-type-system |
| interconnection-direction |
| interconnection-security |
| inventory-item-allows-authenticated-scan |
| inventory-item-public |
| inventory-item-virtual |
| marking |
| missing-response-components |
| party-has-name |
| privilege-level |
| prop-response-point-has-cardinality-one |
| resource-has-base64-or-rlink |
| resource-has-title |
| responsible-party-is-person |
| responsible-party-prepared-by |
| responsible-party-prepared-by-location-valid |
| role-defined-authorizing-official-poc |
| role-defined-information-system-security-officer |
| role-defined-prepared-by |
| role-defined-system-owner |
| scan-type |
| security-level |
| security-sensitivity-level-matches-security-impact-level |
| user-has-authorized-privilege |
| user-has-privilege-level |
| user-has-role-id |
| user-has-sensitivity-level |
| user-has-user-type |
| user-privilege-level |
| user-sensitivity-level |
| user-type |
#END_DYNAMIC_CONSTRAINT_IDS
Loading
Loading