Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't make dupe ADR 6, properly number ADR 8 #733

Merged
merged 4 commits into from
Sep 26, 2024
Merged

Don't make dupe ADR 6, properly number ADR 8 #733

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f5a2258
[skip ci] No dupe ADR 6, properly number ADR 8
aj-stein-gsa Sep 26, 2024
5db1b63
[skip ci] Rename file from ADR 6 -> ADR 8
aj-stein-gsa Sep 26, 2024
0e682c7
[skip ci] 0-pad ADR 7 filename to match others
aj-stein-gsa Sep 26, 2024
39213eb
Updated ADR #8 status
Gabeblis Sep 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
0 ...-signal-unsupportent-content-in-github.md → ...-signal-unsupportent-content-in-github.md
View file
Open in desktop
File renamed without changes.
8 changes: 4 additions & 4 deletions ...ts/adr/0006-test-data-optimization-ADR.md → documents/adr/0008-test-data-optimization.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# 6. Test data optimization ADR
Date: 2024-09-20
# 8. Test data optimization ADR
Date: 2024-09-26

## Status
Proposed
Accepted

## Context
Currently, all tests for constraints used by the constraint validation tool rely on the same invalid test data file (`ssp-all-INVALID.xml`). This results in conflicts between many constraints, preventing the tests from failing properly when using a shared invalid test data file. An example of this conflict is:
Expand All @@ -19,7 +19,7 @@ We have one constraint, `information-type-has-categorization`, that checks the e
2. Currently, the test runner has a feature where the positive (`-PASS.yaml`) and negative (`-FAIL.yaml`) tests are scaffolded for a given constraint by running `npm run constraint` in a terminal window. These yaml test files reference the corresponding test data files that the constraint should be validated against. We can add functionality to this existing feature to prompt a user if they would like to generate new invalid test data (`-all-INVALID.xml`) as well, or use the existing invalid test data file for that model (e.g., ssp-all-INVALID.xml). If a user chooses to generate new invalid test data, we can scaffold a new invalid test data file, specific to the constraint, that uses a lean template. This template would contain only the necessary context to validate the constraint, and schema validation would be disabled to ensure that the invalid test data is as lean as possible. The yaml test files can then automatically reference the newly generated invalid test data file to validate the constraint. If a user chooses to use the existing invalid test data file for that model, then the yaml test files will reference the existing invalid test data file for that model to validate the constraint. Adding this functionality to the `npm run constraint` command would make sure that all necessary content to validate a constraint is generated at the same time, and that the yaml test files reference the correct test data file. This solution would solve the problem of constraints not successfully passing the fail case when referencing the same invalid test data file by adding the flexbility to isolate the testing of a constraint by scaffolding a new invalid test data file specific to the constraint. The current method of validating the pass case of all constraints in the same valid test data file (`ssp-all-VALID.xml`) will not be changed or impacted by this solution.

## Decision
No decision has currently been made.
The FedRAMP Automation Team will move forward with Solution #2. The changes that implement this ADR can be found here: `https://github.com/GSA/fedramp-automation/pull/690`.

## Consequences
Solution 1:
Expand Down
Loading