Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add ability to always save test file #948

Merged
merged 2 commits into from
Jul 5, 2023

Conversation

DaveTryon
Copy link
Contributor

Details

#935 requests that we provide a way for CLI users to create an .a11ytest file even when no errors are reported. This adds a --alwayssavetestfile option to the CLI, then plumbs it through to take effect. People using the automation interface can access this directly via the Config.Builder.WithAlwaysSaveTestFile method.

Motivation

Address #935

Context

Pull request checklist

@DaveTryon DaveTryon requested a review from a team as a code owner July 4, 2023 04:50
@@ -16,7 +16,7 @@ You invoke the scanner by running `AxeWindowsCLI.exe`. By default, this tool get
If you run the tool without parameters, you'll be presented with the help screen. An example follows:

```
AxeWindowsCLI 2.1.1
AxeWindowsCLI 2.1.4
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Future note--we should bump this when we update package versions after each release. It would appear that we overlooked this in the last 3 releases ☹️

@@ -117,6 +120,15 @@ public Builder WithCustomUIAConfig(string path)
return this;
}

/// <summary>
/// Override the default behavior of only creating a11ytest files if errors are found.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// Override the default behavior of only creating a11ytest files if errors are found.
/// Configure Axe.Windows to always save an a11ytest file, even if no errors are found.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tweaked this a bit

@@ -55,9 +59,10 @@ verbosity|Identifies the level of detail you want in the output. Valid values ar
showThirdPartyNotices|If specified, displays the third party notices for components used by AxeWindowsCLI. This information is also available in the `thirdpartynotices.html` file that is installed with AxeWindowsCLI.
delayInSeconds|Optionally inserts a delay before triggering the scan. This allows transient controls (menus, drop-down-lists, etc.) to be scanned.
customUia|Optionally provides a path to a [custom UIA configuration file](../../docs/CustomUIA.md). By default, only system-defined UIA properties will be included in the scan.
alwayssavetestfile|Optionally causes the test file to always be saved. By defualt, the test file is saved only if errors are found.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
alwayssavetestfile|Optionally causes the test file to always be saved. By defualt, the test file is saved only if errors are found.
alwayssavetestfile|Optionally causes the test file to always be saved. By default, the test file is saved only if errors are found.

@codecov-commenter
Copy link

Codecov Report

Merging #948 (22945d3) into main (4fc19c3) will increase coverage by 1.04%.
The diff coverage is 79.13%.

❗ Current head 22945d3 differs from pull request most recent head 9dd0f0d. Consider uploading reports for the commit 9dd0f0d to get more accurate results

@@            Coverage Diff             @@
##             main     #948      +/-   ##
==========================================
+ Coverage   73.70%   74.74%   +1.04%     
==========================================
  Files         398      423      +25     
  Lines       12046    13053    +1007     
==========================================
+ Hits         8878     9757     +879     
- Misses       3168     3296     +128     
Impacted Files Coverage Δ
src/Actions/Actions/ControlPatternAction.cs 0.00% <0.00%> (ø)
src/Actions/Actions/CustomUIAAction.cs 0.00% <0.00%> (ø)
src/Actions/Actions/ListenAction.cs 0.00% <0.00%> (ø)
src/Actions/Actions/PrivacyExtensions.cs 100.00% <ø> (ø)
src/Actions/Actions/SetDataAction.cs 0.00% <0.00%> (ø)
...rc/Actions/Attributes/InteractionLevelAttribute.cs 0.00% <0.00%> (ø)
src/Actions/Misc/ExtensionMethods.cs 7.46% <0.00%> (-31.60%) ⬇️
src/Actions/Resources/ErrorMessages.Designer.cs 0.00% <ø> (-55.56%) ⬇️
src/Automation/AxeWindowsAutomationException.cs 33.33% <ø> (-33.34%) ⬇️
src/Automation/Data/OutputFile.cs 88.88% <ø> (ø)
... and 144 more

... and 241 files with indirect coverage changes

@DaveTryon DaveTryon merged commit 11982cb into microsoft:main Jul 5, 2023
@DaveTryon DaveTryon deleted the enable-always-save-test-file branch July 19, 2023 21:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants