Not possible to configure Axe rules with configuration as JSON

[Ant0n777]

Have you read the documentation?

• Yes, I've read the how to make a reproducable bug guide
• Yes, I've read the how to debug my script guide

URL

any

What are you trying to accomplish

I need to configure AXE accessibility rules to run. Specific set of rules (all WCAG rules + some Best Practices). I created config.json with

{
  "axe": {
    "run": {
      "runOnly": {
        "type": "tag",
        "values": ["wcag2a", "wcag2aa", "wcag21aa", "wcag22aa"]
      },
      "rules": {
        "empty-heading": { "enabled": true }
      }
    }
  }
}

But it does not work because config options with '-' symbols automatically replaced with camelCase. So, in this case "empty-heading" is replaced with "emptyHeading".
Almost all AXE rules are with dash(es). So, it's not possible to configure.

Same configuration but with rule that does not have '-' in the name works fine! For ex.:

      "rules": {
        "accesskeys": { "enabled": true }
      }

works fine.

What browser did you use?

Chrome

How to reproduce

- Create config.json to enable or disable AXE rule that has '-' in the Rule ID. For ex. "empty-heading":
{
  "axe": {
    "run": {
      "runOnly": {
        "type": "tag",
        "values": ["wcag2a", "wcag2aa", "wcag21aa", "wcag22aa"]
      },
      "rules": {
        "empty-heading": { "enabled": true }
      }
    }
  }
}
- Run sitespeed.io for any url using that config:
sitespeed.io -n 1 --multi --axe.enable ./test.mjs --config config.json

- Test hangs and fails with timeout because additional CamelCase "emptyHeading" rule added automatically to the AXE config.
[2024-07-30 18:07:03] INFO: Run AXE with run options {"runOnly":{"type":"tag","values":["wcag2a","wcag2aa","wcag21aa","wcag22aa"]},"rules":{"empty-heading":{"enabled":true},"emptyHeading":{"enabled":true}}}

Log output

$ sitespeed.io -n 1 --multi --axe.enable ./test.mjs --config config.json
[2024-07-30 18:06:47] INFO: Versions OS: win32 10.0.19045 nodejs: v20.15.1 sitespeed.io: 34.7.1 browsertime: 22.6.0 coach: 8.0.2
[2024-07-30 18:06:47] INFO: Axe version 4.9.1 plugin activated
[2024-07-30 18:06:47] INFO: Running tests using Chrome - 1 iteration(s)
[2024-07-30 18:06:48] INFO: Start to measure homepage
[2024-07-30 18:06:48] INFO: Navigating to url https://www.store.com/ iteration 1
[2024-07-30 18:07:01] INFO: Take after page complete check screenshot
[2024-07-30 18:07:02] INFO: Take cumulative layout shift screenshot
[2024-07-30 18:07:03] INFO: Take largest contentful paint screenshot
[2024-07-30 18:07:03] INFO: Configure AXE with {"enable":true}
[2024-07-30 18:07:03] INFO: Run AXE with run options {"runOnly":{"type":"tag","values":["wcag2a","wcag2aa","wcag21aa","wcag22aa"]},"rules":{"empty-heading":{"enabled":true},"emptyHeading":{"enabled":true}}}
[2024-07-30 18:09:03] ERROR: Could not run the AXE script, no AXE information collected ScriptTimeoutError: script timeout
  (Session info: chrome=127.0.6533.73)
  (Session info: chrome=127.0.6533.73)
    at Object.throwDecodedError (C:\Users\username\AppData\Roaming\npm\node_modules\sitespeed.io\node_modules\selenium-webdriver\lib\error.js:521:15)
    at parseHttpResponse (C:\Users\username\AppData\Roaming\npm\node_modules\sitespeed.io\node_modules\selenium-webdriver\lib\http.js:514:13)
    at Executor.execute (C:\Users\username\AppData\Roaming\npm\node_modules\sitespeed.io\node_modules\selenium-webdriver\lib\http.js:446:28)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Driver.execute (C:\Users\username\AppData\Roaming\npm\node_modules\sitespeed.io\node_modules\selenium-webdriver\lib\webdriver.js:742:17)
    at async runAxe (C:\Users\username\AppData\Roaming\npm\node_modules\sitespeed.io\lib\plugins\axe\axePostScript.cjs:31:20)
    at async Measure.collect (file:///C:/Users/username/AppData/Roaming/npm/node_modules/sitespeed.io/node_modules/browsertime/lib/core/engine/command/measure.js:664:7)
    at async measureSimpleUrl (file:///C:/Users/username/Repos/Performance/Browser/accessibility/accessibility_storefront_dev.mjs:23:9)
    at async default (file:///C:/Users/username/Repos/Performance/Browser/accessibility/accessibility_storefront_dev.mjs:26:5)
    at async file:///C:/Users/username/AppData/Roaming/npm/node_modules/sitespeed.io/node_modules/browsertime/lib/core/engine/run.js:4:7
    at async Iteration.run (file:///C:/Users/username/AppData/Roaming/npm/node_modules/sitespeed.io/node_modules/browsertime/lib/core/engine/iteration.js:162:9)
    at async Engine.runByScript (file:///C:/Users/username/AppData/Roaming/npm/node_modules/sitespeed.io/node_modules/browsertime/lib/core/engine/index.js:308:20)
    at async analyzeUrl (file:///C:/Users/username/AppData/Roaming/npm/node_modules/sitespeed.io/lib/plugins/browsertime/analyzer.js:202:12)
    at async BrowsertimePlugin.processMessage (file:///C:/Users/username/AppData/Roaming/npm/node_modules/sitespeed.io/lib/plugins/browsertime/index.js:173:26)

[soulgalore]

Hi @Ant0n777 thanks for creating the issue. I pushed a fix in a PR but it needs to go in a major release since it will potentially change behaviour for some users (but the change makes sense).

[Ant0n777]

Thanks @soulgalore.
Could you please help me with some workaround for now?
I was thinking about updating contex.options.axe directly in the test script and remove duplicated (camelCase) rules from it.
Would be nice if you can give some example how to do that (I'm not so experienced in JS).

[soulgalore]

Let me think during the weekend, maybe there's a simpler smart way to do it.

[Ant0n777]

@soulgalore, I found the workaround:
update axe configuration directly in the test script and don't use config.json
Like this:

    context.options.axe = {
        enable: true,
        run: {
          runOnly: {
            type: "tag",
            values: [
              "wcag2a",
              "wcag2aa",
              "wcag21aa",
              "wcag22aa",
            ],
          },
          rules: {
            "aria-allowed-role": { enabled: true },
            "empty-heading": { enabled: true },
            "heading-order": { enabled: true },
            ...
          },
        },
      };