Add non-experimental axe tests. #1768

robdodson · 2017-02-23T21:33:35Z

Big PR I know :P

This adds most of the axe tests with a few exceptions:

I removed some that seemed low priority, like testing for the <blink> tag and some stuff around image maps
I'm not using any of their experimental tests, though I think that would be cool at some point. I think it'd be nice to add an option to enable/disable experimental tests someday.
I'm not using their best-practice tests. Same as above, I'd like to add these someday but want to give folks an option to enable/disable. The exception to this is the tabindex test which is a simple test, and was already checked into lighthouse so I figure it's ok to leave it in place.

paulirish · 2017-02-24T01:35:00Z

Screenshot of the report payload:

paulirish · 2017-02-24T01:48:03Z

lighthouse-core/audits/accessibility/accesskeys.js

+      name: 'accesskeys',
+      description: 'All accesskeys in the document are unique.',
+      helpText: '`accesskey` attributes allow the user to quickly activiate or focus part ' +
+          'of the page.' +


need an extra space

ebidel · 2017-02-24T04:15:10Z

Whoa that is HUUUUUUGE. Not sure what to do about it, but if 1-more audits fails that whole thing will be open. That'll probably be the case for most sites.

paulirish · 2017-02-24T04:18:42Z

First up. This was some work! nice job wrangling it all together. I really like getting the helptext quickly as I don't understand all of these too well. :)

A few ideas and questions..

Combine into fewer audits

This is a larger issue.. but I think we can combine a bunch of these into fewer audits. As a result we won't have a wall of text when the Accessibility section is there. (as in the above screenshot)

So in this approach. You have a group like "Elements applying ARIA follow correct practices". Then if there is a failure, you open it up and see the results grouped by element. (Basically a list of elements and associated axe violations)

I spent some time trying to group them. I also ran the tests on a bunch of sites to make sure the ones that will fail a lot have a clear top-level description. So here's my proposal ....

Color contrast of elements is satisfactory
- "accessibility/color-contrast",
Elements are well structured
- "accessibility/definition-list",
  "accessibility/dlitem",
  "accessibility/duplicate-id",
  "accessibility/list",
  "accessibility/listitem",
Elements avoid incorrect use of attributes
- "accessibility/accesskeys",
  "accessibility/audio-caption",
  "accessibility/image-alt",
  "accessibility/input-image-alt",
  "accessibility/tabindex",
  "accessibility/td-headers-attr",
  "accessibility/th-has-data-cells",
Elements applying ARIA follow correct practices
- "accessibility/aria-allowed-attr",
  "accessibility/aria-required-attr",
  "accessibility/aria-required-children",
  "accessibility/aria-required-parent",
  "accessibility/aria-roles",
  "accessibility/aria-valid-attr-value",
  "accessibility/aria-valid-attr",
Elements describe their contents well
- "accessibility/document-title",
  "accessibility/frame-title",
  "accessibility/label",
  "accessibility/layout-table",
  "accessibility/object-alt",
  "accessibility/video-caption",
  "accessibility/video-description",
  "accessibility/bypass",
The page's language is specified and valid.
- "accessibility/html-has-lang",
  "accessibility/html-lang-valid",
  "accessibility/valid-lang",
The document does not use <meta http-equiv="refresh">
- "accessibility/meta-refresh",
The <meta name="viewport"> element follows best practices.
- "accessibility/meta-viewport",
Links and buttons have discernable names.
- "accessibility/link-name",
  "accessibility/button-name",

Top level aggregation

I'm working on a separate proposal around overall IA, but part of that proposal is that Accessibility breaks out into a separate top-level Aggregation. (So we'd have: PWA, Security, Accessibility, Performance, Best Practices). These would also be the selectable with checkboxes for people to only get what they want.

Language

I notice we don't always use language consistently. Note how we are referring to elements and attribute values:

We need to be consistent with uses of "All" and "Every" or the lackthereof. (I think the last option is probably best: [role] values are valid) ((I'm really keen on using the [attributename] convention.))

bypass question

is the bypass check satisfied by any <hN> element? If so, let's put "heading" first in the description before "skip link". :) (bypass is also the one audit i'm not entirely happy with my above taxomony with, but lmk wyt)

Audit feedback

Can you add a note on the lang ones that they accept valid BCP 47 tags?

Can you add something like this to the image-alt helptext?

"Ensure all informative elements have short, descriptive alternate text and all decorative elements have empty alt attributes (alt="")

Admittedly both of these changes won't belong in the helptext necessarily if we do the above combining.

that's a lot but that's what i'm thinkin.. wdyt?

paulirish · 2017-03-03T02:41:06Z

FYI My proposed new IA for the report includes the above categorization for accessibility

robdodson · 2017-03-21T02:14:04Z

I updated the help text language to be more consistent.

I may bug @ebidel or @brendankenny tomorrow when we're all back in the office to get their help combining the audits

robdodson · 2017-03-21T19:24:11Z

OK I've added the aggregations. Here's what they look like (I closed the dropdowns in this screenshot)

robdodson · 2017-03-21T19:24:46Z

@paulirish I think this is ready for review again, whenever you have a chance.

ebidel · 2017-03-21T20:30:23Z

lighthouse-core/audits/accessibility/accesskeys.js

+      category: 'Accessibility',
+      name: 'accesskeys',
+      description: '`[accesskey]` values are unique.',
+      helpText: '`accesskey` attributes allow the user to quickly activiate or focus part ' +


ebidel · 2017-03-21T20:30:57Z

lighthouse-core/audits/accessibility/accesskeys.js

@@ -0,0 +1,44 @@
+/**
+ * @license
+ * Copyright 2016 Google Inc. All rights reserved.


2017 for these new files

ebidel · 2017-03-21T20:34:04Z

lighthouse-core/audits/accessibility/aria-required-children.js

+'use strict';
+
+/**
+ * @fileoverview Ensures elements with an ARIA role that require child roles contain them.


This sentence was hard to parse. Something like "Ensures elements contain a child role when their parent ARIA role requires it."

ebidel · 2017-03-21T20:48:25Z

lighthouse-core/audits/accessibility/aria-required-parent.js

+'use strict';
+
+/**
+ * @fileoverview Ensures elements with an ARIA role that require parent roles are contained by them.


same comment

ebidel · 2017-03-21T20:54:38Z

lighthouse-core/audits/accessibility/input-image-alt.js

+      category: 'Accessibility',
+      name: 'input-image-alt',
+      description: '`<input type="image">` elements have `[alt]` text.',
+      helpText: 'When an image is being used as an `<input>` button providing alternative text ' +


ebidel · 2017-03-21T21:10:25Z

lighthouse-core/audits/accessibility/valid-lang.js

+      category: 'Accessibility',
+      name: 'valid-lang',
+      description: '`[lang]` attributes have a valid value.',
+      helpText: 'Specifying a valid BCP 47 language on elements helps ensure that text is ' +


add a link: [BCP 47 language](LINK?)

ebidel · 2017-03-21T21:10:44Z

lighthouse-core/audits/accessibility/video-caption.js

+'use strict';
+
+/**
+ * @fileoverview Ensures every HTML document has a `lang` attribute.


ebidel

LGTM. One more thing.

ebidel · 2017-03-22T18:35:53Z

lighthouse-core/audits/accessibility/button-name.js

+    return {
+      category: 'Accessibility',
+      name: 'button-name',
+      description: 'buttons have an accessible name.',


spotted this one. The lower case looks weird:

How about buttons -> `<button>`s

paulirish · 2017-03-22T18:53:33Z

🎱🍐✨✨!

paulirish reviewed Feb 24, 2017

View reviewed changes

robdodson force-pushed the add-a11y-tests branch 2 times, most recently from 021a04f to 9c4c463 Compare March 21, 2017 02:37

ebidel suggested changes Mar 21, 2017

View reviewed changes

ebidel suggested changes Mar 22, 2017

View reviewed changes

robdodson added 7 commits March 22, 2017 11:38

Add non-experimental axe tests.

d609321

More consistent language in descriptions

f421253

Roll a11y tests into buckets

1807992

Update copyrights, comments

dad2296

Fixup copyrights

656a62f

Fixup tests

6c348ea

Capitalize "buttons"

a39dff3

robdodson force-pushed the add-a11y-tests branch from c7ad289 to a39dff3 Compare March 22, 2017 18:38

ebidel approved these changes Mar 22, 2017

View reviewed changes

paulirish merged commit 2a12d5c into master Mar 22, 2017

paulirish deleted the add-a11y-tests branch March 22, 2017 18:49

paulirish mentioned this pull request Oct 3, 2018

Clean up aXe audits #6169

Closed

connorjclark mentioned this pull request May 5, 2023

new_audit: td-has-header #15051

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add non-experimental axe tests. #1768

Add non-experimental axe tests. #1768

robdodson commented Feb 23, 2017

paulirish commented Feb 24, 2017

paulirish Feb 24, 2017

ebidel commented Feb 24, 2017

paulirish commented Feb 24, 2017

paulirish commented Mar 3, 2017

robdodson commented Mar 21, 2017

robdodson commented Mar 21, 2017

robdodson commented Mar 21, 2017

ebidel Mar 21, 2017

ebidel Mar 21, 2017

ebidel Mar 21, 2017

ebidel Mar 21, 2017

ebidel Mar 21, 2017

ebidel Mar 21, 2017

ebidel Mar 21, 2017

ebidel left a comment

ebidel Mar 22, 2017

paulirish commented Mar 22, 2017

Add non-experimental axe tests. #1768

Add non-experimental axe tests. #1768

Conversation

robdodson commented Feb 23, 2017

paulirish commented Feb 24, 2017

Choose a reason for hiding this comment

ebidel commented Feb 24, 2017

paulirish commented Feb 24, 2017

Combine into fewer audits

Top level aggregation

Language

bypass question

Audit feedback

paulirish commented Mar 3, 2017

robdodson commented Mar 21, 2017

robdodson commented Mar 21, 2017

robdodson commented Mar 21, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ebidel left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paulirish commented Mar 22, 2017