feat: expect(locator).toHaveAccessibleErrorMessage #33904

pengooseDev · 2024-12-07T12:54:35Z

Closes #31249

Add new toHaveAccessibleErrorMessage matcher for expect(locator).

Looking forward to teams feedback. Thanks! :)

packages/playwright-core/src/server/injected/roleUtils.ts

dgozman

This looks very good! Just a few comments about aria spec/semantics, and this will be good to do.

packages/playwright/src/matchers/matchers.ts

packages/playwright-core/src/server/injected/roleUtils.ts

dgozman

Thank you for the PR, looks very good!

dgozman · 2024-12-17T10:39:04Z

packages/playwright-core/src/server/injected/roleUtils.ts

+export const kInputValidityAttributes = ['required', 'pattern', 'min', 'max', 'minlength', 'maxlength', 'step', 'type', 'value'];
+
+export function hasValidationAttributes(element: Element): boolean {
+  if (!(element instanceof HTMLInputElement))


validity is defined for a lot of elements, e.g. <select>, <textarea>, <object>, <button>, <fieldset>, probably more. I'd drop this check, and assume that element.validity?.valid === false means it is invalid.

dgozman · 2024-12-17T10:39:30Z

packages/playwright-core/src/server/injected/roleUtils.ts

+  const role = getAriaRole(element) || '';
+  if (!role || !kAriaInvalidRoles.includes(role))
+    return 'false';
+  if (element instanceof HTMLInputElement && hasValidationAttributes(element) && element.validity)


Perhaps check both aria-invalid and element.validity?.valid === false, and consider the element invalid when either of the indicators is present?

I understood it as being asked to remove hasValidationAttributes and evaluate only with element.validity?.valid === false without assessing the kAriaInvalidRoles attributes. Would that be correct? :)

According to the spec, it seems that the return values of the getAriaInvalid function should be 'false' | 'true' | 'grammar' | 'spelling'.
Rather than evaluating validity within this function, I think it would be better to separate it into a dedicated function, such as getValidityInvalid. I look forward to your feedback on this! :)

packages/playwright-core/src/server/injected/roleUtils.ts

…essage references

…plementation

github-actions · 2024-12-22T14:21:41Z

Test results for "tests 1"

8 flaky

⚠️ [firefox-page] › page/page-evaluate.spec.ts:403:3 › should throw for too deep reference chain @firefox-ubuntu-22.04-node18
⚠️ [playwright-test] › ui-mode-test-watch.spec.ts:184:5 › should watch new file @ubuntu-latest-node20-1
⚠️ [webkit-library] › library/browsercontext-clearcookies.spec.ts:146:3 › should remove cookies by name and domain @webkit-ubuntu-22.04-node18
⚠️ [webkit-library] › library/browsercontext-har.spec.ts:529:3 › should ignore aborted requests @webkit-ubuntu-22.04-node18
⚠️ [webkit-library] › library/logger.spec.ts:34:3 › should log context-level @webkit-ubuntu-22.04-node18
⚠️ [webkit-library] › library/trace-viewer.spec.ts:149:1 › should open simple trace viewer @webkit-ubuntu-22.04-node18
⚠️ [webkit-page] › page/page-set-input-files.spec.ts:147:3 › should upload large file @webkit-ubuntu-22.04-node18
⚠️ [playwright-test] › ui-mode-test-watch.spec.ts:145:5 › should watch all @windows-latest-node18-1

37461 passed, 650 skipped
✔️✔️✔️

Merge workflow run.

dgozman

This looks great! Thank you for the PR! and following through the review process. Merging in.