Investigate adding automated accname tests in wpt/webdriver

[jnurthen]

all the accname tests should be able to be rewritten to use webdriver.

[MelSumner]

ok! @accdc let's schedule some time to get on the same page about this and come up with an action plan. 👍

[cookiecrook]

Implementation test results:
https://wpt.fyi/results/webdriver/tests/get_computed_label/get.py?label=master&label=experimental&aligned

Source for those label and role tests:
https://github.com/web-platform-tests/wpt/blob/c478d0f4a1/webdriver/tests/get_computed_label/get.py
https://github.com/web-platform-tests/wpt/blob/c478d0f4a1/webdriver/tests/get_computed_role/get.py

[cookiecrook]

Implemented in Chromium and WebKit.

Gecko tracker: https://bugzilla.mozilla.org/show_bug.cgi?id=1788867

[cookiecrook]

Docs on running the WPT tests: https://github.com/web-platform-tests/wpt/blob/c478d0f4a1c3cd64a35b6f451679365dddbed8f5/README.md

[cookiecrook]

Taking assignment as this is likely to be part of the InterOp 2023 Accessibility Investigation.

[cookiecrook]

FWIW, the Mac configuration steps on the WPT site didn't work for me, so here's my variant of what worked.

Setting up a new Mac system to run WPT tests

1. In Terminal, make sure you're using the system variant of python3: % which python3 should return /usr/bin/python3
2. Download and install Safari Technology Preview.
3. Add the following to ~/.zprofile or wherever you define your shell path.

export PATH="$( xcrun python3 -m site --user-base )/bin:$PATH"

4. To get the change, open a new shell tab/window, or paste the same line into your current shell.
5. In Terminal, cd to a directory where you want the WPT source to live, then:

xcrun git clone https://github.com/web-platform-tests/wpt.git
xcrun pip3 install --user virtualenv
cd wpt
sudo /Applications/Safari\ Technology\ Preview.app/Contents/MacOS/safaridriver --enable
./wpt make-hosts-file | sudo tee -a /etc/hosts

6. In Safari Tech Preview, select Safari > Settings… > Advanced > "Show Develop menu in menu bar" then select Develop > Allow Remote Automation.
7. Attempt to run a single test from Terminal.

python3 ./wpt run --log-mach-level debug --log-mach - --webdriver-arg="--diagnose" safari /webdriver/tests/get_computed_label/get.py

You should get some percentage of passes and failures. The specific passes/failures should match the results on WPT.fyi for the same test.

Note: Watch out for double hyphens in CLI params. Depending on the text field you copy out of, autocorrect may convert them to en dashes (–) which can look identical in monospace.

[cookiecrook]

testdriver extension PR here: https://github.com/web-platform-tests/wpt/pull/38758/files

[cookiecrook]

Once that lands, you should be able to do:

python3 ./wpt run --log-mach-level debug --log-mach - --webdriver-arg="--diagnose" safari /accname/basic.html
python3 ./wpt run --log-mach-level debug --log-mach - --webdriver-arg="--diagnose" safari /aria/role/basic.html

[cookiecrook]

It should work in Chromium, too. Gecko has the prerequisite webdriver support planned for the April/May timeframe.

[cookiecrook]

Firefox patch here: https://bugzilla.mozilla.org/show_bug.cgi?id=1585622#c21 🎉

[cookiecrook]

Work continues in the following:

• Discuss how to break up the computedlabel tests in wpt/accname (to verify AccName computation) #190

Should we close this AccName #174 issue as completed?

[cookiecrook]

I think this is done now... There are currently 382 AccName tests among the other WPT Accessibility tests. Closing.