Investigate solution for operating the file upload component with Dragon

[owenatgov]

What

Investigate if it's possible to enhance the file upload component in order for it to be operable via Dragon.

Why

This is in response to alphagov/govuk-design-system#4031. This was raised in a recent DAC audit that dictation software Dragon can't find the file upload button in the file upload component. More detail is available in that issue.

Timebox

1 week (preliminary)

Questions to answer

• Is DAC's proposed solution suitable?
• Are there other solutions?
• Can we make Dragon recognise <input type="file"> elements?

Done when

• Questions have been answered or we have a clearer idea of how to get to our goal
• Findings have been reviewed and agreed with at least one other person
• Findings have been shared, e.g: via a write-up on the ticket, at a show & tell or team meeting

[owenatgov]

Update

@querkmachine has built a prototype that expands on DAC's proposed solution that also allows us to control the content of the file upload component and adds a rough drag and drop feature.

We'll be testing this prototype against all assistive tech to assess effectiveness. If testing is successful, we'll next be roughly assessing the design to make sure it doesn't create a regression in usability with intent to eventually merge.

[owenatgov]

Following testing with some hard to analyse results (see comment by @selfthinker), we revisited the above solution and found the following:

When testing the solution together with the browser console open (Chrome), we noticed that when trying to click the file upload button with dragon that a warning was showing up in the console:

File chooser dialog can only be shown with a user activation

This was also true of us testing other similar live file upload variants such as the one used in the passport service (need to go through the first 3 questions to reach the custom file upload). We're confident that this is an example of transient activation. From the MDN page on features gated by user activation:

Transient activation is a window state that indicates a user has recently pressed a button, moved a mouse, used a menu, or performed some other user interaction. Transient activation expires after a timeout (if not renewed by further interaction) and may also be consumed by some APIs (like Window.open()).

What we think is happening is that Dragon interacting with elements isn't registering as an event to browsers such as a click or a keypress, therefore browsers are presuming that we're trying to open a users file explorer dialogue using javascript without an inciting interaction, which could be malicious code, so the simulated click is blocked with a warning. This is very difficult for us to get around without intervention by either browsers or Dragon to change how their software works.

The next steps for this are to:

1. investigate examples of other file upload components by large organisations to see if this problem persists
2. discuss this with DAC and get their input

[querkmachine]

A quick footnote that, when investigating examples from other organisations, we need to be sure they're using an input[type="file"]-based method, as this is a practical limitation of the Design System not being responsible for the underlying file upload and management process.

That said, the File System API—which would be the pure JavaScript method of building something like this—also prevents opening the file dialog until after user interaction, so would also be a non-starter option.

[owenatgov]

We're going to keep this issue open as we want to talk to DAC as mentioned in my previous comment and @selfthinker wants to test our solution with other assistive tech.

[selfthinker]

I've tested our solution with other assistive tech.
My detailed screen reader findings are in this spreadsheet and screenshots of testing colour changes are in this folder (both only visible internally).

Here is a summary of those findings:

The original file upload input is not hidden from screen reader users, so they will experience two different buttons doing the same thing. Keyboard-only users will also be able to tab to the invisible form input. Not sure if it's possible to hide it from tabbing as well as screen readers and it can still be used the way it is? I'm not sure of that because we're making direct use of the hidden input field. That's something we should try to answer before closing this investigation.

With the native file upload input, not all screen readers announce the chosen file (sometimes): JAWS and NVDA both with Chrome and Firefox, VO on macOS with Safari (Chrome seems to be fine).
That means that our solution can fix other problems with the native input. Our solution currently doesn't announce the file name, but I assume that's easy to add.

The words used are obviously different, so users won't get the same experience as with the native element. But I assume that is fine as long as the labels and announcements are clear.

I have also tested in Windows High Contrast Mode, changes to Firefox browser colours and used Chrome High Contrast extension. The differences are all fine.

Another difference I noticed is that the native field is highlighted after selecting a file and our version is not. That might be a usability thing we should probably add.

We don't need to fix any of this right now for the investigation to be complete. But we should try to answer the question if it is at all possible to let screen reader and keyboard-only users experience just one field and not two.

[selfthinker]

I've just tested DAC's proposed solution.
I've added their solution from alphagov/govuk-design-system#4031 into this JSbin.

That one works better than our version because there is no second button.
That at least means we know that it's possible which will help close the investigation.
I would still like to check out other file upload components, but don't expect to find much.

I also tested DAC's version with Dragon.
And, as expected, it also only works the first time Dragon is used, and every following interaction leads to the 'File chooser dialog can only be shown with a user activation' message.

[owenatgov]

I've updated the prototype with a new commit 9cfd3be

It's messy because of some local issues I had with typescript so that I was able to push the change, but this change adds aria-hidden="true" and tabindex="-1" to the input once everything is initialised. From some quick local testing, I can no longer access the native element with keyboard navigation or VO on Chrome. The enhancement otherwise still works. That feels positive to me, however I'd like @selfthinker to check it one more time with Dragon when you have a sec to double check this doesn't degrade the solution.

After that, I suggest we close this. If it works, we take this to DAC and then start working on a production solution if they're happy with it. If it doesn't, we drop the solution with a note.

[selfthinker]

I've also was just looking at what other file upload components behave like.
I tested everything in The Component Gallery that passed a first sniff test (which were 8) plus the multi file upload from MoJ.
As expected, either Dragon doesn't work at all with them or only works the first time with the same security message in the console.

[selfthinker]

I'd like @selfthinker to check it one more time with Dragon when you have a sec to double check this doesn't degrade the solution.

@owenatgov, just done that and can confirm it's the same experience as before. It works the first time.

[owenatgov]

Following @selfthinker's comment above and a chat off-github, we're going to close this investigation as we have all the answers we need. Whilst we're going to let DAC know about our findings, we're confident that our solution improves the experience for Dragon users even if it's imperfect and that it has numerous other benefits so we'll be pressing on to productionify our prototype.

[selfthinker]

Although this is closed now, I had an idea today that I wanted to try out which led to further interesting findings.
I thought I'd document them on here.

I had a great idea that turned out to theoretically work but practically not work.
Because the <summary> element can be made clickable by Dragon via adding a role=button, I thought the same would probably fix the file input as well and tested it out in a JSbin.

And it turns out, it works!
Meaning, when giving a role=button to an <input type=file />, Dragon activates it successfully when saying "click button". (Saying "choose file" doesn't work, though. )

But it doesn't work any subsequent time due to the same problem we encountered when testing all other solutions as we get the 'File chooser dialog can only be shown with a user activation' message in the console.

This is not considering yet if adding role=button would be a good, bad or neutral thing for screen reader users.

But this finding tells us that even if Nuance fixed Dragon to be able to use the file input, it would most probably still not work consistently.

I also found that there is a way to circumvent only being able to use it the first time.
I noticed one of DAC's testers often uses the 'mouse click' command to get the focus on the browser window.
When you say 'mouse click' in between clicking on the file input button, it keeps on working as I guess Chrome thinks there was some user activation between the clicks.