Keyboard Shortcuts

[gkatsev]

Criteria:

• by default don't bind globally, only when "the player" is in focus
• an option to bind keyboard shortcuts globally
• when focused on a particular control, that control's functionality takes precedent
  • if the control doesn't have functionality for the event, it should get delegate to the media-chrome keyboard shortcuts
• controls should expose a list of keys that they use
• Listen to media-controller or a provided element for keyup events that bubble up to it and handle the event appropriately
• allow the keyboard shortcuts to be configured (in the future, it would be good to allow viewers to configure these shortcuts for themselves)
• make sure that things still work in react and other frameworks
• default starting set of shortcuts:
  • k - toggle playback
  • space - toggle playback
  • m - toggle mute
  • f - toggle fullscreen
  • left arrow - seek back
  • right arrow - seek forward

---

Keyboard Shortcuts are an important accessibility feature that enables often-used functionality to be quickly available. In addition, since, media-chrome/things built with it are unlikely to be full page apps themselves, these shortcuts should only function when focus is inside the player, perhaps with an option to enable it to be on the entire page as well.

Currently, the keyboard functionality in media-chrome requires being focused on a particular button and then Enter or Space keys will let you interact with the button, in addition to any native behavior that's available (for example, the sliders automatically let the arrow keys change the current time). Ideally, the keyboard shortcuts will work no matter on where the focus is, assuming that the focused button doesn't have functionality for that button already. The biggest example of that is that the space key should toggle playback when just focused on the player itself, but when focused on a particular button, it should toggle the button instead.

Additionally, the media-controller element may itself be wrapped in a separate element that represents the "player", so we want to be able to handle the keyboard shortcuts on that element as well.

Options:

1. Do nothing. This is the base case and would require that media-chrome users implement it all themselves.
2. Implement keyboard shortcuts only on media-controller. This would mean that keyboard shortcuts won't be delegated when focused on a button and will only be available when focused on media-controller itself.
3. Implement it in media-chrome-button and media-controller and manually delegate the events.
4. Implement keyboard shortcuts on the media-controller via event bubbling.
   1. Expose a list of keys that the button uses
   2. Assume that the buttons will stopPropagation appropriately

I think that option 3 would be the best to implement Since we want the event delegation to happen based on the DOM structure and not based on the inheritance of the elements in the source. Though, I'm not currently sure how it'll behave in the react builds. I think I would lean towards 3.i because then each button or input doesn't necessarily need to make sure that stopPropagation is properly called depending on which buttons were pressed.

I don't have a good idea for how to add keyboard handling to a wrapper element, but I see a few potential solutions currently:

1. pass the element to listen to the keyboard shortcuts as an argument to media-chrome
2. "retrigger" the keyup event onto the media-controller element (this would require cloning the KeyboardEvent, as you can't redispatch these events directly).
3. trigger a new event on the media-controller with the expected behavior. The downside here is that it requires more duplicated logic.
   The benefit of 1 is that it's probably less code for users of media-chrome, but 2 and 3 will also help with a standalone control bar, which 1 doesn't help with.

---

Here's Youtube's keyboard shortcuts (openable via ? (shift-/)) for inspiration of specific key to use

)

[heff]

I'm glad we're thinking about this. The space bar to play at least feels pretty common and expected, and opens the question of how this is achieved in the media chrome world.

After a quick pass on youtube/vimeo/netflix, it does look like there's similar behavior but far from strictly standardized.
Interesting details like on youtube, if you tab to the mute button - space bar triggers mute, but if you click on the mute button - space bar triggers playback. So they're tracking tab vs. click entrance into focus. While with Netflix, spacebar toggles mute in both scenarios. Also whether or not up and down, left and right arrows control sliders, or just one of those pairs.
The differences at least makes me feel like there's some flexibility in choosing the easiest way to implement this, with the first pass at least.

Is there world where controls just stopPropagation on the keyboard events they want to handle and media-controller catches everything else? Do native input elements already do that? I'm not remembering some of the details of keyboard event handling, like do those events fire on the element in focus first or always fire from the window?

how to add keyboard handling to a wrapper element
Is there an immediate need/example for this, or is it something we could wait on and reduce the initial scope of this?

[gkatsev]

The differences at least makes me feel like there's some flexibility in choosing the easiest way to implement this, with the first pass at least.

Yup, plus, we can always start with a few well-known commands and expand it as we go. I wonder if YouTube treats using the mouse as not focusing on the button. I think YouTube has some weird accessibility behavior in some cases.
I would lean towards space keeping existing behavior. So, clicking a button with the mouse focuses it (though, it is not focus visible). Then, you can press space to toggle the keyboard.
For the arrow keys, looks like input ranges by default have both up and right bound to increase the value and left and down bound to decrease the value. But I think the player global initial run with just left/right should be fine, especially since up/down is often bound to volume for the player global shortcuts.

Is there world where controls just stopPropagation on the keyboard events they want to handle and media-controller catches everything else?

I think this is definitely a possibility. My thoughts were to make it easier to implement new components and buttons while minimizing how much existing components need to get updated. By having the component expose the buttons it uses, all the logic involved with knowing how to handle and delegate the keyboard shortcuts is in one place. Then, the components only need to worry about handling the things they care about. For example, right now, media-chrome-button calls handleClick unconditionally on a keyup handler and the media-play-button for example triggers a MEDIA_PLAY_REQUEST or MEDIA_PAUSE_REQUEST without anything else.

Do native input elements already do that? I'm not remembering some of the details of keyboard event handling, like do those events fire on the element in focus first or always fire from the window?

Looks like they do not stop propagation by default. So, the keyup for arrow keys on a range input bubble up.

how to add keyboard handling to a wrapper element

Is there an immediate need/example for this, or is it something we could wait on and reduce the initial scope of this?

We can probably keep it out for an initial PR, but I think we'd want it quickly. Both because this is how Mux Player functions (with the wrapper element that's focusable) but also because we do document how to use media-chrome with a standalone media-controller.

[heff]

My thoughts were to make it easier to implement new components...

Can't we still do that by having the control-specific logic in MediaChromeButton rather than MediaContainer?

Then, the components only need to worry about handling the things they care about.

But if someone is building a custom button, can they change how key controls work just for that button or are we limiting that?

I could be wrong but it feels like we should keep control-specific logic in the controls, if we can. Otherwise we're gonna end up with the MediaContainer blessed way of keyboard controls and then a second way for people do something custom.

[cjpillsbury]

I like the general approach here. I think one other criterion we should explicitly call out is that we don't want to break VoiceOver/other SR keyboard functionality. This is one of the places where some players opt for requiring SR users to rely on their built in keyboard shortcuts, but this creates both discoverability friction and an unnecessary learning burden that isn't being put on sighted folks.

[cjpillsbury]

We may also want to anticipate that, while having english-only screen reader/ARIA verbiage creates some friction for folks outside of the US, keyboard shortcuts are arguably a bigger deal for international adoption in countries that don't have a latin-based written language. At least food for thought.

[gkatsev]

yeah, this is a reason why it would be nice to have viewer-configurable shortcuts in the longer term. I guess media-chrome user configurable shortcuts would be good too (and easier to implement)

[cjpillsbury]

yup as long as we have the right "seams" with the assumption that the keyboard keys may change, that would help with this. We may also want to think of any default as a data object, that way we could eventually key it off of e.g. lang with different default data objects.

[gkatsev]

Also, it seems like special handling for a wrapper element probably isn't necessary considering #263. At least in the case of Mux Player it seems to work as expected. We will need a follow-up to figure out how to handle this for the separate media-controller usecase (one potential solution to that is only allow it via global keyboard shortcuts support).

[gkatsev]

An additional thing we should think about is now that we have keyboard shortcuts for seeking via the left and right arrow keys, we potentially want to make this configurable on the media-controller. However, we also have the seek buttons, which are each configurable separately.
There are several options here, some of which aren't necessarily independant of each other:

• add media-forward-seek-offset and media-backward-seek-offset attributes to media-controller
• a single media-seek-offset attribute
• have media-controller see if there are seek offset buttons configured and get their configured values
• have the seek buttons inherit the attribute value from the media-controller unless they are explicitly given another value