Media Controller Rearchitecture

[cjpillsbury]

Overview

As part of our Media Chrome v1.0 efforts, we are planning on reworking and decoupling some of the core architecture owned by and instantiated with Media Controller. We're doing this now to avoid a potential major refactor later down the road to unlock and/or simplify potential "blue sky"/future possible uses of Media Chrome. That said, there are still a lot of TBDs when it comes to where we want these pieces to settle before an official media-chrome@1.0 release.

What I'm hoping for with this discussion/outline is to get folks thinking about what we or users of Media Chrome might want now or in the future. The point isn't to necessarily implement all of these things, but rather to:

1. Make sure we avoid "coding ourselves into a corner" if possible, based on some attention to unnecessary assumptions and some known possible future goals.
2. Avoid excessive "bikeshedding" or debate as a pre-condition for even starting implementation. If we can acknowledge that different "blue sky" goals are not completely unreasonable and we can use (1) to make sure we aren't inherently coding ourselves away from those goals, I think we'll be in a much better place to have a maintainable, versatile, and evolvable architecture without having to "burn it all down".
3. Make sure we don't start with the overly-opinionated ways of solving (or solving for) a particular goal/problem. As many of us already know, it's much easier to add new features/functionalities/conveniences than it is to remove or completely refactor them.

The initial steps for this effort have already begun in earnest in this PR

Media Controller, a story in three parts

You can think of our current core architecture for Media Chrome as being owned by a set of three inter-related and currently "coupled" parts of the codebase that can be used via an instance of the <media-controller> element. These are:

1. <media-container>

The <media-container> element is primarily responsible for a convenient reasonable layout of a "Player UI". It includes layering, well-defined slots, automatic showing/hiding for things like:

• poster image elements
• gesture/keypress detection
• control/display elements
• the media element itself

In addition, it is, at least currently, responsible for some state management, particularly state that is more explicitly about the (non-media) UI, such state related to showing/hiding aspects of the UI based on user interactions.

2. Media Controller

The Media Controller itself can be thought of as two inter-related but distinct things, and the code is somewhat (though not entirely) written to reflect this:

2.a. State Management (JS)

While the bulk of this code still "lives in" the media-controller module, it's mostly broken up into into its own parts of the codebase. This is the part of the code that's responsible for, in particular:

• Monitoring for potential state changes on the media element and related (via e.g. events)
• Getting the current (modeled) state from the media element and related to propagate
• Attempting to update the state of the media element and related based on state change requests.

These are all JavaScript only. Currently, there are some assumptions on how this state is propagated and "wired up" that tie it to use with HTMLElements (including but not limited to Web Components), but these mostly occur "around the edges".

2.b. <media-controller> (HTML/Declarative)

Even though this element currently is a subclass of <media-container>, it can be thought of and is sometimes used as as having its own responsibility, namely: It is a simple, declarative way via HTML of having access to the state management defined in 2.a., above. In other words, one can get all of the state management without having to write a lick of JS simply by adding <media-controller> to an HTML page. Even if most of the state management logic moves to a standalone JavaScript entity, this feels like an independently valuable feature worth mentioning, particularly for DevEx/"ergonomics"/ease of use.

What does it mean to decouple these three parts?

Minimally, if we want to be able to treat these three as distinct, we'll need to:

• Move 2.a. (State Management) into its own JavaScript Module
  • Remove assumptions that are closely coupled to HTMLElements for data flow, such as: serializing state to strings (on the assumption that they'll be used to set HTMLElement attributes), getting/setting attributes on "Media State Receiver" HTMLElements, and listening to DOM events for media state change requests.
• Update <media-controller> to not be a subclass of <media-container>
  • Make <media-controller> a non-visual, non-interactive element
  • NOTE: This does not necessarily mean that <media-container> cannot have state management "built in" by default, only that it would have to handled through "composition" instead of via subclassing. In other words, this is a separate decision. (more on this, below)
• Update <media-container> to look more like just another "Media State Receivers"
  • This will likely need to involve some way for the slotted media element in <media-container> is "shared" with the relevant 2.a. (State Management) instance (potentially but not necessarily simply using our "media state change request" infrastructure, where the value would be the media element itself).
  • Make sure any state that should be part of 2.a. (State Management) is migrated. (more on this, below)
  • Make sure any code assumed for <media-container> UI is made available somehow (either provided or "baked in to its class def"). (more on this, below)

Why decouple these three parts?

The short answer here (at least in my opinion) is that we can make our code less "opinionated" or "presumptuous" on how it has to be used, and we can do so without too much work or making the "under the hood" code especially more complex. By removing these assumptions and deep coupling, we can reduce the amount of friction for at least some categories of use cases and feature additions, even ones that we might not currently be envisionioning. Below are just a few "blue skies" examples. More "blue sky" ideas from other contributors are encouraged so that we can have as many of those as possible available when moving forward with architecture decisions and implementation.

As a developer, I would like to use Media Chrome UI elements but with my own controller logic

Example 1 - Extending the Media Controller to add logic for your use case:

<!-- New architecture would make it easier to extend just the state mgmt -->
<my-extended-media-controller id="custom-controller"></my-extended-media-controller>
<media-container controller="custom-controller">
  <video slot="media">
  </video>
  <media-control-bar>
    <media-play-button></media-play-button>
    <media-mute-button></media-mute-button>
    <custom-state-ui></custom-state-ui>
  </media-control-bar>
</media-container>

Example 2 - Replacing the media controller with a different media domain such as realtime:

<!-- New architecture would make it easier to create a different domain-specific controller/mapping -->
<!-- NOTE That this may be centralized elsewhere on the page and may "outlive" e.g. different re-renderings of (the UI parts of) the DOM -->
<realtime-media-controller id="realtime-controller"></realtime-media-controller>
<media-container controller="realtime-controller">
  <realtime-video local slot="media">
  </realtime-video>
  <media-control-bar>
    <media-play-button></media-play-button>
    <media-mute-button></media-mute-button>
  </media-control-bar>
</media-container>
<chat-component controller="realtime-controller"></chat-component>

As a React developer, I would like to be able to use Media Chrome but as idiomatic React

// index.jsx
import React from 'react';
import ReactDOM from 'react-dom';
// Writing an "idiomatically React" way of managing and using state becomes easier
// with separate JS-only state mgmt. In this fake example, it could resemble Redux,
// though it doesn't need to.
import { Provider, createMediaStore } from 'react-media-chrome';
import AppWithPlayerUI from './AppWithPlayerUI';

const mediaStore = createMediaStore();

// As of React 18
const root = ReactDOM.createRoot(document.getElementById('root'));
root.render(
  <Provider mediaStore={mediaStore}>
    <AppWithPlayerUI />
  </Provider>
);

// PlayButton.jsx
import React from 'react';
// We're not building these now, but we could (or someone else could)
import { useSelector, useDispatch } from 'react-media-chrome';
import PauseSvg from './pause-svg';
import PlaySvg from './play-svg';

// This also has the advantage of easily allowing folks to write custom UI components
// in React in a way that would be both simple and familiar to them, so both
// the state mgmt APIs and the component development would have very little learning
// curve.
export const PlayButton = (/*props*/) => {
  const mediaPaused = useSelector((state) => state.mediaPaused);
  const dispatch = useDispatch();
  return (
    <button
      onClick={() =>
        dispatch({
          type: mediaPaused ? 'mediaplayrequest' : 'mediapauserequest',
        })
      }
    >
      {mediaPaused ? <PlaySvg /> : <PauseSvg />}
    </button>
  );
};

Here are a few (non-exhaustive) very broad categories of what can be done or done more easily by separating out the three pieces.

• Use Media Chrome's state management but use their own components, potentially written in the framework of their choice
• Use Media Chrome's UI components, including <media-container>, but with their own State Management infrastructure.
• More easily persist state management across page/state updates

What still needs to be figured out?

Even if we move forward with splitting these three pieces into their own entities, there are still open questions in the details. At least some of these can also be approached with a "blue skies" approach. Here are a few:

1. Should folks still be able to get the "status quo" where they can drop a single element on the page and get all 3 parts. E.g. should <media-container> have state mgmt by default or with minimal additional code? Or should we assume the state management controller and container UI will be separate elements, at least for the time being? This discussion was begun with a slightly different framing here.

2. What state actually belongs to the controller? Some state is very obviously media state (e.g. muted, paused). Others are slightly less obvious (e.g. fullscreen), more descriptive of the runtime environment (e.g. airplay availability), or exclusively about UI (e.g. user inactivity). Here's a rough Venn Diagram of state:

3. <media-container>, as a UI element, will still need a slot for a media element for layout purposes. How should we model the relationships between the state management controller and the slotted media element in <media-container>? What if someone doesn't want to use <media-container> for UI layout?

4. How/where should we handle/define state propagation for our official UI components? If we don't want to deeply assume "Media State Receivers" will be web components, we still need something to be responsible as the "glue"/translation layer for this aspect of Media Chrome state management.

5. Similar to the previous one, how/where should we handle/define media state requests? If we don't want to deeply assume DOM events, we still need something to be responsible as the "glue"/translation layer for this aspect of Media Chrome state management.

It's worth noting that some of these decisions (or others that are unlisted) don't need to be fully fleshed out before we do an official v1.0 Media Chrome release. That said, as mentioned at the top, we should try to avoid "coding ourselves into a corner" that would make some of these decisions "implicitly decided" prematurely.

What's next?

Ideally, folks will start coming up with other "blue sky" goals/possibilities for what the new rearchitecture could unlock. Similarly, if there are other open questions that should be on our radar sooner rather than later, we should try to identify those now.

Prior Thoughts/Prior Art

Over a year ago, I drafted this potential architecture diagram as a longer term goal. While we might not want to do everything here, it may be helpful as a way to get bearings on some of the moving pieces. If this is not helpful, feel free to ignore!