Ability to detect whether an element is rendered.

[vmpstr]

This came up several times during discussions:

It would be nice to know if an element is rendered from script. In one context, it may act as a signal whether to do any script side updates to the element. In another similar context, it may act as a signal to stop or start doing resources fetches for a component.

We should add a way to detect from script whether an element is rendered. Some questions to consider:

• Render subtree element is always itself rendered (unless it is under another locked element), so presumably it would return "rendered is true" even if its locked, and one would have to check its children?
  • should this be some sort of an enum "element_and_subtree_rendered, element_is_rendered_subtree_is_not, neither_element_nor_subtree_rendered"?
• How do we handle frames? Should this go up the local frame chain or should this value be relative to the frame's document?

[vmpstr]

/cc @chrishtr

[vmpstr]

/cc @bgirard @fullstackhacker

To answer my own question about frames: I prefer that we limit this to the document level so that the value would not cross frames.

[bgirard]

We had a use case for this that came up while prototyping this. When performing an interaction we'll collect metrics after the final paint. During interactions we will watch for DOM mutations and after the final paint we will walk over the rendered images and make sure they are visible amongst other checks (like making sure it's downloaded). Without knowing if the element is in an activated subtree we query the bounding client react which forces activation of the element. Normally since we just painted the layout information should be current. But this is no longer true if you're within a tree that isn't activated.

If we can query if the element is not activated then we can ignore these subtrees and avoiding activating them.

The other use case that's worth exploring is how frameworks like React could benefit from this. Imagine a frameworks knows that an element is not activated and that it's stale. It can check and defer the rendering of stale elements, from VDOM->DOM) to get the paint out sooner. This use case is more complex so perhaps it warrants a separate deep dive.

[chrishtr]

@bgirard, a question: is it just images in the viewport that you want to check for download+render? If so, why does a regular IntersectionObserver on them not work?

Also, does your current code walk the entire DOM after DOM mutations looking for such images?

@vmpstr and I brainstormed a possible API change to IntersectionObserver where you could observe subtree-visibility skipped state, which would allow your code to sync its observations with what the user agent thinks should be skipped, but we're not sure if that is actually necessary for your use-case, since the user agent will likely have some amount of root margin for skipped.

[bgirard]

The maintainer will see if we can move entirely to IntersectionObserver. Right now it's half IntersectionObserver and half getClientBoundRect as some cases were more convenient done this way.

It might be worthwhile having good WPT coverage around the interaction of this spec and IntersectionObserver.

What are the performance implications of calling querySelecterAll on an ancestor of an invisible subtree for different selector? I filed #144 which may be useful to guide web authors to use this properly correctly.

[chrishtr]

What are the performance implications of calling querySelecterAll on an ancestor of an invisible subtree for different selector? I filed #144 which may be useful to guide web authors to use this properly correctly.

querySelectorAll will force-compute style for any invisible subtrees. The performance of selectors themselves is almost certainly not going to be a big part of the cost, unless there are many matches.

[chrishtr]

I take back my previous comment, it was just incorrect. querySelectorAll does not force-compute styles.

[chrishtr]

Another use case: determining whether a subtree-visibility: hidden element has its subtree skipped. AMP saw this use case when testing with hidden-matchable. They wanted a notification that state changed on the subtree-visibility element. IntersectionObserver didn't work because the root does not change its visibility. (*)

subtree-visibility: auto elements don't have this problem because the only way for the subtree.

In the old API, hidden-matchable elements could change state due to automatic UA activation. The new API doesn't have this, and is instead developer controlled. Since the developer is always aware of what happened to change subtree-visibility, it seems we don't need a new event.

(*) They also observed that this caused a site compatibility bug for them when removing the previous display:none approach, because display:none causes an IntersectionObserver on the element with display changing to get a notification, but subtree-visibility changes do not. They worked around this by force-positioning the subtree-visibility element offscreen via left:9999px when its subtree was hidden-matchable, because in those cases they didn't want to paint the element anyway.

[chrishtr]

Update: the previous use case about detecting subtree skipping turned out to be only for a specific AMP component, and was able to be achieved in a different way.

The use case @bgirard mentioned was able to be solved with IntersectionObserver.

The React use case mentioned (skip Virtual DOM (VDOM) rendering for subtrees that are not on screen) is still quite interesting, because React sites (and those from other frameworks that do client-side hydration) still suffer from the cost of rendering VDOM into DOM, even if that DOM ends up off-screen and has content-visibility: auto. Observing when content-visibility: auto elements start rendering, and conditioning VDOM rendering on that, might not be very hard to implement on top of Suspense/Concurrent-like technologies.

One question that might come up is whether the observer really needs to be in sync with the browser rendering state, or if an IntersectionObserver with generous margins would suffice. It may even be better to use a larger margin for VDOM rendering, because gives more time for the VDOM rendering to occur and therefore reduce checkerboarding / flashing of non-rendered content.

Finally, a special case that is very important is the first render. If a React site is loaded that has all content under content-visibility: auto chunks, or new content is added dynamically to a page with content-visibility: auto, it is critical to render the on-screen content in the first paint and avoid flickering. This was recently fixed in the spec and implementation in Chromium. The internal implementation used a special, internal-to-browser-only IntersectionObserver feature where the first time an observation is made on an element after receiving content-visibility: auto, we re-render (style, layout, etc) if it is visible. Without the corresponding feature for VDOM there would be on-screen flickering.

It seems the solution is to expose a special event to script just for this situation, either via IntersectionObserver or a one-off event, such as firstvisible that would be fired on all content-visibility: auto elements which have not rendered since receiving that CSS property and are about to render, and cause a synchronous re-render afterward if needed. React would use this to synchronously hydrate the corresponding VDOM subtree; which would invalidate that area of DOM and thereby induce the synchronous re-render.

*(Note that this event can be polyfilled via ResizeObserver, so I think would not be introducing a new unachievable footgun.)

[chrishtr]

@vmpstr raised a very good point, which is that, since contain:size depends on whether or not contents are skipped, any attempt to use something other than a direct signal of whether the UA is skipping the content will be a heuristic, and run the risk of the VDOM being out of sync with DOM state for one or more frames.

This seems to be a good enough motivation for exposing an event contentvisibilitychanged that fires when content-visibility: auto element's contents go from skipped to not skipped or vice-versa. The timing of this event should be the same as IntersectionObservation deliveries, and in particular should be sync the first time content-visibility:auto elements become not-skipped (see my previous comment for why).

[developit]

I created a preact-based clone of Una's demo that combines content-visibility:auto with progressive hydration. All hydration of the Virtual DOM for subtrees with content-visibility:auto is deferred (no cost paid up-front) until the subtree becomes visible. It's currently using IntersectionObserver, but could be easily modified to use a contentvisibilitychanged event.

[chrishtr]

New use case: detecting whether an iframe embedded in a cross-origin, possibly not trusted, context is hidden.

The iframe would reasonably want to know whether it is hidden, so that it can throttle work.

[developit]

@chrishtr would the Page Visibility API not suffice there?

[chrishtr]

@developit yes, if we decided that iframes received page visibilty lifecycle events when hidden by content-visibility. Right now they don't...