Log if a page was ever visible

[n8schloss]

A common issue that we're seeing at Facebook is that a good number of pages are never visible while they are loading. Due to background tab throttling these loads are often slower, but that's fine because no one is looking at them. To ensure that we're correctly monitoring this, we'd love to track if pages are ever visible. Unfortunately there's a brief period of time before our JavaScript runs on the page and during this time we have no clue as to the page's visibility state.

To solve this, we'd love if currently hidden pages had a way to tell us if they were ever visible and if yes how long was it visible for. Thoughts?

[igrigorik]

Trying to wrap my head around the use case..

1. User clicks on a link -> stares at spinner -> switches to another tab -> ... -> comes back to page
2. User cmd-clicks link -> page loads in background -> switches to page

You want to distinguish between 1 and 2? Practically speaking, if you execute visibility test as first thing on the page (i.e. in the head) with existing API.. Doesn't it give you all the information you need -- the page is hidden hence load is/was throttled. Why does it matter if the user was looking at the page earlier or not?

[n8schloss]

Yeah, that's exactly what we're trying to distinguish between. In case 1 we could have shown something quicker and in case 2 if the user isn't actively looking at the page we care about it less. We can totally put something in the head, but if the user is blocked on the network and stared at a spinner we'd still like to know that.

[igrigorik]

In case 1 we could have shown something quicker and in case 2 if the user isn't actively looking at the page we care about it less.

How so? Practically speaking you're stuck on the response bytes from the server.. I understand that it may be nice to know if the user was "waiting" for those bytes, but practically speaking there is nothing you can do to improve the speed here. If your first bytes query visibility you know your answer, right?

[n8schloss]

but practically speaking there is nothing you can do to improve the speed here

That's not true, it'd be a good indication that there are network issues and that we need to make improvements to CDN, network architecture, etc.

[igrigorik]

@n8schloss this came up in a different discussion today. Question for you...

One solution could be to resurrect ~prerenderSwitch concept in Nav Timing. Something like:

The visibilitySwitch attribute must return a DOMHighResTimeStamp with a time value equal to the first time when document visibility changes from "hidden".

Would that address your use case?

[n8schloss]

yep, that would totally address our use case :)

[spanicker]

[copy-pasting my response from internal thread]

Re: what can be made available on Navigation Timing: we could expose a field indicating whether the page was initially hidden or visible (background nav vs foreground).
Beyond that not sure it makes sense to wait and see when the visibility changed:

• how long would we wait - some finite length of time right? then it's not exactly "when" visibility changed but rather "whether visibility changed in the first M seconds of load" or something.
• queueing NT entry should not be blocked on the above.

Given all this I'm not sure on the value of making this a timestamp, maybe a boolean on whether initial visibility was hidden or visible (background or foreground load)?

[igrigorik]

Is initial state sufficient to cover the original use case?

It would solve the question for "was the page in background", but it wouldn't capture the case where user clicks on a link.. waits a while.. switches to a different page while navigation is loading, right? In this case the initial state is "visible" but then switches to "hidden", and if the page or network (or both) are slow enough, the user may even do this a few times -- I've certainly done it myself, where I toggle to check if the page has loaded a few times before the answer is yes.

If we wanted to solve the full case, it seems that we'd need to log all the visibilitychange events and make them available? E.g. one plausible solution could be: prior to end onload, emit a new ~VisibilityPerformanceEntry logging the state change with timestamp. The page can harvest these later to figure out if the user was toggling visibility and how long the page was in background? </brainstorm>

[n8schloss]

It would solve the question for "was the page in background", but it wouldn't capture the case where user clicks on a link.. waits a while.. switches to a different page while navigation is loading, right?

@igrigorik, that's correct. I think we'd prefer to be able to answer the question about when the page was visible or not instead of just knowing about it's initial state. Given the current page visibility API though our main gap is around the initial state before our JS loads. If we're open to a bigger change, I think VisibilityPerformanceEntry we definitely be the ideal solution here. I could see it growing large though, so perhaps there should be some size limit with a default of like 100 entries or something.

[igrigorik]

@igrigorik, that's correct. I think we'd prefer to be able to answer the question about when the page was visible or not instead of just knowing about it's initial state.

We've received feedback before that asking "inline this in the head" is a tough sell for all but a few hand-tuned sites. For example, if you have existing RUM analytics.. it's usually loaded async (as it should) and "in the head" requirement is not compatible with current deployments. Hence my thought around buffering events to allow them to harvest such data.

[spanicker]

Buffering up to max(N entries, time limit) seems reasonable given the inlining in head constraint.
This requires a new Entry type though.

[igrigorik]

Right. Following the pattern we converged at F2F — see w3c/performance-timeline#81 (comment) — I think we'd be looking at following changes in PV:

1. Define new ~VisibilityPerformanceEntry interface
2. Define an associated buffer + buffer size
3. Update processing section to queue new perf entries defined in (1)

Conceptually, nothing hard.. but it does add some new non-trivial surface to the PV spec.

@toddreifsteck would love to hear your thoughts on this one.

[toddreifsteck]

We have seen the need for this use-case in discussions with developers. I agree that we should tackle this use-case, but believe it should be in L3.

[spanicker]

Todd's proposal from public-web-perf meeting: for near term expose "wasHidden" and "wasVisible" to indicate whether the Document was "ever hidden" or "ever visible".
Nate, that would address this usecase?

[n8schloss]

@spanicker, yeah for the short term that would be sufficient. Tackling the full visibility history in L3 sounds good to me.

[igrigorik]

Hmm, so I'm less keen on 'hack a solution for L2, solve it in L3' approach.. If we think that exposing the booleans is not sufficient to address all the underlying use cases, then we should work on the full solution and then figure out if it belongs in L2 or L3.

[npm1]

If someone was following this issue, I should note that there is some related discussion in w3c/performance-timeline#105