Implement FastMode MVP

[patrickhulce]

The future is comprised of many things, one of which is FastMode. This issue is a placeholder and gathering point for all things FastMode. Excerpts from the yet-to-be-finalized doc are below.

Goals

• Deliver Lighthouse's functional audits and mobile performance insights as fast as the PageSpeed Insights web frontend.
• Provide a coarse mobile performance score using a trace and network log that were taken from a page load that was not throttled.
• Identify and fix any missing and necessary critical path information in the tracing/network stack in Chrome.
• (Stretch) Estimate the First Meaningful Paint and Time to Consistently Interactive with high enough precision to disable throttling by default in the DevTools audits panel.

Background

Lighthouse's performance evaluations today are heavily based on metrics that analyze a specific load of the page (first meaningful paint, time to consistently interactive, perceptual speed index, etc) under the assumption that the load being analyzed is representative of the load a user would experience. To receive an accurate picture of mobile performance, Lighthouse loads the page under mobile emulation with heavy network and CPU throttling. The metrics on the throttled run then provide a somewhat realistic view of end-user-perceived load performance. However, this throttled load of the page comes at a cost. Getting results is significantly slower, the size of trace files that need to be parsed is much larger, and the resources of the host machine are not fully utilized during the run. Lighthouse FastMode would need to be able to evaluate expected performance on mobile without relying on network and CPU throttling.

Lighthouse today also performs several checks that require significant evaluation time in the browser context such as the aXe accessibility library, re-compression of images, and walking the entire DOM tree for problematic event listeners. Lighthouse FastMode might need to move some of this logic into Chrome to see the speed improvements necessary for delivering results in a PageSpeed Insights timeframe.

Overview

The performance of a webpage is determined by the execution path of its resources and tasks. If a system could perfectly capture the dependency tree and execution plan of every resource and task, estimating performance under different conditions would be as simple as manipulating the duration of each node in this execution plan DAG and selecting the completion time of the maximal node. Unfortunately, there are many challenges with capturing the dependency tree that complicate this method that Lighthouse FastMode will need to overcome.

To predict how a webpage will perform in a constrained mobile environment, FastMode will analyze the resources loaded by the page and generate multiple potential execution paths with the given network and CPU constraints. These potential execution paths represent the many different possible subgraphs of the dependency trees at the moments of key metrics such as first meaningful paint or time to interactive. The duration of these different execution paths will be estimated and combined to determine a final performance score.

internal doc at go/lighthouse-fast-mode

[mixed]

Hi. @patrickhulce.
WOW. It is awesome project. I'll be looking forward to it.
I have some questions. :) I think that The project is focused on fast mode, but the above aticle is similar to what I'm currently working on.

I am working on a small project inspired by #1451. The project display metrics on a Network Waterfall and mark improvement point in resources.

I finished some work. I am going to make the relationship between the main thread, the network, metrics after I make a graph of the main thread. This part looks somewhat similar to the above mentioned task. Would you let me know more detail what you are making? I added the prototype. I think I am going to wait for your work if you do something similar. :)

Thanks.

[patrickhulce]

thanks @mixed! The bulk of this has already been completed in the PRs if you want to checkout those branches and play around with the graph you get out of it. For constructing a waterfall visualization you'll probably be most interested in the estimateWithDetails output that provides node start and end times for the given network conditions.

[mixed]

Thank you @patrickhulce . :)
Do you mean that estimateWithDetails is graphs?

[patrickhulce]

Those graphs will give you subsets of the page load as it happened. I was referring to
https://github.com/GoogleChrome/lighthouse/pull/3162/files#diff-4f68e0c9c7e4e44501bbda145b66f9b7R326 which lets you simulate the graph under different network/CPU conditions that gives you start and end times for the new version. Your visualization would be super helpful for debugging changes to the estimation logic too!

[paulirish]

@patrickhulce remind me what's remaining here before we close this ticket?

[patrickhulce]

there's a few items that impact accuracy i'm evaluating now, the impls were trivial just need validation it moves things in the right direction

1. add dependency on URLs in stack trace for all trace events
2. bring over redirect initiator logic from the python chrome repo
3. remove all the cycle checks in favor of a single isValidGraph method

[vinamratasingal-zz]

@patrickhulce want to move this to Sprint Dos then?

[patrickhulce]

I believe Paul just did, and I approve :)

[patrickhulce]

additional stack trace URLs had no noticeable impact, so I'm inclined to drop that to keep trace event dependencies clear

adding redirect initiator logic improved FMP accuracy 1% but regressed TTCI accuracy 1%, so seems like additional research is needed in the future to look into that

cycle checks seem to be a-ok on the test data set so I'll put that up into a PR 👍