Define a list of metrics that we need to measure

[eugene-manuilov]

We need to conduct a list of metrics that we need to measure in order to understand whether our changes improve the performance or not. This is the first step we need to make before we can start discussing tools that can help us to measure desired metrics and environments where we will run performance tests.

Since this plugin is intended to improve frontend and backend performance, we will need to conduct two lists of metrics. Please, suggest metrics that you think we need to track and provide some details about why we need them.

Frontend metrics:

• TBD

Backend metrics:

• Wall Time of the main function
• CPU Time
• Memory usage

[dainemawer]

I think this is a great starting point and very valid argument @eugene-manuilov !
That being said, I think some metrics are going to blur between backend and frontend.

Let me give you an example. Time to First Byte: measures how long it takes the user’s browser to get (not render) the very first byte of information from the server - this is an incredibly important metric that could be out of the control of WordPress Core itself. It depends on the server infrastructure, hosting provider etc. However, it is incremental for having a healthy FID and thus Speed Index.

As far as front-end goes, I think we should focus on:

• Largest Contentful Paint
• First Contentful Paint
• Total Blocking Time
• Cumulative Layout Shift (though this is becoming less of a priority as the rules have changed)

I mention these metrics specifically, because they are largely comprised of smaller, lesser known browser metrics, but also, they are directly focused around good user experience, i.e getting meaning content to the browser sooner.

The other side to this, is in what context are we measuring? Desktop or Mobile? Mobile devices have significantly less resources (device wise) than Desktop devices, which means that having good mobile scores, by default should dictate good / more performant Desktop scores. It's my assumption then, that we should always measure / improve / iterate on improving metrics for mobile, and ignore Desktop somewhat - taking the mobile first approach.

[aristath]

Some additional:

• Number and size of enqueued styles
• Number and size of enqueued scripts.

[tillkruss]

TTFB

[eugene-manuilov]

Ok, probably, the most important metric from the backend perspective is the Wall Time of the main function. The wall time represents the total time took by a function for execution. For the main function it means the total time that has been spent on processing the current request by PHP. If our improvements prove to reduce the total wall time of the main function, then we can say that we have optimised performance for a certain request.

Two more meaningful metrics for backend performance are CPU time and memory consumption. Making lower any of these metrics doesn't always lead to general improvements of performance, but usually it is good to try to keep these metrics as low as possible.

[lolautruche]

Actually, it may not be a good idea to test against wall time. Time is indeed always a consequence. When using a profile, best is usually to test the number of function calls.
We can still use wall-time to define boundaries though.

[eugene-manuilov]

Actually, it may not be a good idea to test against wall time. Time is indeed always a consequence. When using a profile, best is usually to test the number of function calls. We can still use wall-time to define boundaries though.

Yes, I agree with you, but in our case, this approach is harder to use because we don't have a single function that we can test. We have many different function in the plugin that affect performance in one way or another. That's why I think it would be easier to just rely on the wall time of the main function when we need to gauge performance of a certain page.

[AymenLoukil]

I would suggest these metrics on Mobile/Desktop:

• First Contentful Paint
• Largest Contentful Paint
• Total Blocking Time
• Cumulative Layout Shift
• Time To First Byte
• Total JS size
• Total Img size
• Total CSS size