Reduce the size of the trace

[patrickhulce]

Tracing causes many sites on WPT to crash Chrome on the phone. Reducing the required number of categories or streaming these few necessary events is the only path forward to making these failure go away.

Lantern relies heavily on quite a few trace events, isolating the key events necessary, and migrating those to Audits.* domain events to eliminate the need for tracing could be helpful since a majority of time spent is in trace parsing

[patrickhulce]

We might want to revisit priority of this since it's responsible for ~10% failure rate on WPT

[tdresser]

This feels like it's moving towards a world where catapult and lighthouse are less similar, which is seems like a step backwards to me.

Can we accomplish the same goal by splitting some trace categories into finer grained sub-categories?

[patrickhulce]

The long term goal was to enable many of Lighthouse insights on all page runs, not just ones where we can capture a trace. In a way, this is much more aligned with native implementation of TTI.

The short-term goal and realistic version of this is exactly what you describe, moving/copying critical trace events to their own smaller category such that we don't need everything under the sun.

[tdresser]

Sound good. For metrics like TTI in particular, long term we hope to just spit out the final values into the trace. Perhaps these final values should be in their own trace category?

[paulirish]

Additionally, tracing causes many sites on WPT to crash Chrome on the phone.

We're also interested in adding another category to the TaskQueueManager::ProcessTaskFromWorkQueue trace event (within Chromium) so that Lighthouse doesn't need to request the toplevel category, which is usually most of the size of the trace:

[tdresser]

What fraction of toplevel is TaskQueueManager::ProcessTaskFromWorkQueue? What else is in toplevel? This might make a bunch of our metrics analysis much easier too.

[tdresser]

Should the title of this issue be something like "Shrink set of required trace events", or similar?

[paulirish]

What fraction of toplevel is TaskQueueManager::ProcessTaskFromWorkQueue?

Here's a 74 mb trace of theverge loading with lighthouse's categories:

In the script, i separated ProcessTaskFromWorkQueue into its own category, so the topmost toplevel doesn't include it.
So to answer the question, that event is 21% of toplevel here. Often I see it being less than 20% of the toplevel bytes.

Shrink set of required trace events

probably. right now its including both 1) moving things from the trace to the protocol and 2) shrinking the trace

---

update feb 2018: CL to tweak trace event category: https://chromium-review.googlesource.com/c/chromium/src/+/791314

[paulirish]

CL updated: https://chromium-review.googlesource.com/c/chromium/src/+/791314

I think we can also get rid of the 'loading' category. probably others.

[paulirish]

As of r588753 (m71), there's a new event for scheduleable tasks. we can now drop toplevel, and add the new category.

"cat":"disabled-by-default-lighthouse","name":"RunTask",

https://bugs.chromium.org/p/chromium/issues/detail?id=874982#c14

[paulirish]

just landed:

• core(driver): request smaller trace in m71+ core(driver): request smaller trace in m71+ #6117
• core(driver): reduce required traceCategories again core(driver): reduce required traceCategories again #6271

The two of these dropped one site's trace from 110MB to 48MB. That definitely is the low hanging fruit.

---

There are probably 2 more obvious improvements we could make.

First, here's what's taking up space (top events, then top categories):

1. screenshots are big. This trace has 398 screenshots. That'll do it. Next is throttling the screenshot capture, i think.
2. the UpdateLayer event from disabled-by-default-devtools.timeline is SUPER chatty. It's worth hunting down why devtools cares about it. perhaps there's a parent task we can use instead? ALL of them are under 0.3ms long, though 15,000 of them do add up to 376ms. But still....
   

That's all that stands out to me, the next biggest things don't seem worth it to deal with.

[paulirish]

We're good for now here. Biggest next item is throttling the screenshots, tracked by #4713