[Reporting] Add Debug logging for metrics on JS bundle sizes and page weight

[tsullivan]

Problem: The page weight of the Kibana app impacts the performance of Kibana Reporting, but there are no metrics with which to measure. Finding out if performance trends are going up or down across different commits is very difficult. It's also very difficult to give recommendations about how much memory should be available for Reporting, since it depends on which app is used to export, and the amount of data loaded on the page.

@elastic/kibana-app-arch team has done research on measuring the bundle size in the browser, and other page performance metrics. It would be valuable to capture and log these metrics during Reporting screenshot job, and correlate the information to how long a report takes. This information could be used to highlight milestones in Reporting performance.

As a log message, it should be a debug level, and the metric data should be logged as a JSON formatted string. That would allow experiments that involve recording a lot of data points.

cc @streamich @lukeelmers

[lukeelmers]

The two tools I've been relying on for gathering metrics are the CiStatsReporter which generates stats on every PR, and manually running the optimizer with the --profile flag.

But these are both things that are run against the repo itself -- I'm not exactly sure what the best way would be for the reporting service to access this data at runtime.

[joelgriffith]

What about using something like lighthouse to gague holistic performance? Or are we more concerned with just bundle size?

[tsullivan]

@joelgriffith in the description I wrote that we could record the bundle size in the browser, but thinking about it more, we don't need stats that can be automated when testing PRs and release builds. We also don't need to add anything for page load time and number of requests, we can glean that info by looking at the timings between different kinds of debug logs.

That narrows it down just to size of memory being used by the Chromium tab. That number like it would largely relate to bundle size, but also how many requests are running, and how much data the requests are returning.

Other than memory size for the tab I would add CPU utilization, as a percentage numeral, not a profile object, but I'm not sure if that's possible.

This helps us measure how to make things tip over. Also, as we go into streamlining the apps for reporting, this would show the fruit of the efforts.

I think we could close this issue out by simply adding a debug log for the result of window.performance every 1 second. We just need something in the logs to derive info, and that would do it.

[elasticmachine]

Pinging @elastic/kibana-reporting-services (Team:Reporting Services)

[tsullivan]

1. This article: https://media-codings.com/articles/automatically-detect-memory-leaks-with-puppeteer is a good one that explores how an author. It talks about taking heap snapshots using CDP and counting objects on the page to detect memory leaks.

2. @joelgriffith wrote an article that talks about using treekill to clean up helper processes that were spawned by Chromium. We haven't had reports of zombie processes, but I'm unaware if the current Kibana Reporting code does a good job of avoiding zombie processes.

3. Chromium has hundreds of launch args available which need to be reviewed periodically to see if any improve observability of page performance. https://peter.sh/experiments/chromium-command-line-switches/

[tsullivan]

I think we could close this issue out by simply adding a debug log for the result of window.performance every 1 second. We just need something in the logs to derive info, and that would do it.

Besides just logging the data and scraping it from the logs, Reporting also has integration with APM. I think there may be a way to capture the metrics data and index it back into ES through APM metadata.

[tsullivan]

Today I looked more at Chrome DevTools Protocol for ideas on how to track Chromium resource usage from the OS perspective.

The Chrome DevTools Protocol offers methods in the Tracing domain: https://chromedevtools.github.io/devtools-protocol/tot/Tracing

With a "memory-infra" trace, we can get a "Chrome-scale" view of memory allocation: https://chromium.googlesource.com/chromium/src/+/master/docs/memory-infra/README.md

[tsullivan]

The Chrome DevTools Protocol offers methods in the Tracing domain: https://chromedevtools.github.io/devtools-protocol/tot/Tracing

Looks like this idea is the same as what is offered by the page.tracing methods: https://pptr.dev/#?product=Puppeteer&version=v5.4.1&show=api-class-tracing which means we don't have to use the lower-level Chrome Devtools Protocol