[operations] Track changes to important metrics in CI builds

[spalger]

In 7.7 we had a bit of a fiasco when we realized how large all of the bundles had grown all of the sudden, and we really want to make it easier to spot the changes which lead up to the bundle sizes increasing dramatically on a pr-by-pr basis.

To do this we want to start tracking high-level metrics in an external service like the following:

• size of artifacts loaded on the home page
• number of modules in each KP bundle
• number of files in the distributables
• size of distributables

Additionally, we want to automatically notify PR authors when their PR effects those metrics in a way that the PR author can act on.

At some point we would ideally require sign off when changes to the metrics are large enough, but we will probably need to collect data for a while before we add additional hurdles in front of contributors getting code into the repo.

Current tasks:

• @spalger Collect information about the size of distributables and module counts of specific plugin bundles Collect more build metrics #65408
• @dmlemeshko Collect the size of assets loaded on every page: [ftr] report all file sizes loaded on each page load #65524
• @spalger Collect some test reports that detail what comments on PRs might look like once this is turned on
• Put comments on PRs, probably by pulling them in via the Jenkins pipeline code

---

The initial version of this metrics reporting is already in the repo: create a CiStatsReporter and call reporter.metric() to record a metric for the current CI build. Calls to this method when the CiStatsReporter context is not available in the environment will be a noop.

Here is an example of what we track for each build:

{
  "jenkinsJobName": "elastic+kibana+pipeline-pull-request",
  "jenkinsJobId": "45975",
  "prId": "65177",
  "startedAt": "2020-05-06T14:10:18.660Z",
  "branch": "backport/7.x/pr-63473",
  "commit": "88f12f3645c9db7db51aacacda805fa124486c72",
  "targetBranch": "7.x",
  "mergeBase": "3122a1497579b0e3da9eb2b606babdc7fb529f65",
  "completedAt": "2020-05-06T16:09:13.803Z",
  "result": "SUCCESS"
}

When a metric() call is executed the following is recorded as well:

{
  "name": "@kbn/optimizer bundle module count",
  "subName": "core",
  "value": 279,
  "buildId": "f546075f-1a47-4a31-8e80-28d69e2a8f27"
}

(we're planning to rename "name" to "group", and "subName" to "metricName")

The intention behind this design is to make it easy to produce a table like below that can be logged on a PR:

@kbn/optimizer bundle module count

Name	Value	Change from 3122a14
core	279	+55
data	1258	+0
canvas	549	+0

Additionally, to make this data easier to run ad-hoc visualizations on in the kibana-stats cluster, we plan to create summary documents that combine the data from the build documents and all the metric documents that will be stored in the kibana-stats cluster.

[elasticmachine]

Pinging @elastic/kibana-operations (Team:Operations)

[spalger]

We have solved this with a number of different PRs

• [kbn/optimizer] report sizes of assets produced by optimizer #71319
• Report page load asset size #66224
• Collect more build metrics #65408
• Implement basic CI metric reporting #64263

As well as a new service that tracks the reported metrics: https://github.com/elastic/kibana-ci-stats