[telemetry] _xpack/usage endpoint missing dataframe

[Bamieh]

Hello 👋

We have a test in kibana that checks that we are sending all the telemetry fields. Currently it is failing on master because xpack.data_frame is missing from the payload.

All the stack_stats.xpack.* telemetry data is passed directly from elasticsearch _xpack/usage endpoint.

Ideally it would be great to have the same tests we have on kibana checking for those fields in elasticsearch as well, this way we'll have increased confidence that those fields will return what we expect when swapping between snapshots.

We do check for the following fields returned from elasticsearch:

expect(stats.stack_stats.xpack.graph).to.be.an('object');
expect(stats.stack_stats.xpack.data_frame).to.be.an('object');
expect(stats.stack_stats.xpack.data_frame.available).to.be.an('boolean');
expect(stats.stack_stats.xpack.data_frame.enabled).to.be.an('boolean');
expect(stats.stack_stats.xpack.ilm).to.be.an('object');
expect(stats.stack_stats..xpack.logstash).to.be.an('object');
expect(stats.stack_stats.xpack.ml).to.be.an('object');
expect(stats.stack_stats.xpack.monitoring).to.be.an('object');
expect(stats.stack_stats.xpack.rollup).to.be.an('object');

currently we get the following keys in the payload (data_frame is missing)

curl -XGET localhost:9200/_xpack/usage -u elastic:changeme -k | jq 'keys'                                                 
[
  "analytics",
  "ccr",
  "flattened",
  "frozen_indices",
  "graph",
  "ilm",
  "logstash",
  "ml",
  "monitoring",
  "rollup",
  "security",
  "spatial",
  "sql",
  "transform",
  "vectors",
  "voting_only",
  "watcher"
]

Related: elastic/kibana#45983
Related: elastic/kibana#45884

[elasticmachine]

Pinging @elastic/es-core-infra

[elasticmachine]

Pinging @elastic/ml-core

[alpar-t]

👍 on adding similar tests to ES

[hendrikmuhs]

This is by design as the data frame plugin gets renamed to transform for 7.5. The kibana side needs updates.

Moving to kibana: elastic/kibana#46325

+1 on adding similar tests to ES

There are tests which tests the usage endpoint, which have been updated as part of the PR. I missed to update kibana tests as I wasn't aware of it.