[Monitoring] Ensure monitoring bulk uploader and stats api are more closely aligned #73325
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The stack monitoring team maintains a set of parity tests between legacy and metricbeat collection, and we recently noticed failures as a result of #70677 in relation to the
kibana_stats.usage.search
data set. The metricbeat-collected document didn't contain thetotal
field whereas the legacy-collected document did. After digging a bit, thetotal
field does not actually come from the newly added collector, but rather it's from an older one: https://github.com/elastic/kibana/blob/master/src/plugins/kibana_usage_collection/server/collectors/kibana/get_saved_object_counts.ts#L73.The parity test failure is a result of handling key merging differently: using defaultsDeep in legacy collection and using object spreading for metricbeat collection.
This PR ensures we use the same merging in both places