[APM] Make sure telemetry queries are scoped and have a timeout

[dgieselaar]

Once every twelve hours, we sequentially run a set of queries server-side (with the internal user) to collect telemetry about APM data. We should make sure that these queries only consider relevant data for performance and privacy reasons, and that they're guarded with a timeout. Right now this is not always the case. E.g., the cardinality task queries all indices, making it a relatively (and unnecessarily) expensive request. We should make both index and timeout required in the client type that is passed down to the telemetry tasks, to prevent this situation from occurring.

kibana/x-pack/plugins/apm/server/lib/apm_telemetry/collect_data_telemetry/tasks.ts

Lines 1034 to 1060 in d260172

	name: 'cardinality',
	executor: async ({ search }) => {
	const allAgentsCardinalityResponse = await search({
	body: {
	size: 0,
	timeout,
	query: {
	bool: {
	filter: [range1d],
	},
	},
	aggs: {
	[TRANSACTION_NAME]: {
	cardinality: {
	field: TRANSACTION_NAME,
	},
	},
	[USER_AGENT_ORIGINAL]: {
	cardinality: {
	field: USER_AGENT_ORIGINAL,
	},
	},
	},
	},
	});
	const rumAgentCardinalityResponse = await search({

[elasticmachine]

Pinging @elastic/apm-ui (Team:APM)