[Telemetry] The Enterprise Search solution usage measure *search* should exclude internal indices

[aznick]

The Kibana telemetry collector has an Enterprise Search index pattern definition *search* that is too broad. For example, it inadvertently includes deployments using async search (which can create an .async_search index).

As a suggestion, could we please create an exclusion rule so the only matches for the search index name are user-generated indices?

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[afharo]

This is interesting! I thought we were excluding dot-indices unless explicitly declared. I'll create a test to double confirm this behaviour! Thanks for bringing it up @aznick!

[afharo]

As a suggestion, could we please create an exclusion rule so the only matches for the search index name are user-generated indices?

@aznick by "user-generated indices" do you mean "no dot-indices"? Or any other check?

The logic already has in place a way to discard dot-indices (unless they are Data Streams .ds-*, or they are explicitly added to the index pattern definition).

We even have a specific piece of logic to ensure dot-indices don't match more general rules. For example:

Index name	Matched patterns	Comments
wordpress-search / wordpressearch / a-wordpress-index-for-search	*wordpress* and *search*	By design, we report all the patterns that are matched
.ent-search-test	.ent-search-*	dot-indices are only compared to dot-patterns
.async-search or .async_search	none	I've created #96693 to add an explicit test to confirm that's the case

I hope it makes sense.

cc-ing @alexfrancoeur @thesmallestduck @kobelb as they helped on the initial design of this feature :)

[elasticmachine]

Pinging @elastic/kibana-telemetry (Team:KibanaTelemetry)

[aznick]

by "user-generated indices" do you mean "no dot-indices"? Or any other check?

I was thinking no dot-indices, but I'm certainly open to other suggestions. Just to make sure I understand, @afharo – are .async-search or .async_search already not counted as an Enterprise Search index pattern?

[aznick]

An update to the above, I just tested again and it looks like the current implementation does not include .async_search in that telemetry count.

@afharo thank you for your help digging into it and the additional test! Closing this issue; please feel free to reopen if necessary.