Add selector syntax to index expressions

[jbaiera]

In order to make adoption of failure stores simpler for all users, we are introducing a new syntactical feature to index expression resolution: The selector.

Selectors, denoted with a :: followed by a recognized suffix will allow users to specify which component of an index abstraction they would like to operate on within an API call. In this case, an index abstraction is a concrete index, data stream, or alias; Any abstraction that can be resolved to a set of indices/shards. We define a component of an index abstraction to be some searchable unit of the index abstraction.

To start, we will support two components: data and failures. Concrete indices are their own data components, while the data component for index aliases are all of the indices contained therein. For data streams, the data component corresponds to their backing indices. Data stream aliases mirror this, treating all backing indices of the data streams they correspond to as their data component.

The failure component is only supported by data streams and data stream aliases. The failure component of these abstractions refer to the data streams' failure stores. Indices and index aliases do not have a failure component.

In order to select which components a user wants to operate on within an API call, the components are selected using the new selector syntax in an index expression:

GET my-data-stream::failures/_search

On supporting APIs, selector syntax is evaluated first in every index expression. The syntax only accepts a predefined list of selector names or the match all * wildcard. It does not support any other selector names and it does not support simple match syntax. Any unsupported usage of the selector syntax will raise an exception.

Examples:

// Accepted
my-data-stream::data // Resolves to the data stream's backing indices only
my-data-stream::failures // Resolves to the data stream's failure indices only
my-data-stream::* // Resolves to both the data stream's backing and failure indices

// Rejected
my-data-stream::d*ta // Simple match syntax is not supported
my-data-stream::custom // Syntax only accepts predefined selector names (currently `*`, `data`, and `failures`)

Selector syntax is compatible with other expression syntax like date math and wildcard matching of index names:

GET <logs-daily-{now/d}>::failures/_settings
GET logs-*::failures/_mappings
GET *::*,-logs-*::data/_search

Selector syntax will not be supported on every API as it does not always make sense. For example, the create and delete index APIs operate on indices only and will not support selectors. In these cases, any selector usage will raise an exception:

PUT logs-my-index::data // Raises exception because selector was used on unsupported API

API's that support selectors will default to using the ::data selector when selectors are not provided on an index expression. When evaluating expressions that have an explicit ::data selectors on them, the resolution code will translate the expressions back into names without selectors for consistency's sake.

This PR was reconstructed from #113144 which it supercedes.

[jbaiera]

@elasticmachine run buildkite/docs-build-pr

[jbaiera]

@elasticmachine run elasticsearch-ci/part-1

[elasticsearchmachine]

💔 Backport failed

Status	Branch	Result
❌	8.x	Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 118614

[jbaiera]

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Questions ?

Please refer to the Backport tool documentation