Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support index pattern selector syntax in SQL #120845

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

jbaiera
Copy link
Member

@jbaiera jbaiera commented Jan 24, 2025

Selector syntax (#118614) was introduced as part of the work to support implementing failure stores in data streams. This is a new feature of index patterns which allows a user to specify which indices inside of a data stream should be used in an action. Selectors are denoted by using the :: separator between a data stream name and the component of the data stream the user wants to target for an operation.

To search a data stream's backing indices only, the ::data selector is used:

SELECT * FROM "my-data-stream::data"

To search a data stream's failure indices only, the ::failures selector is used:

SELECT * FROM "my-data-stream::failures

To search both a data stream's backing and failure indices, the ::* selector is used:

SELECT * FROM "my-data-stream::*

By default, when a data stream has no selector specified, the ::data selector is implied to maintain backwards compatibility with current search functionality. The ::data selector primarily exists as a way to explicitly select the backing indices, but is not required for normal usage.

This PR updates the SQL grammar to include the selector portion of an index pattern. The qualifiedIndex() has been updated to include selectors in the resulting expression. Underlying search operations should already support this functionality, so this is primarily wiring it up where needed.

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@elasticsearchmachine elasticsearchmachine added Team:Data Management Meta label for data/management team Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) labels Jan 24, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

*/
@Override
public void visitErrorNode(ErrorNode node) {}
/**
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This entire file was regenerated with inconsistent formatting. Is this something that should be checked in as part of this PR? Does this get regenerated as needed during the build?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/SQL SQL querying :Data Management/Data streams Data streams and their lifecycles >non-issue Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) Team:Data Management Meta label for data/management team v8.19.0 v9.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants