[ES|QL] High-level AST mutation API #191812
Assignees
Labels
enhancement
New value added to drive a business result
Feature:ES|QL
ES|QL related features in Kibana
impact:medium
Addressing this issue will have a medium level of impact on the quality/strength of our product.
Team:ESQL
ES|QL related features in Kibana
Comments
vadimkibana
added
Feature:ES|QL
|
Pinging @elastic/kibana-esql (Team:ESQL) |
stratoula
added
enhancement
2 tasks
vadimkibana
added a commit
that referenced
this issue
Sep 26, 2024
## Summary
TL;DR
- Adds ability to parse out comments from source to AST.
- Adds ability for every AST node to have *decoration*—comments,
which can be attached from left, top, and right from the node.
- Implements routine which attached comments to AST nodes.
- In `BasicPrettyPrinter` adds support only for *left* and *right*
comment printing, as the basic printer prints only on one line.
- In `WrappingPrettyPrinter` adds support for all comment printing for
all AST nodes.
- Introduces a `Query` object and `query` AST node, which represent
thole query—the root node, list of commands.
- The ES|QL AST example plugin now displays the pretty-printed text
version.
### Comments
This PR introduced an optional `formatting` field for all AST nodes. In
the `formatting` field one can specify comment decorations from
different sides of a node.
When parsing, once can now specify the `{ withComments: true }` option,
which will collect all comments from the source while parsing using the
`collectDecorations` routine. It will then also call the
`attachDecorations`, which walks the AST and assigns each comment to
some AST node.
Further, traversal and pretty-print API have been updated to work with
comments:
- The `Walker` has been updated to be able to walk all comments from the
AST.
- The `BasicPrettyPrinter` adds support only for *left* and *right*
inline comment printing, as the basic printer prints only on one line.
- The `WrappingPrettyPrinter` adds support for all comment printing for
all AST nodes. It switches to line-break printing mode if it detects
there are comments with line breaks (those could be multi-line comments,
or single line comments—single line comments are always followed
by a line break). It also correctly inserts punctuation, when an AST
node is surrounded by comments.
### Parsing utils
All parsing utils have been moved to the `/parser` sub-folder.
Files in the `/parser` folder have been renamed as per Kibana convention
to reflect what is inside the file. For example, the `EsqlErrorListener`
class is in a file named `esql_error_listener.ts`.
A `Query` class and `ESQLAstQueryExpression` AST nodes have been
introduced. They represent the result of a full query parse. (Before
that, the AST root was just an array of command nodes, now the AST root
is represented by the `ESQLAstQueryExpression` node.)
### Builder
I have started the implementation of the `Builder` static class in the
`/builder` folder. It is simply a collection of stateless AST node
factories—functions which construct AST nodes.
Some of the `Builder` methods are already used by the parser, more will
follow. We will also use the `Builder` in upcoming [*Mutation
API*](#191812).
### ES|QL Example Plugin
This PR sets up Storybook and implements few Storybook stories for the
ES|QL AST example plugin, run it with:
```
yarn storybook esql_ast_inspector
```
This PR updates the *ES|QL AST Explorer* example plugin. Start Kibana
with example plugins enabled:
```
yarn start --run-examples
```
And navigate to
[`/app/esql_ast_inspector`](http://localhost:5601/app/esql_ast_inspector)
to see the new example plugin UI.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### For maintainers
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Sep 26, 2024
## Summary
TL;DR
- Adds ability to parse out comments from source to AST.
- Adds ability for every AST node to have *decoration*—comments,
which can be attached from left, top, and right from the node.
- Implements routine which attached comments to AST nodes.
- In `BasicPrettyPrinter` adds support only for *left* and *right*
comment printing, as the basic printer prints only on one line.
- In `WrappingPrettyPrinter` adds support for all comment printing for
all AST nodes.
- Introduces a `Query` object and `query` AST node, which represent
thole query—the root node, list of commands.
- The ES|QL AST example plugin now displays the pretty-printed text
version.
### Comments
This PR introduced an optional `formatting` field for all AST nodes. In
the `formatting` field one can specify comment decorations from
different sides of a node.
When parsing, once can now specify the `{ withComments: true }` option,
which will collect all comments from the source while parsing using the
`collectDecorations` routine. It will then also call the
`attachDecorations`, which walks the AST and assigns each comment to
some AST node.
Further, traversal and pretty-print API have been updated to work with
comments:
- The `Walker` has been updated to be able to walk all comments from the
AST.
- The `BasicPrettyPrinter` adds support only for *left* and *right*
inline comment printing, as the basic printer prints only on one line.
- The `WrappingPrettyPrinter` adds support for all comment printing for
all AST nodes. It switches to line-break printing mode if it detects
there are comments with line breaks (those could be multi-line comments,
or single line comments—single line comments are always followed
by a line break). It also correctly inserts punctuation, when an AST
node is surrounded by comments.
### Parsing utils
All parsing utils have been moved to the `/parser` sub-folder.
Files in the `/parser` folder have been renamed as per Kibana convention
to reflect what is inside the file. For example, the `EsqlErrorListener`
class is in a file named `esql_error_listener.ts`.
A `Query` class and `ESQLAstQueryExpression` AST nodes have been
introduced. They represent the result of a full query parse. (Before
that, the AST root was just an array of command nodes, now the AST root
is represented by the `ESQLAstQueryExpression` node.)
### Builder
I have started the implementation of the `Builder` static class in the
`/builder` folder. It is simply a collection of stateless AST node
factories—functions which construct AST nodes.
Some of the `Builder` methods are already used by the parser, more will
follow. We will also use the `Builder` in upcoming [*Mutation
API*](elastic#191812).
### ES|QL Example Plugin
This PR sets up Storybook and implements few Storybook stories for the
ES|QL AST example plugin, run it with:
```
yarn storybook esql_ast_inspector
```
This PR updates the *ES|QL AST Explorer* example plugin. Start Kibana
with example plugins enabled:
```
yarn start --run-examples
```
And navigate to
[`/app/esql_ast_inspector`](http://localhost:5601/app/esql_ast_inspector)
to see the new example plugin UI.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
### For maintainers
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
(cherry picked from commit 2217337)
kibanamachine
referenced
this issue
Sep 26, 2024
# Backport This will backport the following commits from `main` to `8.x`: - [[ES|QL] Comment parsing and pretty-printing (#192173)](#192173) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Vadim Kibana","email":"82822460+vadimkibana@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-09-26T10:34:38Z","message":"[ES|QL] Comment parsing and pretty-printing (#192173)\n\n## Summary\r\n\r\nTL;DR\r\n\r\n- Adds ability to parse out comments from source to AST.\r\n- Adds ability for every AST node to have *decoration*—comments,\r\nwhich can be attached from left, top, and right from the node.\r\n- Implements routine which attached comments to AST nodes.\r\n- In `BasicPrettyPrinter` adds support only for *left* and *right*\r\ncomment printing, as the basic printer prints only on one line.\r\n- In `WrappingPrettyPrinter` adds support for all comment printing for\r\nall AST nodes.\r\n- Introduces a `Query` object and `query` AST node, which represent\r\nthole query—the root node, list of commands.\r\n- The ES|QL AST example plugin now displays the pretty-printed text\r\nversion.\r\n\r\n\r\n### Comments\r\n\r\nThis PR introduced an optional `formatting` field for all AST nodes. In\r\nthe `formatting` field one can specify comment decorations from\r\ndifferent sides of a node.\r\n\r\nWhen parsing, once can now specify the `{ withComments: true }` option,\r\nwhich will collect all comments from the source while parsing using the\r\n`collectDecorations` routine. It will then also call the\r\n`attachDecorations`, which walks the AST and assigns each comment to\r\nsome AST node.\r\n\r\nFurther, traversal and pretty-print API have been updated to work with\r\ncomments:\r\n\r\n- The `Walker` has been updated to be able to walk all comments from the\r\nAST.\r\n- The `BasicPrettyPrinter` adds support only for *left* and *right*\r\ninline comment printing, as the basic printer prints only on one line.\r\n- The `WrappingPrettyPrinter` adds support for all comment printing for\r\nall AST nodes. It switches to line-break printing mode if it detects\r\nthere are comments with line breaks (those could be multi-line comments,\r\nor single line comments—single line comments are always followed\r\nby a line break). It also correctly inserts punctuation, when an AST\r\nnode is surrounded by comments.\r\n\r\n\r\n### Parsing utils\r\n\r\nAll parsing utils have been moved to the `/parser` sub-folder.\r\n\r\nFiles in the `/parser` folder have been renamed as per Kibana convention\r\nto reflect what is inside the file. For example, the `EsqlErrorListener`\r\nclass is in a file named `esql_error_listener.ts`.\r\n\r\nA `Query` class and `ESQLAstQueryExpression` AST nodes have been\r\nintroduced. They represent the result of a full query parse. (Before\r\nthat, the AST root was just an array of command nodes, now the AST root\r\nis represented by the `ESQLAstQueryExpression` node.)\r\n\r\n\r\n### Builder\r\n\r\nI have started the implementation of the `Builder` static class in the\r\n`/builder` folder. It is simply a collection of stateless AST node\r\nfactories—functions which construct AST nodes.\r\n\r\nSome of the `Builder` methods are already used by the parser, more will\r\nfollow. We will also use the `Builder` in upcoming [*Mutation\r\nAPI*](https://github.com/elastic/kibana/issues/191812).\r\n\r\n\r\n### ES|QL Example Plugin\r\n\r\nThis PR sets up Storybook and implements few Storybook stories for the\r\nES|QL AST example plugin, run it with:\r\n\r\n```\r\nyarn storybook esql_ast_inspector\r\n```\r\n\r\nThis PR updates the *ES|QL AST Explorer* example plugin. Start Kibana\r\nwith example plugins enabled:\r\n\r\n```\r\nyarn start --run-examples\r\n```\r\n\r\nAnd navigate to\r\n[`/app/esql_ast_inspector`](http://localhost:5601/app/esql_ast_inspector)\r\nto see the new example plugin UI.\r\n\r\n\r\n\r\n\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n\r\n### For maintainers\r\n\r\n- [x] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>\r\nCo-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>","sha":"2217337c5d91340ba67e0bedaab0762502518993","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["review","release_note:skip","v9.0.0","backport:prev-minor","Feature:ES|QL","Team:ESQL","v8.16.0"],"title":"[ES|QL] Comment parsing and pretty-printing","number":192173,"url":"https://github.com/elastic/kibana/pull/192173","mergeCommit":{"message":"[ES|QL] Comment parsing and pretty-printing (#192173)\n\n## Summary\r\n\r\nTL;DR\r\n\r\n- Adds ability to parse out comments from source to AST.\r\n- Adds ability for every AST node to have *decoration*—comments,\r\nwhich can be attached from left, top, and right from the node.\r\n- Implements routine which attached comments to AST nodes.\r\n- In `BasicPrettyPrinter` adds support only for *left* and *right*\r\ncomment printing, as the basic printer prints only on one line.\r\n- In `WrappingPrettyPrinter` adds support for all comment printing for\r\nall AST nodes.\r\n- Introduces a `Query` object and `query` AST node, which represent\r\nthole query—the root node, list of commands.\r\n- The ES|QL AST example plugin now displays the pretty-printed text\r\nversion.\r\n\r\n\r\n### Comments\r\n\r\nThis PR introduced an optional `formatting` field for all AST nodes. In\r\nthe `formatting` field one can specify comment decorations from\r\ndifferent sides of a node.\r\n\r\nWhen parsing, once can now specify the `{ withComments: true }` option,\r\nwhich will collect all comments from the source while parsing using the\r\n`collectDecorations` routine. It will then also call the\r\n`attachDecorations`, which walks the AST and assigns each comment to\r\nsome AST node.\r\n\r\nFurther, traversal and pretty-print API have been updated to work with\r\ncomments:\r\n\r\n- The `Walker` has been updated to be able to walk all comments from the\r\nAST.\r\n- The `BasicPrettyPrinter` adds support only for *left* and *right*\r\ninline comment printing, as the basic printer prints only on one line.\r\n- The `WrappingPrettyPrinter` adds support for all comment printing for\r\nall AST nodes. It switches to line-break printing mode if it detects\r\nthere are comments with line breaks (those could be multi-line comments,\r\nor single line comments—single line comments are always followed\r\nby a line break). It also correctly inserts punctuation, when an AST\r\nnode is surrounded by comments.\r\n\r\n\r\n### Parsing utils\r\n\r\nAll parsing utils have been moved to the `/parser` sub-folder.\r\n\r\nFiles in the `/parser` folder have been renamed as per Kibana convention\r\nto reflect what is inside the file. For example, the `EsqlErrorListener`\r\nclass is in a file named `esql_error_listener.ts`.\r\n\r\nA `Query` class and `ESQLAstQueryExpression` AST nodes have been\r\nintroduced. They represent the result of a full query parse. (Before\r\nthat, the AST root was just an array of command nodes, now the AST root\r\nis represented by the `ESQLAstQueryExpression` node.)\r\n\r\n\r\n### Builder\r\n\r\nI have started the implementation of the `Builder` static class in the\r\n`/builder` folder. It is simply a collection of stateless AST node\r\nfactories—functions which construct AST nodes.\r\n\r\nSome of the `Builder` methods are already used by the parser, more will\r\nfollow. We will also use the `Builder` in upcoming [*Mutation\r\nAPI*](https://github.com/elastic/kibana/issues/191812).\r\n\r\n\r\n### ES|QL Example Plugin\r\n\r\nThis PR sets up Storybook and implements few Storybook stories for the\r\nES|QL AST example plugin, run it with:\r\n\r\n```\r\nyarn storybook esql_ast_inspector\r\n```\r\n\r\nThis PR updates the *ES|QL AST Explorer* example plugin. Start Kibana\r\nwith example plugins enabled:\r\n\r\n```\r\nyarn start --run-examples\r\n```\r\n\r\nAnd navigate to\r\n[`/app/esql_ast_inspector`](http://localhost:5601/app/esql_ast_inspector)\r\nto see the new example plugin UI.\r\n\r\n\r\n\r\n\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n\r\n### For maintainers\r\n\r\n- [x] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>\r\nCo-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>","sha":"2217337c5d91340ba67e0bedaab0762502518993"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/192173","number":192173,"mergeCommit":{"message":"[ES|QL] Comment parsing and pretty-printing (#192173)\n\n## Summary\r\n\r\nTL;DR\r\n\r\n- Adds ability to parse out comments from source to AST.\r\n- Adds ability for every AST node to have *decoration*—comments,\r\nwhich can be attached from left, top, and right from the node.\r\n- Implements routine which attached comments to AST nodes.\r\n- In `BasicPrettyPrinter` adds support only for *left* and *right*\r\ncomment printing, as the basic printer prints only on one line.\r\n- In `WrappingPrettyPrinter` adds support for all comment printing for\r\nall AST nodes.\r\n- Introduces a `Query` object and `query` AST node, which represent\r\nthole query—the root node, list of commands.\r\n- The ES|QL AST example plugin now displays the pretty-printed text\r\nversion.\r\n\r\n\r\n### Comments\r\n\r\nThis PR introduced an optional `formatting` field for all AST nodes. In\r\nthe `formatting` field one can specify comment decorations from\r\ndifferent sides of a node.\r\n\r\nWhen parsing, once can now specify the `{ withComments: true }` option,\r\nwhich will collect all comments from the source while parsing using the\r\n`collectDecorations` routine. It will then also call the\r\n`attachDecorations`, which walks the AST and assigns each comment to\r\nsome AST node.\r\n\r\nFurther, traversal and pretty-print API have been updated to work with\r\ncomments:\r\n\r\n- The `Walker` has been updated to be able to walk all comments from the\r\nAST.\r\n- The `BasicPrettyPrinter` adds support only for *left* and *right*\r\ninline comment printing, as the basic printer prints only on one line.\r\n- The `WrappingPrettyPrinter` adds support for all comment printing for\r\nall AST nodes. It switches to line-break printing mode if it detects\r\nthere are comments with line breaks (those could be multi-line comments,\r\nor single line comments—single line comments are always followed\r\nby a line break). It also correctly inserts punctuation, when an AST\r\nnode is surrounded by comments.\r\n\r\n\r\n### Parsing utils\r\n\r\nAll parsing utils have been moved to the `/parser` sub-folder.\r\n\r\nFiles in the `/parser` folder have been renamed as per Kibana convention\r\nto reflect what is inside the file. For example, the `EsqlErrorListener`\r\nclass is in a file named `esql_error_listener.ts`.\r\n\r\nA `Query` class and `ESQLAstQueryExpression` AST nodes have been\r\nintroduced. They represent the result of a full query parse. (Before\r\nthat, the AST root was just an array of command nodes, now the AST root\r\nis represented by the `ESQLAstQueryExpression` node.)\r\n\r\n\r\n### Builder\r\n\r\nI have started the implementation of the `Builder` static class in the\r\n`/builder` folder. It is simply a collection of stateless AST node\r\nfactories—functions which construct AST nodes.\r\n\r\nSome of the `Builder` methods are already used by the parser, more will\r\nfollow. We will also use the `Builder` in upcoming [*Mutation\r\nAPI*](https://github.com/elastic/kibana/issues/191812).\r\n\r\n\r\n### ES|QL Example Plugin\r\n\r\nThis PR sets up Storybook and implements few Storybook stories for the\r\nES|QL AST example plugin, run it with:\r\n\r\n```\r\nyarn storybook esql_ast_inspector\r\n```\r\n\r\nThis PR updates the *ES|QL AST Explorer* example plugin. Start Kibana\r\nwith example plugins enabled:\r\n\r\n```\r\nyarn start --run-examples\r\n```\r\n\r\nAnd navigate to\r\n[`/app/esql_ast_inspector`](http://localhost:5601/app/esql_ast_inspector)\r\nto see the new example plugin UI.\r\n\r\n\r\n\r\n\r\n\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n\r\n### For maintainers\r\n\r\n- [x] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>\r\nCo-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>","sha":"2217337c5d91340ba67e0bedaab0762502518993"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Vadim Kibana <82822460+vadimkibana@users.noreply.github.com>
2 tasks
vadimkibana
added a commit
that referenced
this issue
Oct 8, 2024
## Summary Partially addresses #191812 This PR implements some generic ES|QL AST mutation APIs and specifically APIs for working with `FROM` command `METADATA` fields: - `from.metadata.list()` — List all `METADATA` fields. - `from.metadata.find()` — Find a `METADATA` field by name. - `from.metadata.removeByPredicate()` — Remove a `METADATA` field by matching a predicate. - `from.metadata.remove()` — Remove a `METADATA` field by name. - `from.metadata.insert()` — Insert a `METADATA` field. - `from.metadata.upsert()` — Insert `METADATA` field, if it does not exist. ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Oct 8, 2024
## Summary Partially addresses elastic#191812 This PR implements some generic ES|QL AST mutation APIs and specifically APIs for working with `FROM` command `METADATA` fields: - `from.metadata.list()` — List all `METADATA` fields. - `from.metadata.find()` — Find a `METADATA` field by name. - `from.metadata.removeByPredicate()` — Remove a `METADATA` field by matching a predicate. - `from.metadata.remove()` — Remove a `METADATA` field by name. - `from.metadata.insert()` — Insert a `METADATA` field. - `from.metadata.upsert()` — Insert `METADATA` field, if it does not exist. ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) (cherry picked from commit a819d65)
vadimkibana
added a commit
that referenced
this issue
Oct 15, 2024
## Summary Partially addresses #191812 Implements the following high-level ES|QL AST manipulation methods: - `.generic` - `.appendCommandArgument()` — Add a new main command argument to a command. - `.removeCommandArgument()` — Remove a command argument from the AST. - `.commands` - `.from` - `.sources` - `.list()` — List all `FROM` sources. - `.find()` — Find a source by name. - `.remove()` — Remove a source by name. - `.insert()` — Insert a source. - `.upsert()` — Insert a source, if it does not exist. - `.limit` - `.list()` — List all `LIMIT` commands. - `.byIndex()` — Find a `LIMIT` command by index. - `.find()` — Find a `LIMIT` command by a predicate function. - `.remove()` — Remove a `LIMIT` command by index. - `.set()` — Set the limit value of a specific `LIMIT` command. - `.upsert()` — Insert a `LIMIT` command, or update the limit value if it already exists. ### Checklist Delete any items that are not applicable to this PR. - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Oct 15, 2024
## Summary Partially addresses elastic#191812 Implements the following high-level ES|QL AST manipulation methods: - `.generic` - `.appendCommandArgument()` — Add a new main command argument to a command. - `.removeCommandArgument()` — Remove a command argument from the AST. - `.commands` - `.from` - `.sources` - `.list()` — List all `FROM` sources. - `.find()` — Find a source by name. - `.remove()` — Remove a source by name. - `.insert()` — Insert a source. - `.upsert()` — Insert a source, if it does not exist. - `.limit` - `.list()` — List all `LIMIT` commands. - `.byIndex()` — Find a `LIMIT` command by index. - `.find()` — Find a `LIMIT` command by a predicate function. - `.remove()` — Remove a `LIMIT` command by index. - `.set()` — Set the limit value of a specific `LIMIT` command. - `.upsert()` — Insert a `LIMIT` command, or update the limit value if it already exists. ### Checklist Delete any items that are not applicable to this PR. - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels) (cherry picked from commit 10364fb)
2 tasks
vadimkibana
added a commit
that referenced
this issue
Oct 30, 2024
## Summary Partially addresses #191812 - Adds traversal and manipulation APIs for `SORT` command. - `commands.sort.listCommands()` - `commands.sort.getCommand()` - `commands.sort.list()` - `commands.sort.findByPredicate()` - `commands.sort.find()` - `commands.sort.remove()` - `commands.sort.insertIntoCommand()` - `commands.sort.insertExpression()` - `commands.sort.insertCommand()` - Refactors "generic" AST manipulation routines into (1) `commands`, (2) `commands.args`, (3) `commands.options`. - `generic.commands.*` - `generic.commands.args.*` - `generic.commands.options.*` ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels) --------- Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Oct 30, 2024
## Summary Partially addresses elastic#191812 - Adds traversal and manipulation APIs for `SORT` command. - `commands.sort.listCommands()` - `commands.sort.getCommand()` - `commands.sort.list()` - `commands.sort.findByPredicate()` - `commands.sort.find()` - `commands.sort.remove()` - `commands.sort.insertIntoCommand()` - `commands.sort.insertExpression()` - `commands.sort.insertCommand()` - Refactors "generic" AST manipulation routines into (1) `commands`, (2) `commands.args`, (3) `commands.options`. - `generic.commands.*` - `generic.commands.args.*` - `generic.commands.options.*` ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels) --------- Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co> (cherry picked from commit 0cc8945)
vadimkibana
added a commit
that referenced
this issue
Nov 12, 2024
## Summary Partially addresses #191812 This PR implement higher-level convenience methods for working with `STATS` commands: - `commands.stats.list()` - iterates over all `STATS` commands. - `commands.stats.byIndex()` - retrieves the Nth `STATS` command. - `commands.stats.summarize()` - returns summary about fields used in aggregates and grouping for all `STATS` commands in the query. - `commands.stats.summarizeCommand()` - same as `.summarize()`, but returns a summary only about the requested command. Usage: ```ts const query = EsqlQuery.fromSrc('FROM index | STATS a = max(b)'); const summary = commands.stats.summarize(query); // [ { aggregates: { a: { fields: ['b'] }} ] ``` ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Nov 12, 2024
## Summary Partially addresses elastic#191812 This PR implement higher-level convenience methods for working with `STATS` commands: - `commands.stats.list()` - iterates over all `STATS` commands. - `commands.stats.byIndex()` - retrieves the Nth `STATS` command. - `commands.stats.summarize()` - returns summary about fields used in aggregates and grouping for all `STATS` commands in the query. - `commands.stats.summarizeCommand()` - same as `.summarize()`, but returns a summary only about the requested command. Usage: ```ts const query = EsqlQuery.fromSrc('FROM index | STATS a = max(b)'); const summary = commands.stats.summarize(query); // [ { aggregates: { a: { fields: ['b'] }} ] ``` ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels) (cherry picked from commit be1a4bb)
3 tasks
tkajtoch
pushed a commit
to tkajtoch/kibana
that referenced
this issue
Nov 12, 2024
## Summary Partially addresses elastic#191812 This PR implement higher-level convenience methods for working with `STATS` commands: - `commands.stats.list()` - iterates over all `STATS` commands. - `commands.stats.byIndex()` - retrieves the Nth `STATS` command. - `commands.stats.summarize()` - returns summary about fields used in aggregates and grouping for all `STATS` commands in the query. - `commands.stats.summarizeCommand()` - same as `.summarize()`, but returns a summary only about the requested command. Usage: ```ts const query = EsqlQuery.fromSrc('FROM index | STATS a = max(b)'); const summary = commands.stats.summarize(query); // [ { aggregates: { a: { fields: ['b'] }} ] ``` ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels)
vadimkibana
added a commit
that referenced
this issue
Nov 13, 2024
## Summary Partially addresses #191812 - Correctly extracts summary from of fields from the `BY` clause of `STATS` command. - The `.summarize()` command now returns `newFields` and `usedFields` properties. The `newFields` is a list of newly created fields by the `STATS` command. The `usedFields` is a list of all fields which were used by the `STATS` command. - Improves parameter node handling. ### Example Extract all "new" and "used" fields from all `STATS` commands: ```ts const query = EsqlQuery.fromSrc('FROM index | STATS a = max(b), agg(c) BY d'); const summary = mutate.commands.stats.summarize(query); console.log(summary.newFields); // [ 'a', '`agg(c)`' ] console.log(summary.usedFields); // [ 'b', 'c', 'd' ] ``` ### Checklist Delete any items that are not applicable to this PR. - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels)
vadimkibana
added a commit
to vadimkibana/kibana
that referenced
this issue
Nov 13, 2024
## Summary Partially addresses elastic#191812 - Correctly extracts summary from of fields from the `BY` clause of `STATS` command. - The `.summarize()` command now returns `newFields` and `usedFields` properties. The `newFields` is a list of newly created fields by the `STATS` command. The `usedFields` is a list of all fields which were used by the `STATS` command. - Improves parameter node handling. ### Example Extract all "new" and "used" fields from all `STATS` commands: ```ts const query = EsqlQuery.fromSrc('FROM index | STATS a = max(b), agg(c) BY d'); const summary = mutate.commands.stats.summarize(query); console.log(summary.newFields); // [ 'a', '`agg(c)`' ] console.log(summary.usedFields); // [ 'b', 'c', 'd' ] ``` ### Checklist Delete any items that are not applicable to this PR. - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels) (cherry picked from commit d276b48) # Conflicts: # packages/kbn-esql-ast/src/mutate/commands/stats/index.test.ts # packages/kbn-esql-ast/src/mutate/commands/stats/index.ts
CAWilson94
pushed a commit
to CAWilson94/kibana
that referenced
this issue
Nov 18, 2024
## Summary Partially addresses elastic#191812 - Correctly extracts summary from of fields from the `BY` clause of `STATS` command. - The `.summarize()` command now returns `newFields` and `usedFields` properties. The `newFields` is a list of newly created fields by the `STATS` command. The `usedFields` is a list of all fields which were used by the `STATS` command. - Improves parameter node handling. ### Example Extract all "new" and "used" fields from all `STATS` commands: ```ts const query = EsqlQuery.fromSrc('FROM index | STATS a = max(b), agg(c) BY d'); const summary = mutate.commands.stats.summarize(query); console.log(summary.newFields); // [ 'a', '`agg(c)`' ] console.log(summary.usedFields); // [ 'b', 'c', 'd' ] ``` ### Checklist Delete any items that are not applicable to this PR. - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels)
CAWilson94
pushed a commit
to CAWilson94/kibana
that referenced
this issue
Nov 18, 2024
## Summary Partially addresses elastic#191812 This PR implement higher-level convenience methods for working with `STATS` commands: - `commands.stats.list()` - iterates over all `STATS` commands. - `commands.stats.byIndex()` - retrieves the Nth `STATS` command. - `commands.stats.summarize()` - returns summary about fields used in aggregates and grouping for all `STATS` commands in the query. - `commands.stats.summarizeCommand()` - same as `.summarize()`, but returns a summary only about the requested command. Usage: ```ts const query = EsqlQuery.fromSrc('FROM index | STATS a = max(b)'); const summary = commands.stats.summarize(query); // [ { aggregates: { a: { fields: ['b'] }} ] ``` ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels)
CAWilson94
pushed a commit
to CAWilson94/kibana
that referenced
this issue
Nov 18, 2024
## Summary Partially addresses elastic#191812 - Correctly extracts summary from of fields from the `BY` clause of `STATS` command. - The `.summarize()` command now returns `newFields` and `usedFields` properties. The `newFields` is a list of newly created fields by the `STATS` command. The `usedFields` is a list of all fields which were used by the `STATS` command. - Improves parameter node handling. ### Example Extract all "new" and "used" fields from all `STATS` commands: ```ts const query = EsqlQuery.fromSrc('FROM index | STATS a = max(b), agg(c) BY d'); const summary = mutate.commands.stats.summarize(query); console.log(summary.newFields); // [ 'a', '`agg(c)`' ] console.log(summary.usedFields); // [ 'b', 'c', 'd' ] ``` ### Checklist Delete any items that are not applicable to this PR. - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels)
2 tasks
vadimkibana
added a commit
that referenced
this issue
Nov 28, 2024
## Summary Partially addresses #191812 Implements high-level APIs for working with `WHERE` command. - `commands.where.list()` — lists all `WHERE` commands. - `commands.where.byIndex()` — finds the Nth `WHERE` command in the query. - `commands.where.byField()` — finds the first `WHERE` command which uses a specified field or a param. ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels)
2 tasks
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Nov 28, 2024
## Summary Partially addresses elastic#191812 Implements high-level APIs for working with `WHERE` command. - `commands.where.list()` — lists all `WHERE` commands. - `commands.where.byIndex()` — finds the Nth `WHERE` command in the query. - `commands.where.byField()` — finds the first `WHERE` command which uses a specified field or a param. ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [x] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels) (cherry picked from commit 3e899e7)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Implement a high-level API for ES|QL AST mutations.
METADATAfieldsWHEREcommandsLIMITSORTcommand expressionsFROMsourcesSTATSnew field columns usedThe API
The API could look something like below.
Manipulate
METADATAfields:Remove commands:
Change
LIMITof the query:...
The text was updated successfully, but these errors were encountered: