Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhanced ES|QL Query History incl. starring a query #194165

Closed
ninoslavmiskovic opened this issue Sep 26, 2024 · 5 comments · Fixed by #198362
Closed

Enhanced ES|QL Query History incl. starring a query #194165

ninoslavmiskovic opened this issue Sep 26, 2024 · 5 comments · Fixed by #198362
Assignees
Labels
Feature:ES|QL ES|QL related features in Kibana impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. Team:ESQL ES|QL related features in Kibana

Comments

@ninoslavmiskovic
Copy link
Contributor

ninoslavmiskovic commented Sep 26, 2024

Description

We’ve introduced a feature in the ES|QL editor, where we also investigated and discussed future phases of the query history. While the current implementation allows users to track their queries, it lacks advanced features like starring and quickly recalling specific queries.

Image

User story Acceptance criteria
Starring a query in Query History - As a user, I want to star/favorite specific ES|QL queries from my query history for quick recall later. Users can mark queries as “starred”, which will be easily accessible from within the query history window as a separate tab.  Starred queries should remain available even after the main query history is cleared or exceeds its limit.
500 starred queries as max - As a user, I want to view max 100 starred queries in my “Starred” section. The “Starred” part of the query history should be expanded beyond 20 queries to max 100. Perhaps introduce pagination or lazy loading to handle large history efficiently. Users should be able to scroll through older queries or load more on demand.
Warning when I reach my limit of 100 starred queries - As a user, I want to get a warning when I reach my limit of 100 starred queries, so I can manage my queries proactively. Users are notified when their starred query section reaches the maximum limit (e.g. 100 queries)
Sort my “Starred” queries - As a user, I want to be able to sort my “Starred” queries history based on timestamp like on query history items. The starred queries tab should be sortable like the query history tab on timestamp.

Product Requirement Document

Wireframes

@ninoslavmiskovic ninoslavmiskovic self-assigned this Sep 26, 2024
@botelastic botelastic bot added the needs-team Issues missing a team label label Sep 26, 2024
@ninoslavmiskovic ninoslavmiskovic added Feature:ES|QL ES|QL related features in Kibana Team:ESQL ES|QL related features in Kibana and removed needs-team Issues missing a team label labels Sep 27, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-esql (Team:ESQL)

@ninoslavmiskovic ninoslavmiskovic added the impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. label Sep 27, 2024
@stratoula
Copy link
Contributor

stratoula commented Oct 7, 2024

We had a sync withj @rudolf and @Dosant asynchronously. We agreed that:

  • we are going to extend the favorites service to store more information than just ids. Specifically we would like to store something like that per user id
"esql_queries": [
        {
          "queryString": "FROM logs",
          "id": "<a random id>",
          "timeRan": "<timestamp>"
        }
        {
          "queryString": "FROM logs | STATS count = COUNT(*)",
          "id": "<a random id>",
          "timeRan": "<timestamp>"
		},
      ]
  • The service is going to limit the user to max 100 starred queries. We want to be sure that we are not going to cause any performance / storage problem. If there are complains from the users we can rediscuss.
  • The users will be able to star / unstar queries

@azasypkin
Copy link
Member

@stratoula, do we envision any sensitive or PII data being included in the queryString? I assume the answer is "yes, probably", but I'm not sure.

/cc @elastic/kibana-security

@stratoula
Copy link
Contributor

stratoula commented Oct 7, 2024

@azasypkin these are queries that the users are authoring so I am assuming that yes. But which is the difference on what we do already? I mean now the users can write an ES|QL query and save it to a dashboard or Discover saved search. (also as these are SOs they can export and import them to other instances).

@azasypkin
Copy link
Member

But which is the difference on what we do already?

Sure, there doesn't seem to be any difference from what we already do. I don't think there's anything to worry about here. I'm mostly asking to get a fuller picture of what user/user-generated data we store and where, especially with regard to new functionality.

@stratoula stratoula self-assigned this Oct 8, 2024
stratoula pushed a commit to stratoula/kibana that referenced this issue Nov 19, 2024
## Summary

close elastic#194165
close elastic/kibana-team#1245

### User-facing

<img width="1680" alt="image"
src="https://github.com/user-attachments/assets/6df4ee9f-1b4d-404c-a764-592998a1d430">

This PRs adds a new tab in the editor history component. You can star
your query from the history and then you will see it in the Starred
list. The started queries are scoped to a user and a space.

### Server

To allow starring ESQL query, this PR extends [favorites
service](elastic#189285) with ability to
store metadata in addition to an id. To make metadata strict and in
future to support proper metadata migrations if needed, metadata needs
to be defined as schema:

```
plugins.contentManagement.favorites.registerFavoriteType('esql_query', {
       typeMetadataSchema: schema.object({ query: schema.string(), timeRange:...., etc... }),
})
```

Notable changes:

- Add support for registering a favorite type and a schema for favorite
type metadata. Previosly the `dashboard` type was the only supported
type and was hardcoded
- Add `favoriteMetadata` property to a saved object mapping and make it
`enabled:false` we don't want to index it, but just want to store
metadata in addition to an id.
[code](https://github.com/elastic/kibana/pull/198362/files#diff-d1a39e36f1de11a1110520d7607e6aee7d506c76626993842cb58db012b760a2R74-R87)
- Add a 100 favorite items limit (per type per space per user). Just do
it for sanity to prevent too large objects due to metadata stored in
addtion to ids.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
Co-authored-by: Stratoula Kalafateli <stratoula1@gmail.com>
(cherry picked from commit 4597237)

# Conflicts:
#	.github/CODEOWNERS
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this issue Dec 12, 2024
## Summary

close elastic#194165
close elastic/kibana-team#1245

### User-facing

<img width="1680" alt="image"
src="https://github.com/user-attachments/assets/6df4ee9f-1b4d-404c-a764-592998a1d430">

This PRs adds a new tab in the editor history component. You can star
your query from the history and then you will see it in the Starred
list. The started queries are scoped to a user and a space.


### Server

To allow starring ESQL query, this PR extends [favorites
service](elastic#189285) with ability to
store metadata in addition to an id. To make metadata strict and in
future to support proper metadata migrations if needed, metadata needs
to be defined as schema:

```
plugins.contentManagement.favorites.registerFavoriteType('esql_query', {
       typeMetadataSchema: schema.object({ query: schema.string(), timeRange:...., etc... }),
})
```

Notable changes: 

- Add support for registering a favorite type and a schema for favorite
type metadata. Previosly the `dashboard` type was the only supported
type and was hardcoded
- Add `favoriteMetadata` property to a saved object mapping and make it
`enabled:false` we don't want to index it, but just want to store
metadata in addition to an id.
[code](https://github.com/elastic/kibana/pull/198362/files#diff-d1a39e36f1de11a1110520d7607e6aee7d506c76626993842cb58db012b760a2R74-R87)
- Add a 100 favorite items limit (per type per space per user). Just do
it for sanity to prevent too large objects due to metadata stored in
addtion to ids.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
Co-authored-by: Stratoula Kalafateli <stratoula1@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:ES|QL ES|QL related features in Kibana impact:high Addressing this issue will have a high level of impact on the quality/strength of our product. Team:ESQL ES|QL related features in Kibana
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants