Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support KQL in TSVB Filter aggregation #35437

Closed
TinaHeiligers opened this issue Apr 22, 2019 · 4 comments
Closed

Support KQL in TSVB Filter aggregation #35437

TinaHeiligers opened this issue Apr 22, 2019 · 4 comments
Assignees
Labels
Feature:KQL KQL Team:Visualizations Visualization editors, elastic-charts and infrastructure

Comments

@TinaHeiligers
Copy link
Contributor

TinaHeiligers commented Apr 22, 2019

In the Visual Builder, the time series visualization only accepts Lucene syntax in the

  1. Data -> Options -> Filter,
  2. Data -> Metrics -> Group by -> Query string (when grouping by a single filter or many filters),
  3. Panel Options -> Data Panel -> Filter
  4. Annotations -> Data Source -> Query string.

TimeSeries_DataTab_OptionsTab

TimeSeries_DataTab_MetricsTab_GroupBy_QueryString

TimeSeries_DataTab_MetricsTab_GroupBy_Filters_QueryString

TimeSeries_PanelOptionsTab_PanelFilter

TimeSeries_AnnotationsTab_QueryString

Using KQL with the autocomplete it offers will make it easier to create a query without the need to remember the exact field name and it assists with operators and values.

In each of the TSVB situations given above, we need a query bar without a submit button. Ideally, it should be a QueryBarInput component that is just the text input, the query language switcher and autocomplete. One is being created for KQL in the filters aggregation (#29790)

References #29790, #30177

@TinaHeiligers TinaHeiligers added the Team:Visualizations Visualization editors, elastic-charts and infrastructure label Apr 22, 2019
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-app

@TinaHeiligers TinaHeiligers self-assigned this Apr 22, 2019
@TinaHeiligers
Copy link
Contributor Author

The following points were raised after implementation discussions:

  1. The modified version of the QueryBar will need to accept lucene syntax.
  2. The implementation has to deal with the fact that the saved visualization queries are in lucene, and have the ability to detect and differenciate between lucene and KQL syntax in saved visualizations. A saved object migration is necessary for that.
  3. The query string is currently passed as is to the server. The server then parses it into an ES query for the request.
    Everything is more or less in the file src/legacy/core_plugins/metrics/server/lib/vis_data/request_processors/series/query.js
    where we have the main filters, one on each series and one on the panel and they are applied to the ES query as:
if (panel.filter) {
      doc.query.bool.must.push({
        query_string: {
          query: panel.filter,
          analyze_wildcard: true,
        },
      });
    }

    if (series.filter) {
      doc.query.bool.must.push({
        query_string: {
          query: series.filter,
          analyze_wildcard: true,
        },
      });
    }
  1. The filter strings will need to be parsed according to the syntax they are composed in, checked for errors by the QueryBar and then sent as plain query strings to the server.
  2. On the server side, the string needs to be detected and included as part of the other filters used to build the main query in:
doc.query = buildEsQuery(indexPattern, queries, filters, esQueryConfig);

@TinaHeiligers
Copy link
Contributor Author

TinaHeiligers commented May 3, 2019

Steady progress is being made with KQL supported in TSVB using the QueryBar component.
KQL_in_TSVB_progress
Remaining items:

@TinaHeiligers
Copy link
Contributor Author

Closed with merged PR #36784

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:KQL KQL Team:Visualizations Visualization editors, elastic-charts and infrastructure
Projects
None yet
Development

No branches or pull requests

2 participants