KQL in TSVB #36784

TinaHeiligers · 2019-05-21T15:15:17Z

Summary

Using KQL with the autocomplete it offers makes it easier to create a query without the need to remember the exact field name and it assists with operators and values.

This PR replaces the text input fields for filters in the Visual Builder with QueryBarInput components that update as a query is entered with the option of using Kuery syntax with autocomplete or Lucene syntax.

A migration was needed to transform the filter strings in visualizations to filter query objects, containing the original string and the query language. We assume that if the query language is not present, it is Lucene syntax.
References #29790, #30177

PR in favor of #36175

References Support KQL in TSVB Filter aggregations
Adds KQL to Visual Builder (TSVB)

Checklist

Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR.

This was checked for cross-browser compatibility (Chrome Version 74.0.3729.169 (Official Build) (64-bit) & Safari Version 12.1.1 (14607.2.6.1.1)), ~~[including a check against IE11]~~(https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)
~~- [ ] Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support~~
~~- [ ] Documentation was added for features that require explanation or tutorials~~
Unit or functional tests were updated or added to match the most common scenarios
This was checked for keyboard-only ~~and screenreader accessibility~~

For maintainers

This was checked for breaking API changes and was labeled appropriately - no breaking changes are introduced.
~~- [ ] This includes a feature addition or change that requires a release note and was labeled appropriately~~

elasticmachine · 2019-05-21T16:34:35Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

TinaHeiligers · 2019-05-21T16:56:33Z

jenkins, test this

elasticmachine · 2019-05-21T18:16:59Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

TinaHeiligers · 2019-05-22T17:09:52Z

jenkins, test this

elasticmachine · 2019-05-22T20:12:36Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

elasticmachine · 2019-05-22T22:45:04Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

elasticmachine · 2019-05-23T01:43:44Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

elasticmachine · 2019-05-23T16:53:41Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2019-05-23T23:48:24Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2019-05-24T01:54:29Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

elasticmachine · 2019-05-24T18:25:38Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2019-05-24T19:39:56Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2019-05-24T22:23:33Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

elasticmachine · 2019-05-24T22:52:27Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2019-05-24T23:57:47Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

elasticmachine · 2019-05-28T16:00:34Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

elasticmachine · 2019-05-28T17:38:44Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2019-05-28T18:41:53Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

Bargs

A couple other things in addition to the inline comments:

I think we should update the saved objects in the sample data sets and change the queries to use KQL where possible
The current use of screenTitle on the QueryBarInput creates non-sensical aria-labels like “You are on search box of TimeseriesPanelConfigQuery page. Start typing to search and filter the VisEditor”. My filters agg PR makes this prop optional. It also updates the query_bar_input to pass down the ID EUI creates to the underlying input element so that screenreaders will automatically read the correct label for the input (although passing the id explicitly won't be necessary if you end up passing all unknown props to the input element, as I suggested in one of the inline comments). You could cherry-pick the commit where I did that, or simply merge in master if my PR gets merged first.

I've mainly looked at the code so far. I'd like to get some time with you to go through the functionality. I'm having issues getting TSVB to display data, but this is an issue for me in master as well, so I think I'm doing something wrong.

src/legacy/core_plugins/metrics/public/components/splits/filter_items.js

src/legacy/core_plugins/metrics/public/components/vis_types/table/config.js

src/legacy/core_plugins/metrics/public/components/vis_types/timeseries/config.js

src/legacy/core_plugins/metrics/public/components/vis_editor.js

elasticmachine · 2019-06-04T21:51:46Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

…ved_objects data for ecommerce, flights and logs TSVB visualizations

…e optional

elasticmachine · 2019-06-05T00:56:31Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

Bargs

Changes look good, just a couple things that I noticed:

One of the sample saved objects that has a filter string still needs updated: "[eCommerce] Sold Products per Day”
Still need to remove screenTitle prop everywhere now that it's optional

elasticmachine · 2019-06-05T18:41:14Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

… as 'logstash-*'

Bargs

LGTM!

elasticmachine · 2019-06-05T22:51:33Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

* Squashed commit of the following: commit 0cff8243a99ff23421c3cbb15d62b26332a31e21 Author: Christiane Heiligers <christiane.heiligers@elastic.co> Date: Mon May 20 16:16:15 2019 -0700 Adds ignore_lobal_filter YesNo component back Deletes fixtures file commit e54e1690c9c46e2de5e747d14030a7e27e74c138 Author: Christiane Heiligers <christiane.heiligers@elastic.co> Date: Mon May 20 14:39:21 2019 -0700 squash-merges adding KQL with the full QueryBar and SavedObject migrations Debugging Adds filter defaults Updates index pattern for the QueryBar on componentDidUpdate squash-merges adding KQL with the full QueryBar and SavedObject migrations required Adds ignore_lobal_filter YesNo component back Deletes fixtures file Updates migration test number * Changes QueryBar to QueryBarInput in TimeseriesPanelConfigUi Replaces QueryBar with QueryBarInput in Timeseries Changes QueryBar to QueryBarInput in TSVB Metric Changes QueryBar to QueryBarInput in other TSVB visualisation editors Removes async from componentWillMount changes indexPattern to indexPatternObject in series and table query request processors * Squashed commit of the following: commit 2c3107963cff20b2d24323fddfaf80c848b86848 Author: Christiane Heiligers <christiane.heiligers@elastic.co> Date: Thu May 23 15:15:06 2019 -0700 Cleans up comments commit 943150d4b9b71183e9d5b26c95449ea4519aad4c Author: Christiane Heiligers <christiane.heiligers@elastic.co> Date: Thu May 23 14:09:53 2019 -0700 Cleans up parser check more commit d31de88525595720556440ae71515f5772774eaa Author: Christiane Heiligers <christiane.heiligers@elastic.co> Date: Thu May 23 14:01:02 2019 -0700 Cleans up parser check commit b592071b1d939ec2df9b9c921d57a22dac54e703 Author: Christiane Heiligers <christiane.heiligers@elastic.co> Date: Thu May 23 13:16:55 2019 -0700 skips server call on invalid kuery queries Cleans up Deletes duplicate migration function Adds 's' to split_by_filters method Resolves issues after rebase * Removes quotes around index pattern strings passed into fetch_index_patterns, adds comments on reasoning * queryBar to queryBarInput Adds transformFilterStringToQueryObject back to visualizations migrations that was lost during rebase * Deletes unused code * Fixes migrations test * compiles visualizations migrations for version 7.3.0 * temporary change to query_bar_input fetch_index_patterns file formatting Includes code from PR#37413 Clean up * Formatting * Uses series index pattern when override_index_pattern is selected and an index pattern is provided * Removes empty object as alternative to localStorage * Removes handelQueryChange from where it is not needed * extracts retrieval of the default query language from uiSettings into a helper method * Resolves some PR comments * Handles Query Error from an invalid syntax query * Converts string queries into objects with kuery as the language in saved_objects data for ecommerce, flights and logs TSVB visualizations * Deletes unused translation items * Adds optional id prop to the query_bar_input and makes the screenTitle optional * Updates [eCommerce] Sold Products per Day * Removes screenTitle from component QueryBarInput * Wraps index pattern strings in a double quote to allow for names such as 'logstash-*'

TinaHeiligers requested review from a team and Bargs May 21, 2019 15:15

TinaHeiligers mentioned this pull request May 21, 2019

[WIP] KQL in TSVB with full QueryBar #36175

Closed

11 tasks

TinaHeiligers force-pushed the KQL_in_TSVB branch from f5ffb4a to badc57d Compare May 22, 2019 17:04

TinaHeiligers force-pushed the KQL_in_TSVB branch from 409410c to 101523c Compare May 23, 2019 14:58

TinaHeiligers force-pushed the KQL_in_TSVB branch from 101523c to b285f7a Compare May 23, 2019 22:20

TinaHeiligers force-pushed the KQL_in_TSVB branch 2 times, most recently from 1f9ae36 to 6a91b0c Compare May 24, 2019 18:24

TinaHeiligers force-pushed the KQL_in_TSVB branch 2 times, most recently from aa27c84 to 352e227 Compare May 24, 2019 21:58

TinaHeiligers force-pushed the KQL_in_TSVB branch from 1a26770 to 9239571 Compare May 28, 2019 15:40

TinaHeiligers force-pushed the KQL_in_TSVB branch from 906d6b6 to 4c11e28 Compare May 28, 2019 16:41

TinaHeiligers added v8.0.0 release_note:enhancement labels Jun 3, 2019

Bargs suggested changes Jun 3, 2019

View reviewed changes

TinaHeiligers added 2 commits June 4, 2019 09:38

Merge branch 'master' of github.com:elastic/kibana into KQL_in_TSVB

bcf47cc

Resolves some PR comments

f591233

TinaHeiligers added 4 commits June 4, 2019 15:13

Handles Query Error from an invalid syntax query

b72f77d

Converts string queries into objects with kuery as the language in sa…

ac1aa6d

…ved_objects data for ecommerce, flights and logs TSVB visualizations

Deletes unused translation items

7015e00

Adds optional id prop to the query_bar_input and makes the screenTitl…

8365673

…e optional

TinaHeiligers requested a review from Bargs June 4, 2019 23:58

Bargs suggested changes Jun 5, 2019

View reviewed changes

TinaHeiligers added 3 commits June 5, 2019 09:20

Merge branch 'master' of github.com:elastic/kibana into KQL_in_TSVB

32f3eda

Updates [eCommerce] Sold Products per Day

7d9a7bc

Removes screenTitle from component QueryBarInput

1bd40e6

Wraps index pattern strings in a double quote to allow for names such…

8506fd5

… as 'logstash-*'

Bargs approved these changes Jun 5, 2019

View reviewed changes

TinaHeiligers merged commit 84b595e into elastic:master Jun 5, 2019

TinaHeiligers deleted the KQL_in_TSVB branch June 5, 2019 23:07

TinaHeiligers mentioned this pull request Jun 5, 2019

Support KQL in TSVB Filter aggregation #35437

Closed

TinaHeiligers mentioned this pull request Jun 6, 2019

[7.x] KQL in TSVB (#36784) #38333

Merged

alexwizp mentioned this pull request Jun 25, 2019

[TSVB] Group by Terms functionality not working propertly #39580

Closed

tylersmalley mentioned this pull request Dec 10, 2019

Moving saved object management to an ES plugin #49764

Closed

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

KQL in TSVB #36784

KQL in TSVB #36784

TinaHeiligers commented May 21, 2019 •

edited

Loading

elasticmachine commented May 21, 2019

TinaHeiligers commented May 21, 2019

elasticmachine commented May 21, 2019

TinaHeiligers commented May 22, 2019

elasticmachine commented May 22, 2019

elasticmachine commented May 22, 2019

elasticmachine commented May 23, 2019

elasticmachine commented May 23, 2019

elasticmachine commented May 23, 2019

elasticmachine commented May 24, 2019

elasticmachine commented May 24, 2019

elasticmachine commented May 24, 2019

elasticmachine commented May 24, 2019

elasticmachine commented May 24, 2019

elasticmachine commented May 24, 2019

elasticmachine commented May 28, 2019

elasticmachine commented May 28, 2019

elasticmachine commented May 28, 2019

Bargs left a comment

elasticmachine commented Jun 4, 2019

elasticmachine commented Jun 5, 2019

Bargs left a comment

elasticmachine commented Jun 5, 2019

Bargs left a comment

elasticmachine commented Jun 5, 2019

KQL in TSVB #36784

KQL in TSVB #36784

Conversation

TinaHeiligers commented May 21, 2019 • edited Loading

Summary

Checklist

For maintainers

elasticmachine commented May 21, 2019

💔 Build Failed

TinaHeiligers commented May 21, 2019

elasticmachine commented May 21, 2019

💚 Build Succeeded

TinaHeiligers commented May 22, 2019

elasticmachine commented May 22, 2019

💚 Build Succeeded

elasticmachine commented May 22, 2019

💚 Build Succeeded

elasticmachine commented May 23, 2019

💚 Build Succeeded

elasticmachine commented May 23, 2019

💔 Build Failed

elasticmachine commented May 23, 2019

💔 Build Failed

elasticmachine commented May 24, 2019

💚 Build Succeeded

elasticmachine commented May 24, 2019

💔 Build Failed

elasticmachine commented May 24, 2019

💔 Build Failed

elasticmachine commented May 24, 2019

💚 Build Succeeded

elasticmachine commented May 24, 2019

💔 Build Failed

elasticmachine commented May 24, 2019

💚 Build Succeeded

elasticmachine commented May 28, 2019

💚 Build Succeeded

elasticmachine commented May 28, 2019

💔 Build Failed

elasticmachine commented May 28, 2019

💔 Build Failed

Bargs left a comment

Choose a reason for hiding this comment

elasticmachine commented Jun 4, 2019

💔 Build Failed

elasticmachine commented Jun 5, 2019

💚 Build Succeeded

Bargs left a comment

Choose a reason for hiding this comment

elasticmachine commented Jun 5, 2019

💚 Build Succeeded

Bargs left a comment

Choose a reason for hiding this comment

elasticmachine commented Jun 5, 2019

💚 Build Succeeded

TinaHeiligers commented May 21, 2019 •

edited

Loading