Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Add embedded map to geo_point fields for Data Visualizer #88880

Merged
merged 26 commits into from
Jan 27, 2021

Conversation

qn895
Copy link
Member

@qn895 qn895 commented Jan 20, 2021

Summary

This PR is part of #86387. It adds embedded maps to the geo_point fields for the data visualizer (both the file based and index based). Changes include:

  • A new <MlEmbeddComponent/> that can be consumed in other places inside Ml
  • Maps plugin as a required plugin
  • Updated functional tests with a new file for checking geo_point fields
  • Update the kibanaField formatting for the preview distribution for Number rows

Screen Shot 2021-01-20 at 11 51 03

Data Visualizer - Machine Learning - Elastic

Future Todos

  • Support for geo_shape fields

Checklist

Delete any items that are not applicable to this PR.

@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui (:ml)

Copy link
Member

@pheyos pheyos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really cool to have the maps in the data visualizer row expansions, great addition!

Functional tests in the file data viz LGTM, just one small suggestion.

I've noticed that the expanded row content is different in file data viz compared to the index data viz after import:

File data viz before import
image
This shows document stats, top values and the map.

Index data viz after import
image

This shows document stats, examples and the map.
Is this difference intended?

And related to that: The service method assertGeoPointFieldContents will currently only work for the file data viz (because of that difference). And it would be good to also have functional tests for maps in the index data viz, so we should either make both row contents the same or have separate service methods to validate.

updateIndexPatternSearchLayer();
}, [indexPattern, config.fieldName, combinedQuery]);

if (stats?.examples === undefined) return null;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is related to your PR, but for this filebeat-apache-2019.01.30 data set, no examples are displayed for the traefik.access.geoip.location field.

image

I remember seeing something like this in the card based version, which was related to the query we were using to get the examples - https://github.com/elastic/kibana/blob/7.9/x-pack/plugins/ml/public/application/datavisualizer/index_based/components/field_data_card/content_types/geo_point_content.tsx

If the query to get the examples returns no docs, we should probably show some sort of message in that cell saying something like No examples were obtained for this field

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this message here f537b8c
Screen Shot 2021-01-25 at 10 10 23

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Showing the No examples were obtained for this field is good here, but I think we can actually improve the search used to obtain the examples in ml/server/models/data_visualizer/data_visualizer.ts getFieldExamples too. I think we can edit that search to use the fields parameter in place of the current _source parameter. I gave this a quick try and examples now show up for the traefik.access.geoip.location field above:

image

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated here d44517f

The example lists are now using fields API, which will look different since the API auto detects and auto formats.
Screen Shot 2021-01-26 at 13 40 28

Previous format:

@peteharverson
Copy link
Contributor

@pheyos regarding #88880 (review), I think we have to show examples for the index based view as geo fields don't support aggregations. Showing the examples list for the file based view would work for me, if we want to use the same component for both. I guess the radius of the circles on the map gives an indication of the relative frequencies (which is what the top values list provides too).

@pheyos
Copy link
Member

pheyos commented Jan 21, 2021

I agree @peteharverson, showing the examples list in both views sounds good.

Copy link
Member

@pheyos pheyos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to see that file data viz and index data viz are showing the same components for the geo row expansion now 🎉
One suggestion and one comment about the failing tests:

Copy link
Member

@pheyos pheyos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Functional tests LGTM 🎉
Just one question around variable naming.

stats.examples.push(example);
if (stats.examples.length === maxExamples) {
break;
const doc: any = get(hits[i].fields, field);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you consider this instead of any?

Suggested change
const doc: any = get(hits[i].fields, field);
const doc: object[] | undefined = get(hits[i].fields, field);

Copy link
Contributor

@peteharverson peteharverson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested latest edits. LGTM. A great addition!

Copy link
Contributor

@darnautov darnautov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
ml 1723 1728 +5

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
maps 2.6MB 2.6MB +115.0B
ml 6.5MB 6.5MB +9.4KB
total +9.5KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
ml 56.6KB 57.2KB +537.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@qn895 qn895 merged commit da9ad2a into elastic:master Jan 27, 2021
@qn895 qn895 deleted the ml-embedded-map-data-viz branch January 27, 2021 17:39
qn895 added a commit to qn895/kibana that referenced this pull request Jan 27, 2021
@qn895 qn895 mentioned this pull request Feb 5, 2021
12 tasks
@lcawl lcawl added the release_note:feature Makes this part of the condensed release notes label Feb 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:File and Index Data Viz ML file and index data visualizer :ml release_note:enhancement release_note:feature Makes this part of the condensed release notes v7.12.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants