[ML] Categorization wizard #53009

jgowdyelastic · 2019-12-13T17:08:44Z

Adds a wizard for creating categorization jobs.

User can select a Count or Rare detector and the categorization field from a list text and keyword fields.
When a field is selected, a list of example field values is shown with highlighted tokens based on the default analyser.

The analyzer can be edited by the user which will reload these examples.

A validation message is also displayed to show the results of the token counts per field value.
100 values are checked and a percentage show based on how many have more than one token.
If this percentage is less than 75%, the user is warned that field might not be good for categorization.

If the percentage is less than 2%, the use of that field is not allowed.

When running the job, an event rate chart is shown with anomalies.

Along with a list of example categories. If model plot is enabled, this list will also have a count per category and be sorted by the most popular category.

Checklist

Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR.

This was checked for cross-browser compatibility, including a check against IE11
Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
This was checked for keyboard-only and screenreader accessibility

For maintainers

This was checked for breaking API changes and was labeled appropriately
This includes a feature addition or change that requires a release note and was labeled appropriately

elasticmachine · 2019-12-20T19:11:38Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request
Commit: c091cca1e5b47262a729cc5eb85a57727ad5b251

History

💔 Build #16928 failed 29eb049a79a9b9e9a65edf5258b62b797f6c0662
💔 Build #16898 failed 47534015614e34134dcbf1ff7ee695113b394340
💔 Build #16422 failed 2f91dbde24909a9fe11605e231e25585a7df39ea
💔 Build #15680 failed 4f13dedb24008dcbfff6420567003aef692e4ac2
💔 Build #15466 failed 248836e12e810cc7817116f9015bb45da3aaff66

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

elasticmachine · 2020-01-07T16:18:46Z

Pinging @elastic/ml-ui (:ml)

x-pack/legacy/plugins/ml/server/models/job_service/new_job/categorization.ts

x-pack/legacy/plugins/ml/public/application/jobs/new_job/common/job_validator/job_validator.ts

...omponents/common/edit_categorization_analyzer_flyout/edit_categorization_analyzer_flyout.tsx

.../pages/components/pick_fields_step/components/categorization_view/examples_valid_callout.tsx

.../new_job/pages/components/pick_fields_step/components/categorization_view/field_examples.tsx

...ew_job/pages/components/pick_fields_step/components/categorization_view/metric_selection.tsx

x-pack/legacy/plugins/ml/common/types/categories.ts

...omponents/common/edit_categorization_analyzer_flyout/edit_categorization_analyzer_flyout.tsx

darnautov · 2020-01-08T13:28:23Z

...s/components/pick_fields_step/components/categorization_detector/categorization_detector.tsx

+        </h3>
+      </EuiTitle>
+      <EuiSpacer size="s" />
+      <EuiFlexGroup gutterSize="l" style={{ maxWidth: '824px' }}>


can we replace this inline style?

..._job/pages/components/pick_fields_step/components/categorization_detector/detector_cards.tsx

.../pages/components/pick_fields_step/components/categorization_view/examples_valid_callout.tsx

.../new_job/pages/components/pick_fields_step/components/categorization_view/field_examples.tsx

...ages/components/pick_fields_step/components/categorization_view/metric_selection_summary.tsx

x-pack/legacy/plugins/ml/server/models/job_service/new_job/categorization.ts

peteharverson

The text for the categorization field should be edited - removing the first part about it being optional. Maybe something like 'Specifies which field will be categorized. Using text data types is recommended.'

..._job/pages/components/pick_fields_step/components/categorization_detector/detector_cards.tsx

peteharverson

Latest edits LGTM

kibanamachine · 2020-01-09T15:12:28Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 80b802f

History

💚 Build #18943 succeeded 08054ef53e4d7ace84d178ce0f7a091239bc3925
💚 Build #18924 succeeded 218447375720a369acd2969ddffa661400c89fef
💚 Build #18785 succeeded ba099a54a40235e5f10c9eaa49bb5bf16ceaec19
💔 Build #18759 failed e0ba6efbb07612236bafbb3b05a106e78eb0cf34
💔 Build #18743 failed b7ad6cfb2afbc69e4e49846c2c1221f067c206b3

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* [ML] Categorization wizard * fixing js prettier issues * adding basic category field validation * adding rare or count selection * fixing types * category examples changes * improving results search * adding analyzer editing * improving callout * updating callout text * fixing import path * resetting cat analyser json on flyout open * disabling model plot by default * minor refactoring * fixing types * hide estimate bucket span * setting default bucket span * removing ml_classic workaround * changing style of detector selection * fixing convert to advanced issue * removing sparse data checkbox * changes based on review * use default mml * fixing job cloning * changes based on review * removing categorization_analyzer from job if it is same as default * fixing translations * disabling model plot for rare jobs * removing console.error in useResolver

jgowdyelastic force-pushed the categorization-wizard branch 3 times, most recently from 651d7db to 4753401 Compare December 20, 2019 13:23

jgowdyelastic force-pushed the categorization-wizard branch 4 times, most recently from 076162f to f8a975e Compare January 7, 2020 13:57

jgowdyelastic marked this pull request as ready for review January 7, 2020 16:16

jgowdyelastic requested a review from a team as a code owner January 7, 2020 16:16

jgowdyelastic requested review from peteharverson, Jeremy-Walters, darnautov and alvarezmelissa87 January 7, 2020 16:17

jgowdyelastic self-assigned this Jan 7, 2020

jgowdyelastic added :ml release_note:enhancement review labels Jan 7, 2020

jgowdyelastic added v7.6.0 v8.0.0 Feature:Anomaly Detection ML anomaly detection labels Jan 7, 2020

droberts195 reviewed Jan 7, 2020

View reviewed changes

x-pack/legacy/plugins/ml/server/models/job_service/new_job/categorization.ts Outdated Show resolved Hide resolved

jgowdyelastic requested review from walterra and removed request for Jeremy-Walters January 8, 2020 12:25

peteharverson reviewed Jan 8, 2020

View reviewed changes

jgowdyelastic force-pushed the categorization-wizard branch from 01a4b21 to f6fbc8f Compare January 8, 2020 13:52

darnautov reviewed Jan 8, 2020

View reviewed changes

peteharverson reviewed Jan 8, 2020

View reviewed changes

..._job/pages/components/pick_fields_step/components/categorization_detector/detector_cards.tsx Outdated Show resolved Hide resolved

sophiec20 mentioned this pull request Jan 8, 2020

[ML] Warn if ML categorization job is using data that does not categorize well elastic/elasticsearch#50749

Closed

jgowdyelastic added 20 commits January 9, 2020 13:51

updating callout text

d38050f

fixing import path

868da11

resetting cat analyser json on flyout open

6f0f363

disabling model plot by default

84acf3c

minor refactoring

1955044

fixing types

03235c3

hide estimate bucket span

42cc293

setting default bucket span

9b5cce9

removing ml_classic workaround

f982bea

changing style of detector selection

8130a34

fixing convert to advanced issue

7593215

removing sparse data checkbox

529f109

changes based on review

2b4f3b8

use default mml

74d31c8

fixing job cloning

48c1857

changes based on review

2141d04

removing categorization_analyzer from job if it is same as default

b5d4fca

fixing translations

81ca64c

disabling model plot for rare jobs

e869c4b

removing console.error in useResolver

80b802f

jgowdyelastic force-pushed the categorization-wizard branch from 08054ef to 80b802f Compare January 9, 2020 13:52

peteharverson approved these changes Jan 9, 2020

View reviewed changes

jgowdyelastic merged commit 36abed3 into elastic:master Jan 9, 2020

jgowdyelastic deleted the categorization-wizard branch January 9, 2020 15:21

jgowdyelastic mentioned this pull request Jan 9, 2020

[7.x] [ML] Categorization wizard (#53009) #54353

Merged

This was referenced Jan 14, 2020

[DOCS] Adds categorization wizard elastic/stack-docs#804

Merged

[DOCS] Updates categorization examples with wizard screenshots elastic/elasticsearch#51133

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ML] Categorization wizard #53009

[ML] Categorization wizard #53009

jgowdyelastic commented Dec 13, 2019 •

edited

Loading

elasticmachine commented Dec 20, 2019

elasticmachine commented Jan 7, 2020

darnautov Jan 8, 2020

peteharverson left a comment

peteharverson left a comment

kibanamachine commented Jan 9, 2020

[ML] Categorization wizard #53009

[ML] Categorization wizard #53009

Conversation

jgowdyelastic commented Dec 13, 2019 • edited Loading

Checklist

For maintainers

elasticmachine commented Dec 20, 2019

💔 Build Failed

History

elasticmachine commented Jan 7, 2020

darnautov Jan 8, 2020

Choose a reason for hiding this comment

peteharverson left a comment

Choose a reason for hiding this comment

peteharverson left a comment

Choose a reason for hiding this comment

kibanamachine commented Jan 9, 2020

💚 Build Succeeded

History

jgowdyelastic commented Dec 13, 2019 •

edited

Loading