[ML] DF Analytics: Creation wizard part 1 #67564

alvarezmelissa87 · 2020-05-27T22:09:29Z

Summary

Related meta issue: #66661
This is the first PR for switching over to the new creation wizard.

To be addressed in follow ups:

job cloning
hyper parameter validation via explain API
- disable continue if there is invalid input in the Hyper parameters form
remove deprecated code and state handling refactor
misc style tweaks

NOTE:

Cloning action and tests have been commented out for now until cloning has been updated for the new wizard

Adds source selection modal for saved searches/indices:

Config step:

Preview source data:

Shows table of included fields for excludes selection

Additional Options

Job details

Creation

View summary of each step and select to start job immediately

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Unit or functional tests were updated or added to match the most common scenarios
This was checked for keyboard-only and screenreader accessibility
This was checked for cross-browser compatibility, including a check against IE11

elasticmachine · 2020-05-27T23:08:42Z

Pinging @elastic/ml-ui (:ml)

qn895 · 2020-05-28T00:46:54Z

For this one, and it's more of a user experience thing, I think it's more intuitive to either mark the Job ID and the Destination Index as required or at least somehow highlight that I need to input something before the continue button becomes enabled.

Also, turning on the JSON editor after creating the job will show just this without the "return to the job list" button.

The search index feature doesn't work with - or _

x-pack/plugins/ml/public/application/data_frame_analytics/common/analytics.ts

x-pack/plugins/ml/public/application/data_frame_analytics/common/index.ts

...ication/data_frame_analytics/pages/analytics_creation/components/create_step/create_step.tsx

...ta_frame_analytics/pages/analytics_creation/components/details_step/details_step_details.tsx

...ion/data_frame_analytics/pages/analytics_management/hooks/use_create_analytics_form/state.ts

...analytics/pages/analytics_creation/components/configuration_step/configuration_step_form.tsx

...ata_frame_analytics/pages/analytics_creation/components/advanced_step/advanced_step_form.tsx

.../data_frame_analytics/pages/analytics_creation/components/advanced_step/hyper_parameters.tsx

peteharverson · 2020-06-01T10:47:03Z

.../data_frame_analytics/pages/analytics_creation/components/advanced_step/hyper_parameters.tsx

+              'Regularization parameter to prevent overfitting on the training data set. Must be a non negative value',
+          })}
+        >
+          <EuiFieldNumber


Can the Continue button be disabled if there is invalid input in the Hyper parameters form?

Yes - that's planned for a follow-up as we will need to hit the explain API to validate this area and that requires some refactoring.

.../data_frame_analytics/pages/analytics_creation/components/advanced_step/hyper_parameters.tsx

...e_analytics/pages/analytics_creation/components/configuration_step/analysis_fields_table.tsx

...ication/data_frame_analytics/pages/analytics_creation/components/create_step/create_step.tsx

peteharverson · 2020-06-01T11:30:51Z

...ication/data_frame_analytics/pages/analytics_creation/components/create_step/create_step.tsx

+
+  return (
+    <Fragment>
+      <Messages messages={requestMessages} />


Maybe we should clear the messages if the user goes back to edit some of the settings to correct the original errors? Otheriwse it can be slighly confusing when the messages are still there when they try to create the job a second time.

Will address in the follow up for getting job cloning working 👌

x-pack/test/functional/apps/ml/data_frame_analytics/classification_creation.ts

walterra

Great job overall! Added some comments, but nothing major.

walterra · 2020-06-03T14:13:16Z

x-pack/plugins/ml/public/application/data_frame_analytics/common/analytics.ts

@@ -421,7 +436,7 @@ export function getEvalQueryBody({

  const searchQueryClone = cloneDeep(searchQuery);

-  if (isResultsSearchBoolQuery(searchQueryClone)) {
+  if (searchQueryClone && isResultsSearchBoolQuery(searchQueryClone)) {


As isResultsSearchBoolQuery is a type guard can we move the check against searchQueryClone && inside that function?

Moved in cbdf3d5495501b77667dcb273df7817594782d25

walterra · 2020-06-03T14:13:32Z

x-pack/plugins/ml/public/application/data_frame_analytics/common/analytics.ts

@@ -431,7 +446,7 @@ export function getEvalQueryBody({
    }

    query = searchQueryClone;
-  } else if (isQueryStringQuery(searchQueryClone)) {
+  } else if (searchQueryClone && isQueryStringQuery(searchQueryClone)) {


As isQueryStringQuery is a type guard can we move the check against searchQueryClone && inside that function?

Updated in cbdf3d5495501b77667dcb273df7817594782d25

walterra · 2020-06-03T14:18:29Z

..._frame_analytics/pages/analytics_creation/components/advanced_step/advanced_step_details.tsx

+  const advancedFirstCol: ListItems[] = [];
+  const advancedSecondCol: ListItems[] = [];
+  const advancedThirdCol: ListItems[] = [];
+
+  const hyperFirstCol: ListItems[] = [];
+  const hyperSecondCol: ListItems[] = [];
+  const hyperThirdCol: ListItems[] = [];


Nit: For readability and the ability to write a unit test in isolation we could move all of this list items preparation (also the code below this that set's up the translate strings) to a helper function or custom hook.

Happy to do this in a follow-up 👍

walterra · 2020-06-03T14:21:40Z

..._frame_analytics/pages/analytics_creation/components/advanced_step/advanced_step_details.tsx

+    standardizationEnabled,
+  } = form;
+
+  const isDepVarJob =


Nit: In another file the same check is called isRegOrClassJob . We should pick one for consistency or move both usages to a helper function.

Updated in cbdf3d5495501b77667dcb273df7817594782d25

walterra · 2020-06-03T14:22:05Z

...ata_frame_analytics/pages/analytics_creation/components/advanced_step/advanced_step_form.tsx

+    modelMemoryLimitValidationResult,
+  ]);
+
+  const isRegOrClassJob =


Nit: In another file the same check is called isDepVarJob. We should pick one for consistency or move both usages to a helper function.

Updated in cbdf3d5495501b77667dcb273df7817594782d25

walterra · 2020-06-03T14:50:47Z

...nalytics/pages/analytics_creation/components/configuration_step/supported_fields_message.tsx

+const containsOutlierFieldsCb = ({ name, type }: Field) =>
+  !OMIT_FIELDS.includes(name) && name !== EVENT_RATE_FIELD_ID && BASIC_NUMERICAL_TYPES.has(type);
+
+const callbacks: any = {


any could be something like Record<ANALYSIS_CONFIG_TYPE, (f: Field) => boolean>.

walterra · 2020-06-03T14:51:32Z

...nalytics/pages/analytics_creation/components/configuration_step/supported_fields_message.tsx

+  [ANALYSIS_CONFIG_TYPE.OUTLIER_DETECTION]: containsOutlierFieldsCb,
+};
+
+const messages: any = {


any could be something like Record<ANALYSIS_CONFIG_TYPE, JSX.Element>.

walterra · 2020-06-03T14:59:56Z

...ion/data_frame_analytics/pages/analytics_management/hooks/use_create_analytics_form/state.ts

+    jobConfig?.analysis?.classification !== undefined &&
+    formState.numTopClasses !== undefined
+  ) {
+    // @ts-ignore


Can you add a note why the ignore is necessary?

walterra · 2020-06-03T15:00:26Z

...ion/data_frame_analytics/pages/analytics_management/hooks/use_create_analytics_form/state.ts

+        standardization_enabled: formState.standardizationEnabled,
+      }
+    );
+    // @ts-ignore


Please add a note why the ignore is necessary.

walterra · 2020-06-03T15:00:44Z

...ion/data_frame_analytics/pages/analytics_management/hooks/use_create_analytics_form/state.ts

@@ -279,6 +356,11 @@ export function getCloneFormStateFromJobConfig(
    resultState.dependentVariable = analysisConfig.dependent_variable;
    resultState.numTopFeatureImportanceValues = analysisConfig.num_top_feature_importance_values;
    resultState.trainingPercent = analysisConfig.training_percent;
+
+    if (isClassificationAnalysis(analyticsJobConfig.analysis)) {
+      // @ts-ignore


Please add a note why the ignore is necessary.

pheyos

Functional test changes LGTM 🎉

pheyos · 2020-06-03T15:38:43Z

Checking stability of the test changes in https://kibana-ci.elastic.co/job/kibana+flaky-test-suite-runner/505/ ...

For some reason, Jenkins has issues fetching the branches of this fork, so the flaky runner doesn't start.

peteharverson · 2020-06-04T11:45:23Z

...ck/plugins/ml/public/application/components/custom_selection_table/custom_selection_table.js

-          key={item.id}
-          isSelected={isItemSelected(item.id)}
+          key={item[tableItemId]}
+          isSelected={isItemSelected(item[tableItemId])}
          isSelectable={true}
          hasActions={true}
          data-test-subj="mlFlyoutJobSelectorTableRow"


This should be changed to something like mlCustomSelectionTableRow. The mlFlyoutJobSelectorTable type data-test-subj is used in three places I think in this file.

Updated in fdf122f9dce26a106075d73672c0510fa95dd2f8

peteharverson · 2020-06-04T11:48:56Z

...e_analytics/pages/analytics_creation/components/configuration_step/analysis_fields_table.tsx

+    id: 'checkbox',
+    isCheckbox: true,
+    textOnly: false,
+    width: '24px',


This needs to be 32px to match the width of the euiTableRowCellCheckbox. This ensures the selection border isn't cropped.

24px:

32px:

Updated to 32px in fdf122f9dce26a106075d73672c0510fa95dd2f8

peteharverson · 2020-06-04T11:53:10Z

...ugins/ml/public/application/components/job_selector/job_selector_table/job_selector_table.js

@@ -157,6 +157,9 @@ export function JobSelectorTable({
        filterDefaultFields={!singleSelection ? JOB_FILTER_FIELDS : undefined}
        items={jobs}
        onTableChange={(selectionFromTable) => onSelection({ selectionFromTable })}
+        radioDisabledCheck={(item) => {


Not related to this PR, but as this file is being touched by this PR, could you set the width of the first checkbox column to 32px to stop the selection border being cropped. It is used twice - once for the jobs table, and once for the groups table. It needs to match the width of the euiTableRowCellCheckbox.

Updated to 32px in fdf122f9dce26a106075d73672c0510fa95dd2f8

peteharverson

Latest edits LGTM

walterra

Latest changes LGTM

kibanamachine · 2020-06-04T16:17:28Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: e984987

History

💚 Build #51757 succeeded fdf122f9dce26a106075d73672c0510fa95dd2f8
💚 Build #51558 succeeded cbdf3d5495501b77667dcb273df7817594782d25
💚 Build #51429 succeeded 0406a647b262b1c0a943f03e3056141005938aff
💚 Build #51400 succeeded fbe09905a15b818282298e69d66cfda9b477083a
💚 Build #51386 succeeded 6dee42063a219842c7a5d068310ce07d331c4dde

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* create newJob route and start of wizard * wip: create configStep component * finish configStep form and details * wip: create andvanced step components * create details step component * createStep component * ensure advanced options are correct for each job type * add validation to each step * use custom table for excludes * move customSelectionTable to shared components * form validation for advanced fields * wip: source index selection modal * add source index preview * update details * ensure advanced parameters added to config on creation * can create job from savedSearch. can set source query in ui * validate source object has supported fields * eslint updates * update tests. comment out clone action for now * add create button to advanced editor * remove deprecated test helper functions * fix translation errors * update help text. read only once job created. * fix functional tests * add nextStepNav to df service for tests * fix excludes table page jump and hyperParameter not showing in details * fix checkbox width for custom table

alvarezmelissa87 requested review from a team as code owners May 27, 2020 22:09

alvarezmelissa87 self-assigned this May 27, 2020

alvarezmelissa87 added :ml Feature:Data Frame Analytics ML data frame analytics features release_note:enhancement v7.9.0 labels May 27, 2020

alvarezmelissa87 force-pushed the ml-df-analytics-creation-wizard branch 2 times, most recently from 4be5357 to d0b23d9 Compare May 31, 2020 23:06

alvarezmelissa87 requested review from peteharverson and walterra June 1, 2020 02:10

peteharverson reviewed Jun 1, 2020

View reviewed changes

...ication/data_frame_analytics/pages/analytics_creation/components/create_step/create_step.tsx Outdated Show resolved Hide resolved

peteharverson reviewed Jun 1, 2020

View reviewed changes

alvarezmelissa87 requested review from peteharverson, qn895 and a team June 1, 2020 13:21

alvarezmelissa87 force-pushed the ml-df-analytics-creation-wizard branch from 706f449 to 6dee420 Compare June 3, 2020 13:40

pheyos requested changes Jun 3, 2020

View reviewed changes

x-pack/test/functional/apps/ml/data_frame_analytics/classification_creation.ts Outdated Show resolved Hide resolved

walterra reviewed Jun 3, 2020

View reviewed changes

pheyos approved these changes Jun 3, 2020

View reviewed changes

peteharverson reviewed Jun 4, 2020

View reviewed changes

peteharverson approved these changes Jun 4, 2020

View reviewed changes

walterra approved these changes Jun 4, 2020

View reviewed changes

alvarezmelissa87 added 2 commits June 4, 2020 10:35

create newJob route and start of wizard

661b55b

wip: create configStep component

a002ba2

alvarezmelissa87 added 21 commits June 4, 2020 10:35

ensure advanced options are correct for each job type

633ccf1

add validation to each step

70185db

use custom table for excludes

1461140

move customSelectionTable to shared components

fdca5fd

form validation for advanced fields

3aecbc7

wip: source index selection modal

11dd4e1

add source index preview

12cf4b4

update details

dae899c

ensure advanced parameters added to config on creation

a32ec5a

can create job from savedSearch. can set source query in ui

1e77fbe

validate source object has supported fields

29f80cb

eslint updates

68236b3

update tests. comment out clone action for now

d7f9234

add create button to advanced editor

0b40a5e

remove deprecated test helper functions

732b8c6

fix translation errors

1d1cb3f

update help text. read only once job created.

5a6a567

fix functional tests

2ef783e

add nextStepNav to df service for tests

336bad4

fix excludes table page jump and hyperParameter not showing in details

7c0a368

fix checkbox width for custom table

e984987

alvarezmelissa87 force-pushed the ml-df-analytics-creation-wizard branch from fdf122f to e984987 Compare June 4, 2020 14:35

alvarezmelissa87 merged commit edc4d58 into elastic:master Jun 4, 2020

alvarezmelissa87 mentioned this pull request Jun 4, 2020

[7.x] [ML] DF Analytics: Creation wizard part 1 (#67564) #68296

Merged

alvarezmelissa87 mentioned this pull request Jun 4, 2020

[Meta] [ML] DF Analytics creation wizard #66661

Closed

60 tasks

alvarezmelissa87 deleted the ml-df-analytics-creation-wizard branch June 4, 2020 19:17

pheyos mentioned this pull request Jun 8, 2020

[ML] DF Analytics: Creation wizard part 2 #68462

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ML] DF Analytics: Creation wizard part 1 #67564

[ML] DF Analytics: Creation wizard part 1 #67564

alvarezmelissa87 commented May 27, 2020 •

edited

Loading

elasticmachine commented May 27, 2020

qn895 commented May 28, 2020 •

edited

Loading

peteharverson Jun 1, 2020

alvarezmelissa87 Jun 2, 2020

peteharverson Jun 1, 2020

alvarezmelissa87 Jun 3, 2020

walterra left a comment

walterra Jun 3, 2020

alvarezmelissa87 Jun 3, 2020

walterra Jun 3, 2020

alvarezmelissa87 Jun 3, 2020

walterra Jun 3, 2020

alvarezmelissa87 Jun 3, 2020

walterra Jun 3, 2020

alvarezmelissa87 Jun 3, 2020

walterra Jun 3, 2020

alvarezmelissa87 Jun 3, 2020

walterra Jun 3, 2020

walterra Jun 3, 2020

walterra Jun 3, 2020

walterra Jun 3, 2020

walterra Jun 3, 2020

pheyos left a comment

pheyos commented Jun 3, 2020 •

edited

Loading

peteharverson Jun 4, 2020

alvarezmelissa87 Jun 4, 2020

peteharverson Jun 4, 2020

alvarezmelissa87 Jun 4, 2020

peteharverson Jun 4, 2020

alvarezmelissa87 Jun 4, 2020

peteharverson left a comment

walterra left a comment

kibanamachine commented Jun 4, 2020

[ML] DF Analytics: Creation wizard part 1 #67564

[ML] DF Analytics: Creation wizard part 1 #67564

Conversation

alvarezmelissa87 commented May 27, 2020 • edited Loading

Summary

NOTE:

Config step:

Preview source data:

Shows table of included fields for excludes selection

Additional Options

Job details

Creation

View summary of each step and select to start job immediately

Checklist

elasticmachine commented May 27, 2020

qn895 commented May 28, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

walterra left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pheyos left a comment

Choose a reason for hiding this comment

pheyos commented Jun 3, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

peteharverson left a comment

Choose a reason for hiding this comment

walterra left a comment

Choose a reason for hiding this comment

kibanamachine commented Jun 4, 2020

💚 Build Succeeded

History

alvarezmelissa87 commented May 27, 2020 •

edited

Loading

qn895 commented May 28, 2020 •

edited

Loading

pheyos commented Jun 3, 2020 •

edited

Loading