feat: continue trial from WebUI for multi-trial experiment #9589

gt2345 · 2024-06-27T20:47:09Z

Ticket

Description

Allow user to continue multi-trial experiments if applicable.
Multi-trial experiments that are able to continue must satisfy the following criteria:

The searcher type is random or grid
The state is cancelled or errored

Test Plan

Navigate the the experiment details page of a multi trial experiment, if the experiment is not able to continue, then the Continue Experiment button should not be available.
Find a multi-trial experiment with Continue Experiment button, clicking on the button will continue the experiment.

0627.mp4

Checklist

Changes have been manually QA'd
New features have been approved by the corresponding PM
User-facing API changes have the "User-facing API Change" label
Release notes have been added as a separate file under docs/release-notes/
See Release Note for details.
Licenses have been included for new code which was copied and/or modified from any external code

netlify · 2024-06-27T20:47:27Z

✅ Deploy Preview for determined-ui ready!

Name	Link
🔨 Latest commit	`2082ce5`
🔍 Latest deploy log	https://app.netlify.com/sites/determined-ui/deploys/668d8835afda170009a2c9e8
😎 Deploy Preview	https://deploy-preview-9589--determined-ui.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

codecov · 2024-06-27T20:47:32Z

Codecov Report

Attention: Patch coverage is 58.69565% with 19 lines in your changes missing coverage. Please review.

Project coverage is 45.21%. Comparing base (c40b861) to head (2082ce5).
Report is 42 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #9589      +/-   ##
==========================================
- Coverage   49.82%   45.21%   -4.61%     
==========================================
  Files        1247      923     -324     
  Lines      162287   121992   -40295     
  Branches     2888     2893       +5     
==========================================
- Hits        80855    55160   -25695     
+ Misses      81260    66660   -14600     
  Partials      172      172

Flag	Coverage Δ
harness	`?`
web	`46.16% <58.69%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
webui/react/src/types.ts	`99.68% <100.00%> (+<0.01%)`	⬆️
webui/react/src/utils/experiment.ts	`83.88% <92.30%> (+0.08%)`	⬆️
...ages/ExperimentDetails/ExperimentDetailsHeader.tsx	`73.62% <35.71%> (-1.35%)`	⬇️

... and 324 files with indirect coverage changes

keita-determined

i think its working correctly, but let me try to understand it more.
looks like this experiment (before this change) doesnt show continue button even though its a multi experiment with ASHA and cancelled.
do you have an example how it works right now?

keita-determined · 2024-06-27T22:22:19Z

webui/react/src/utils/experiment.ts

+    experiment?.numTrials === 1 ||
+    (experiment.state !== RunState.Completed &&
+      experiment.numTrials > 1 &&
+      ['random', 'grid'].includes(experiment.config?.searcher.name || ''))) &&


maybe ['random', 'grid'] can be moved out to the component with a good variable name, then export it for both components?
and this can be set instead of array
i'll defer the decision to you

Sure, added a new variable named ContinuableNonSingleSearcherName

keita-determined · 2024-06-27T22:22:36Z

webui/react/src/pages/ExperimentDetails/ExperimentDetailsHeader.tsx

+    });
+    const newPath = paths.experimentDetails(experiment.id);
+    routeToReactUrl(paths.reload(newPath));
+  } finally {


no catch clause?

probably worth notifying the user if the API fails to continue the experiment

keita-determined · 2024-06-27T22:27:37Z

webui/react/src/pages/ExperimentDetails/ExperimentDetailsHeader.tsx

@@ -263,6 +265,19 @@ const ExperimentDetailsHeader: React.FC<Props> = ({
    setIsRunningDelete(experiment.state === RunState.Deleting);
  }, [experiment.state]);

+  const onClickContinueMultiTrialExp = useCallback(async () => {
+    try {


its not formatted? indentation doesnt look aligned

keita-determined · 2024-06-27T22:51:06Z

webui/react/src/utils/experiment.ts

-  (!!trial || experiment?.numTrials === 1) &&
+  (!!trial ||
+    experiment?.numTrials === 1 ||
+    (experiment.state !== RunState.Completed &&


experiment.state !== RunState.Completed is it needed?
we have terminalRunStates.has(experiment.state)

to add to this part, terminalRunStates include:

RunState.Canceled, RunState.Completed, RunState.Error, RunState.DeleteFailed, RunState.Deleted,

It seems like if it's Completed or Deleted it probably can not continue the trials. Might be worth defining a new curated list called continuableRunStates?

Continue on multi trial experiment is only available for Error and Cancel state, so I ended up using the existing erroredRunStates

hkang1

Most of it looks great, just a few of questions and suggestions

hkang1 · 2024-06-28T15:50:41Z

webui/react/src/utils/experiment.ts

-  (!!trial || experiment?.numTrials === 1) &&
+  (!!trial ||
+    experiment?.numTrials === 1 ||
+    (experiment.state !== RunState.Completed &&


to add to this part, terminalRunStates include:

RunState.Canceled, RunState.Completed, RunState.Error, RunState.DeleteFailed, RunState.Deleted,

It seems like if it's Completed or Deleted it probably can not continue the trials. Might be worth defining a new curated list called continuableRunStates?

hkang1 · 2024-06-28T15:52:18Z

webui/react/src/pages/ExperimentDetails/ExperimentDetailsHeader.tsx

              key: 'continue-trial',
              label: experiment.unmanaged ? (
                <Tooltip content={UNMANAGED_MESSAGE}>Continue Trial</Tooltip>
              ) : (
                'Continue Trial'
              ),
-              onClick: ContinueTrialModal.open,
+              onClick: ['random', 'grid'].includes(experiment?.config.searcher.name) ? onClickContinueMultiTrialExp : ContinueTrialModal.open,


suggestion: looks like ['random', 'grid'].includes(experiment?.config.searcher.name) is done more than once, could have a memoized variable isSearcherContinuable or something that does this operation. Or even a more general one that also checks the RunState called isContinuable

hkang1 · 2024-06-28T15:53:58Z

webui/react/src/pages/ExperimentDetails/ExperimentDetailsHeader.tsx

+    });
+    const newPath = paths.experimentDetails(experiment.id);
+    routeToReactUrl(paths.reload(newPath));
+  } finally {


probably worth notifying the user if the API fails to continue the experiment

gt2345 · 2024-06-28T18:43:04Z

i think its working correctly, but let me try to understand it more. looks like this experiment (before this change) doesnt show continue button even though its a multi experiment with ASHA and cancelled. do you have an example how it works right now?

Hi @keita-determined not sure I completely understand your concerns, but multi experiments are only able to continue if the searcher is grid or random, so for ASHA experiments this feature won't be available

keita-determined

\o/

gt2345 added 2 commits June 26, 2024 12:04

show continue btn

1490432

continue trial for multi-trial exp

1b3eac8

gt2345 requested a review from a team as a code owner June 27, 2024 20:47

gt2345 requested a review from hkang1 June 27, 2024 20:47

cla-bot bot added the cla-signed label Jun 27, 2024

gt2345 requested a review from keita-determined June 27, 2024 20:48

keita-determined reviewed Jun 27, 2024

View reviewed changes

hkang1 reviewed Jun 28, 2024

View reviewed changes

gt2345 added 2 commits June 28, 2024 13:33

use named variable

ca43835

catch API error

c7a94ee

gt2345 added 3 commits June 28, 2024 13:46

fix can continue condition

3905297

continue condition

88b6aa7

continue condition

11cda35

gt2345 requested a review from keita-determined June 28, 2024 20:55

keita-determined approved these changes Jul 1, 2024

View reviewed changes

update btn

2082ce5

gt2345 merged commit 58fbf68 into main Jul 9, 2024
84 of 97 checks passed

gt2345 deleted the gt/437-continue-trial branch July 9, 2024 19:21

gt2345 mentioned this pull request Jul 25, 2024

docs: Add release docs for continue trial, edit hp search, resource a… #9729

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: continue trial from WebUI for multi-trial experiment #9589

feat: continue trial from WebUI for multi-trial experiment #9589

gt2345 commented Jun 27, 2024 •

edited

Loading

netlify bot commented Jun 27, 2024 •

edited

Loading

codecov bot commented Jun 27, 2024 •

edited

Loading

keita-determined left a comment

keita-determined Jun 27, 2024

gt2345 Jun 28, 2024

keita-determined Jun 27, 2024

hkang1 Jun 28, 2024

keita-determined Jun 27, 2024

keita-determined Jun 27, 2024

hkang1 Jun 28, 2024

gt2345 Jun 28, 2024

hkang1 left a comment

hkang1 Jun 28, 2024

hkang1 Jun 28, 2024

hkang1 Jun 28, 2024

gt2345 commented Jun 28, 2024 •

edited

Loading

keita-determined left a comment

feat: continue trial from WebUI for multi-trial experiment #9589

feat: continue trial from WebUI for multi-trial experiment #9589

Conversation

gt2345 commented Jun 27, 2024 • edited Loading

Ticket

Description

Test Plan

Checklist

netlify bot commented Jun 27, 2024 • edited Loading

✅ Deploy Preview for determined-ui ready!

codecov bot commented Jun 27, 2024 • edited Loading

Codecov Report

keita-determined left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hkang1 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gt2345 commented Jun 28, 2024 • edited Loading

keita-determined left a comment

Choose a reason for hiding this comment

gt2345 commented Jun 27, 2024 •

edited

Loading

netlify bot commented Jun 27, 2024 •

edited

Loading

codecov bot commented Jun 27, 2024 •

edited

Loading

gt2345 commented Jun 28, 2024 •

edited

Loading