Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve experiments naming #2457

Closed
shcheklein opened this issue Sep 24, 2022 · 6 comments · Fixed by #2956
Closed

Improve experiments naming #2457

shcheklein opened this issue Sep 24, 2022 · 6 comments · Fixed by #2956
Assignees
Labels
A: experiments Area: experiments table webview and everything related A: plots Area: plots webview, side panel and everything related A: trees Area: SCM and DVC-tracked trees priority-p1 Regular product backlog story Product feature aka epic. Discussion, progress, checkboxes for implementation, etc

Comments

@shcheklein
Copy link
Member

shcheklein commented Sep 24, 2022

One of the common feedbacks we are getting is that experiment names are not meaningful w/o metrics in the side panel and in the plots webview.

We need to introduce:

  • columns that we can pin (metrics, params, data) and get this information into experiment names, or replaces hash
  • names - let's try to generate auto names? similar to how codespace does this. Some short adj + short noun
  • tooltips with the full list of metrics, params, full hash, etc

Related discussion on the DVC side: iterative/dvc#7877

@shcheklein shcheklein added story Product feature aka epic. Discussion, progress, checkboxes for implementation, etc priority-p1 Regular product backlog A: experiments Area: experiments table webview and everything related A: trees Area: SCM and DVC-tracked trees A: plots Area: plots webview, side panel and everything related labels Sep 24, 2022
@shcheklein
Copy link
Member Author

Related #2395

@shcheklein
Copy link
Member Author

Related #1820 (comment)

@aschuh-hf
Copy link

Apart from improving the display of auto-generated names, would it make sense to allow the user to name an experiment themselves when creating it? E.g., using Modify & Queue. Right now, the UI goes through a sequence of questions to ask for the different parameters values one indicated wanting to modify for the experiment. Why not also ask (maybe as final step) what to name the experiment, or to keep the auto-generated default otherwise?

@daavoo
Copy link
Contributor

daavoo commented Nov 22, 2022

names - let's try to generate auto names? similar to how codespace does this. Some short adj + short noun

I added a random name generator in DVCLive, mostly out of necessity for the Python-only experiments scenario (it was simpler than changing how DVC currently generates the exp hashes):

iterative/dvclive#366

 ─────────────────────────────────────────────────────────────── 
  Experiment                 Created    foobar   foo.bar   step  
 ─────────────────────────────────────────────────────────────── 
  workspace                  -           4.348    4.0804      5  
  master                     11:40 AM   4.1825    4.7032      4  
  ├── bbddd64 [sworn-snib]   12:06 PM    4.348    4.0804      5  
  ├── 775fb5b [taunt-vina]   12:06 PM    4.297    4.5091      5  
  ├── d08f02e [kraal-jump]   12:06 PM   4.0358    4.0448      5  
  ├── b739189 [color-vega]   12:06 PM   4.9082    4.6159      5  
  └── 34541c3 [mauve-loss]   12:06 PM   4.3448    4.4993      5  
 ─────────────────────────────────────────────────────────────── 

@julieg18
Copy link
Contributor

With tooltips added to the plots ribbon blocks (#2924) and DVC updating experiment names, are we ready to close this issue or is there more to be done?

@shcheklein
Copy link
Member Author

@julieg18 yes, let's close this when the final part is merged!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A: experiments Area: experiments table webview and everything related A: plots Area: plots webview, side panel and everything related A: trees Area: SCM and DVC-tracked trees priority-p1 Regular product backlog story Product feature aka epic. Discussion, progress, checkboxes for implementation, etc
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants