Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

exp queue: remove queued experiments #7591

Closed
2 tasks done
Tracked by #7592
pmrowla opened this issue Apr 19, 2022 · 7 comments · Fixed by #7721
Closed
2 tasks done
Tracked by #7592

exp queue: remove queued experiments #7591

pmrowla opened this issue Apr 19, 2022 · 7 comments · Fixed by #7721
Assignees
@karajan1001
Labels
A: experiments Related to dvc exp ui user interface / interaction

Comments

@pmrowla
Copy link
Contributor

pmrowla commented Apr 19, 2022 •
edited
Loading

  • dvc queue remove: remove inactive experiment(s) from the queue
  • dvc queue kill: kill active experiment(s) (and treat them as cancelled/failed)
@pmrowla pmrowla added ui user interface / interaction A: experiments Related to dvc exp labels Apr 19, 2022
This was referenced Apr 19, 2022
Closed
Closed
@karajan1001 karajan1001 self-assigned this May 10, 2022
@karajan1001 karajan1001 mentioned this issue May 10, 2022
Merged
2 tasks
@karajan1001 karajan1001 added this to DVC May 10, 2022
@karajan1001 karajan1001 moved this to Backlog in DVC May 10, 2022
@karajan1001 karajan1001 moved this from Backlog to Review In Progress in DVC May 10, 2022
karajan1001 added a commit to karajan1001/dvc that referenced this issue May 16, 2022
@karajan1001 @pmrowla
exp queue: remove queued experiments(iterative#7591)
b811da0 
fix: iterative#7591
1. Add a new command `dvc queue`
2. Add two sub-command `dvc queue remove` and `dvc queue kill`
3. Add a unit test to test them

Co-authored-by: Peter Rowlands (변기호) <peter@pmrowla.com>
@karajan1001 karajan1001 linked a pull request May 17, 2022 that will close this issue
exp queue: Add two command queue remove and queue kill #7721
Merged
2 tasks
pmrowla added a commit that referenced this issue May 18, 2022
@karajan1001 @pmrowla
exp queue: remove queued experiments(#7591)
fe08e27 
fix: #7591
1. Add a new command `dvc queue`
2. Add two sub-command `dvc queue remove` and `dvc queue kill`
3. Add a unit test to test them

Co-authored-by: Peter Rowlands (변기호) <peter@pmrowla.com>
@dberenbaum
Copy link
Collaborator

dvc queue remove doesn't work with short names:

$ dvc exp run --queue -S train.min_split=0.005
Queued experiment 'b44d265' for future execution.
$ dvc queue remove b44d265
Removed experiments in queue:
$ dvc exp show --no-pager
 ────────────────────────────────────────────────────────────────────────────────────
  Experiment           Created    State    avg_prec   …   prepare…   …   data/d…   …
 ────────────────────────────────────────────────────────────────────────────────────
  workspace            -          -          0.9202   …   0.2        …   22a1a29   …
  bigrams-experiment   May 06,    -          0.9202   …   0.2        …   22a1a29   …
                       2022
  └── b44d265          02:02 PM   Queued          -   -   0.2        …   22a1a29   …
 ────────────────────────────────────────────────────────────────────────────────────

@dberenbaum
Copy link
Collaborator

Also, if no experiments are provided or any of the provided experiments don't match, we should provide a relevant error message like dvc exp remove:

$ dvc exp remove
ERROR: Either provide an `experiment` argument, or use the `--rev` or `--all-commits` flag.
$ dvc exp remove sdfasd
ERROR: 'sdfasd' are not valid experiment names
$ dvc exp remove b44d265 sfasdfas
ERROR: 'sfasdfas' are not valid experiment names

pmrowla added a commit that referenced this issue Jun 2, 2022
@karajan1001 @pmrowla
exp queue: remove queued experiments(#7591)
a19c3a7 
fix: #7591
1. Add a new command `dvc queue`
2. Add two sub-command `dvc queue remove` and `dvc queue kill`
3. Add a unit test to test them

Co-authored-by: Peter Rowlands (변기호) <peter@pmrowla.com>
@dberenbaum
Copy link
Collaborator

kill combined with stop does not remove the killed experiment from the queue:

$ dvc queue status
Rev                                       Name    Created    Status
646de235a2d6b57b0d28cb81fb93de1c4bee6af0          02:07 PM   Queued
c8b0d8f832d12b1786582ab43ce1c50c3ab2072f          02:07 PM   Queued
20ddab20dbd0ebb467ae857f958d5a8945669426          02:07 PM   Queued
e145857a825cc82f0057cba1443e1108f29fb8b9          02:07 PM   Queued
9443d464c3fd1d247f034544494a3a3c389fe26c          02:07 PM   Queued
$ dvc queue start
Start 1 queue worker to process the queue tasks
$ dvc queue status
Rev                                       Name    Created    Status
646de235a2d6b57b0d28cb81fb93de1c4bee6af0          02:07 PM   Running
c8b0d8f832d12b1786582ab43ce1c50c3ab2072f          02:07 PM   Queued
20ddab20dbd0ebb467ae857f958d5a8945669426          02:07 PM   Queued
e145857a825cc82f0057cba1443e1108f29fb8b9          02:07 PM   Queued
9443d464c3fd1d247f034544494a3a3c389fe26c          02:07 PM   Queued
$ dvc queue stop --kill
All of the Queue tasked had already been killed, Queue workers are stopping.
$ dvc queue status
Rev                                       Name    Created    Status
c8b0d8f832d12b1786582ab43ce1c50c3ab2072f          02:07 PM   Queued
20ddab20dbd0ebb467ae857f958d5a8945669426          02:07 PM   Queued
e145857a825cc82f0057cba1443e1108f29fb8b9          02:07 PM   Queued
9443d464c3fd1d247f034544494a3a3c389fe26c          02:07 PM   Queued
646de235a2d6b57b0d28cb81fb93de1c4bee6af0          02:07 PM   Queued

Similar result if stop is called before kill:

$ dvc queue status
Rev                                       Name    Created    Status
fdc0af95cd02f8de703f199c36e4e600ad42a797          02:09 PM   Queued
1499bb0468036a314702194351bf6c5c50cc4997          02:09 PM   Queued
dc00833188dddb1697993a18939cb1a9928d907d          02:09 PM   Queued
96c42228c8f55e2ddacb0700c4555658a2f3227a          02:09 PM   Queued
d0eaaa93be5660c3fa61af1b4cd409d7e3441b3d          02:09 PM   Queued
$ dvc queue start
Start 1 queue worker to process the queue tasks
$ dvc queue status
Rev                                       Name    Created    Status
fdc0af95cd02f8de703f199c36e4e600ad42a797          02:09 PM   Running
1499bb0468036a314702194351bf6c5c50cc4997          02:09 PM   Queued
dc00833188dddb1697993a18939cb1a9928d907d          02:09 PM   Queued
96c42228c8f55e2ddacb0700c4555658a2f3227a          02:09 PM   Queued
d0eaaa93be5660c3fa61af1b4cd409d7e3441b3d          02:09 PM   Queued
$ dvc queue stop
Queue workers will be stopped after current tasks finished.
$ dvc queue status
Rev                                       Name    Created    Status
fdc0af95cd02f8de703f199c36e4e600ad42a797          02:09 PM   Running
1499bb0468036a314702194351bf6c5c50cc4997          02:09 PM   Queued
dc00833188dddb1697993a18939cb1a9928d907d          02:09 PM   Queued
96c42228c8f55e2ddacb0700c4555658a2f3227a          02:09 PM   Queued
d0eaaa93be5660c3fa61af1b4cd409d7e3441b3d          02:09 PM   Queued
$ dvc queue kill fdc0af
$ dvc queue status
Rev                                       Name    Created    Status
fdc0af95cd02f8de703f199c36e4e600ad42a797          02:09 PM   Queued
1499bb0468036a314702194351bf6c5c50cc4997          02:09 PM   Queued
dc00833188dddb1697993a18939cb1a9928d907d          02:09 PM   Queued
96c42228c8f55e2ddacb0700c4555658a2f3227a          02:09 PM   Queued
d0eaaa93be5660c3fa61af1b4cd409d7e3441b3d          02:09 PM   Queued

@dberenbaum
Copy link
Collaborator

dvc queue kill does not accept assigned experiment names for checkpoint experiments:

$ dvc exp show
 ──────────────────────────────────────────────────────────────────────────────────────>
  Experiment                Created        State     Executor    step      loss   accur>
 ──────────────────────────────────────────────────────────────────────────────────────>
  workspace                 -              Running   workspace      -         -        >
  dvc                       May 27, 2022   -         -              -         -        >
  │ ╓ 98fa405 [exp-405e8]   05:46 PM       Running   temp           1   0.73572     0.7>
  ├─╨ c6773d6               05:45 PM       -         -              0    1.0734    0.74>
  ├── 26f2f1d [exp-0c43f]   05:42 PM       -         -              4   0.67752    0.81>
  ├── b1bd294 [exp-cafc7]   05:40 PM       -         -              4   0.58267    0.83>
  ├── 740d11e [exp-a45e9]   02:04 PM       -         -              0    1.2026    0.73>
  └── 74087e0               05:45 PM       Queued    -              -         -        >
 ──────────────────────────────────────────────────────────────────────────────────────>
$ dvc queue kill exp-405e8
ERROR: 'exp-405e8' are not valid experiment names

@dberenbaum dberenbaum mentioned this issue Jun 8, 2022
Merged
2 tasks
@karajan1001
Copy link
Contributor

dvc queue kill does not accept assigned experiment names for checkpoint experiments:

$ dvc exp show
 ──────────────────────────────────────────────────────────────────────────────────────>
  Experiment                Created        State     Executor    step      loss   accur>
 ──────────────────────────────────────────────────────────────────────────────────────>
  workspace                 -              Running   workspace      -         -        >
  dvc                       May 27, 2022   -         -              -         -        >
  │ ╓ 98fa405 [exp-405e8]   05:46 PM       Running   temp           1   0.73572     0.7>
  ├─╨ c6773d6               05:45 PM       -         -              0    1.0734    0.74>
  ├── 26f2f1d [exp-0c43f]   05:42 PM       -         -              4   0.67752    0.81>
  ├── b1bd294 [exp-cafc7]   05:40 PM       -         -              4   0.58267    0.83>
  ├── 740d11e [exp-a45e9]   02:04 PM       -         -              0    1.2026    0.73>
  └── 74087e0               05:45 PM       Queued    -              -         -        >
 ──────────────────────────────────────────────────────────────────────────────────────>
$ dvc queue kill exp-405e8
ERROR: 'exp-405e8' are not valid experiment names

Could you please provide the result of dvc queue status. On my computer, I can properly kill checkpoint experiments with the name shown in queue status

@dberenbaum
Copy link
Collaborator

I see no name for the running checkpoint experiment in queue status:

$ dvc exp show
 ──────────────────────────────────────────────────────────────────────────────────────>
  Experiment                Created        State     Executor   step      loss   accura>
 ──────────────────────────────────────────────────────────────────────────────────────>
  workspace                 -              -         -             -         -         >
  dvc                       May 27, 2022   -         -             -         -         >
  │ ╓ b31dd1a [exp-2f4c5]   02:05 PM       Running   temp          3   0.73842    0.799>
  │ ╟ 522486b               02:05 PM       -         -             2   0.85006    0.769>
  │ ╟ 8ea872c               02:05 PM       -         -             1   0.96466    0.748>
  ├─╨ 26061df               02:05 PM       -         -             0    1.3088    0.726>
  └── eccf64f               Jun 08, 2022   Queued    -             -         -         >
 ──────────────────────────────────────────────────────────────────────────────────────>
$ dvc queue status
Rev                                       Name       Created       Status
ab3fe5ee4fdb660b9dfaf6fddc5ccdc35c993815             02:03 PM      Running
eccf64ff7bd6da5c25031be1cda61d23cc9a7abe             Jun 08, 2022  Queued
9416b60726c3c3c11ab3b72aef6678ced8185758  exp-e06cf  Jun 08, 2022  Success
268782fa932033919f31494b82b1d4937991817c  exp-72ce4  Jun 08, 2022  Success
f4217e5534e397de055bc0bc9adc25d6c85399e8  exp-5d9f2  Jun 08, 2022  Success
ef64877d6a019da5e081e44b5cc9c1ae6efbefaa  exp-0382c  Jun 08, 2022  Success
eccf64ff7bd6da5c25031be1cda61d23cc9a7abe  exp-29b06  Jun 08, 2022  Success
$ dvc queue kill exp-2f4c5
ERROR: 'exp-2f4c5' are not valid queued experiment names

@dberenbaum dberenbaum mentioned this issue Jun 13, 2022
Closed
dberenbaum pushed a commit that referenced this issue Jun 13, 2022
@karajan1001 @pmrowla
exp queue: remove queued experiments(#7591)
1f45c39 
fix: #7591
1. Add a new command `dvc queue`
2. Add two sub-command `dvc queue remove` and `dvc queue kill`
3. Add a unit test to test them

Co-authored-by: Peter Rowlands (변기호) <peter@pmrowla.com>
pmrowla added a commit that referenced this issue Jun 14, 2022
@karajan1001 @pmrowla
exp queue: remove queued experiments(#7591)
6fdea69 
fix: #7591
1. Add a new command `dvc queue`
2. Add two sub-command `dvc queue remove` and `dvc queue kill`
3. Add a unit test to test them

Co-authored-by: Peter Rowlands (변기호) <peter@pmrowla.com>
pmrowla added a commit that referenced this issue Jul 5, 2022
@karajan1001 @pmrowla
exp queue: remove queued experiments(#7591)
dab95fa 
fix: #7591
1. Add a new command `dvc queue`
2. Add two sub-command `dvc queue remove` and `dvc queue kill`
3. Add a unit test to test them

Co-authored-by: Peter Rowlands (변기호) <peter@pmrowla.com>
@karajan1001
Copy link
Contributor

Closed as all of the features had been implemented

Repository owner moved this from Review In Progress to Done in DVC Jul 5, 2022
pmrowla added a commit that referenced this issue Jul 6, 2022
@karajan1001 @pmrowla
exp queue: remove queued experiments(#7591)
bb38c47 
fix: #7591
1. Add a new command `dvc queue`
2. Add two sub-command `dvc queue remove` and `dvc queue kill`
3. Add a unit test to test them

Co-authored-by: Peter Rowlands (변기호) <peter@pmrowla.com>
pmrowla added a commit that referenced this issue Jul 11, 2022
@karajan1001 @pmrowla
exp queue: remove queued experiments(#7591)
46b479c 
fix: #7591
1. Add a new command `dvc queue`
2. Add two sub-command `dvc queue remove` and `dvc queue kill`
3. Add a unit test to test them

Co-authored-by: Peter Rowlands (변기호) <peter@pmrowla.com>
pmrowla added a commit that referenced this issue Jul 12, 2022
@karajan1001 @pmrowla
exp queue: remove queued experiments(#7591)
a6cb4eb 
fix: #7591
1. Add a new command `dvc queue`
2. Add two sub-command `dvc queue remove` and `dvc queue kill`
3. Add a unit test to test them

Co-authored-by: Peter Rowlands (변기호) <peter@pmrowla.com>
@dberenbaum dberenbaum mentioned this issue Jul 14, 2022
Closed
@dberenbaum dberenbaum mentioned this issue Jul 27, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@karajan1001 karajan1001

Labels
A: experiments Related to dvc exp ui user interface / interaction
Projects
No open projects
DVC
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

exp queue: Add two command queue remove and queue kill karajan1001/dvc
3 participants
@pmrowla @dberenbaum @karajan1001