Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dvc queue logs -f could take more then one job_id #8658

Open
Tracked by #9442
behrica opened this issue Dec 5, 2022 · 6 comments
Open
Tracked by #9442

dvc queue logs -f could take more then one job_id #8658

behrica opened this issue Dec 5, 2022 · 6 comments
Labels
A: task-queue Related to task queue. feature request Requesting a new feature p2-medium Medium priority, should be done, but less important

Comments

@behrica
Copy link

behrica commented Dec 5, 2022

Similar to "dvc exp run --run-all", which follows the log of all running jobs.

so maybe as simple as:

dvc queue logs -f 107f918,a6ca492

or maybe as options to follow "all running"

dvc queue logs -f all
@karajan1001 karajan1001 added A: task-queue Related to task queue. feature request Requesting a new feature labels Dec 7, 2022
@dberenbaum dberenbaum added the p3-nice-to-have It should be done this or next sprint label Dec 20, 2022
@dberenbaum dberenbaum added p2-medium Medium priority, should be done, but less important and removed p3-nice-to-have It should be done this or next sprint labels Dec 30, 2022
@dberenbaum
Copy link
Collaborator

It could be as simple as dvc queue logs without any args follows all tasks.

@dberenbaum
Copy link
Collaborator

This would also be useful as a replacement for dvc exp run --run-all. See #8724.

@ReveStobinson
Copy link

I agree with this, especially given that dvc exp run --run-all will be deprecated in a future release! The documentation says:

dvc exp run --run-all [--jobs] is now a shortcut for dvc queue start [--jobs] followed by dvc queue logs -f.

However, there isn't actually a set of suitable DVC commands that could replace --run-all without the addition of at least the option suggested by @behrica, since dvc queue logs -f requires exactly one task to be specified.

In that spirit, I would go even further and suggest that an option should also be added to dvc queue start (perhaps --logs, --follow, or --attach), which follows the logs of all of the experiments in the queue (for which I imagine the easiest implementation would be creating a --follow-all option for queue logs and making it a shortcut).

@dberenbaum
Copy link
Collaborator

Let's make dvc queue logs show logs for all experiments by default, and then make dvc queue start --follow equivalent to dvc queue start && dvc queue logs --follow.

@dberenbaum dberenbaum mentioned this issue May 30, 2023
4 tasks
@BradyJ27
Copy link
Contributor

Afaik dvc queue logs does NOT show all experiments by default still, and this issue is still valid. Is that correct @dberenbaum ?

@dberenbaum
Copy link
Collaborator

Yes, that's correct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A: task-queue Related to task queue. feature request Requesting a new feature p2-medium Medium priority, should be done, but less important
Projects
None yet
Development

No branches or pull requests

5 participants