dvc exp show with target

[lefos99]

Hello,

In our project, we have different dvc.yaml files, in different folders.

I think it would be great (and much cleaner) if I could provide a target when I run the command dvc exp show.

For example, I can already do dvc status dvc.yaml and get the status for only this specific dvc.yaml, and not for the other ones. Would something similar for dvc exp show be possible?

Thank you in advance 😃

[robSanders818]

I agree with this as well, this would be useful for my use case, as we have distinct stages in different dvc.yaml files, and it would be great to see a tabular summary of results for each distinct dvc.yaml file

[daavoo]

A similar request from a user on discord:

Hi, I'm currently exploring the experiments feature.
I have two different but complementary models that need to be train independently but make use of the same dataset.
After trying the experiments option I realize that I cannot specify an experiment stage, aka I cannot view experiments for each model independently

https://discord.com/channels/485586884165107732/563406153334128681/902556996367171604

[dberenbaum]

I'm curious how much this overlaps with #5966. In practice, I would expect --only-changed to be useful for a lot of the same scenarios where a target would be useful. If anyone is looking for this feature, it would be great to hear whether --only-changed is useful or whether there is a separate need to specify a target.

[lefos99]

To be honest, --only-changed doesn't seem to help me in this case, because we use multiple dvc.yaml files in different folders under the same git project.
If I get it right --only-changed functionality will inspect all the dvc.yaml files in my project, whereas I just want to focus on a specific one in my project and ignore the rest dvc.yaml files by being able to define a target.

[dberenbaum]

@lefos99 If you are experimenting within a single dvc.yaml file, are the other dvc.yaml files likely to change between experiments?

[lefos99]

@dberenbaum Do you mean if the dvc.yaml files are going to change or the dvc.lock files?

But to answer to you question, all dvc.yaml files are also subject to change at any time. Our structure is the following:

git_project
 ->base_code
 ->project_1
  -->code
  -->dvc.yaml
  -->dvc.lock
 ->project_2
  -->code
  -->dvc.yaml
  -->dvc.lock
 ->project_3
  -->code
  -->dvc.yaml
  -->dvc.lock
 ->project_4
  -->code
  -->dvc.yaml
  -->dvc.lock

[daavoo]

@lefos99 Do the projects share some data or are they isolated?

If they are isolated, you could set up each one with dvc init --subir (https://dvc.org/doc/command-reference/init#examples-initializing-dvc-in-a-subdirectory). After that, If you run dvc exp show inside a subdir you will only see the metrics and params associated with that dvc.yaml

[dberenbaum]

@dberenbaum Do you mean if the dvc.yaml files are going to change or the dvc.lock files?

@lefos99 Right, sorry, I meant the dvc.lock files. How likely is it that you are running experiments in multiple project folders at the same time?

Not to discount this feature request, just looking for workarounds that could help.

[lefos99]

@dberenbaum Do you mean if the dvc.yaml files are going to change or the dvc.lock files?

@lefos99 Right, sorry, I meant the dvc.lock files. How likely is it that you are running experiments in multiple project folders at the same time?

Not to discount this feature request, just looking for workarounds that could help.

Simultaneously we don't (and we can't as with DVC and .dvc/tmp/rwlock is not possible).
But in general, the commit messages can be intermingled with different DVC experiments from different projects.

[lefos99]

@lefos99 Do the projects share some data or are they isolated?

If they are isolated, you could set up each one with dvc init --subir (https://dvc.org/doc/command-reference/init#examples-initializing-dvc-in-a-subdirectory). After that, If you run dvc exp show inside a subdir you will only see the metrics and params associated with that dvc.yaml

Oh, that's interesting, I didn't know that. Thanks @daavoo
That would be a nice solution but I wonder if it's too late for my project now and whether it's possible to split it into different sub-projects, as it has been developed for quite some time now.