commit, status: cannot address all foreach stage targets

[davesteps]

I have a pipeline that uses a foreach loop to process different datasets lets call the stage data_prep. I can run dvc repro -s data_prep and it will loop through each dataset specified in the dvc.yaml no problem. If I then modify the code that the stage depends eg data_prep.py all the individual stages will need to be rerun IE dvc status will return something like:

data_prep@data_set_1:
        changed deps:
                modified:           ./data_prep.py
data_prep@data_set_2:
        changed deps:
                modified:           ./data_prep.py
data_prep@data_set_3:
        changed deps:
                modified:           ./data_prep.py
data_prep@data_set_4:
        changed deps:
                modified:           ./data_prep.py
...

The problem is that lets say all I have done is a minor change to the code and I don't want to do a dvc repro all I want to do is dvc commit data_prep but I cannot, I have to commit each substage separtely eg dvc commit data_prep@data_set_4 which is not ideal as I have many such datasets. Can it be implemented that doing a dvc commit data_prep will commit all sub-stages?

Many thanks, DVC is awesome.

Edit: I also do not want to just to a dvc commit to commit everything as there are later stages I don't want to commit

[dberenbaum]

Good point! This appears to be supported for dvc repro data_prep but not for dvc commit data_prep.

@skshetry Any thoughts on this?

[itcarroll]

I'm also encountering this, and would add that dvc status <target> fails in the same way as dvc commit <target> when <target> is a foreach stage.

@dtrifiro Thanks for your help. I found this already existing issue.

[dtrifiro]

Thanks @itcarroll! I'd expect the all commands dealing with parametrized stages to work in a consistent manner (i.e. like repro does). It wouldn't be bad to have this fixed

[dberenbaum]

Support for foreach targets should be extended to:

• commit
• status
• push
• pull
• fetch
• checkout
• remove
• dag
• stage list
• repro
• exp run

[itcarroll]

@dberenbaum And remove?

[dberenbaum]

@skshetry There was interest in contributing a fix in #7462 (comment). What do you think?