-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
repro: Disable run-cache if --no-commit. #8718
repro: Disable run-cache if --no-commit. #8718
Conversation
ac9f77b
to
80a3cc7
Compare
@dberenbaum In the issue there was also a point about |
Codecov ReportBase: 93.54% // Head: 93.55% // Increases project coverage by
Additional details and impacted files@@ Coverage Diff @@
## main #8718 +/- ##
=======================================
Coverage 93.54% 93.55%
=======================================
Files 457 457
Lines 36236 36237 +1
Branches 5259 5260 +1
=======================================
+ Hits 33896 33900 +4
+ Misses 1835 1832 -3
Partials 505 505
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
dd52da5
to
b8f964d
Compare
Thanks @daavoo! It's been a very long time since I looked into this issue, and I'm no longer sure about it, and I'm hesitant to put it in a minor version release. Let's say I'm using experiments, and my training stage outputs both a model that's cached in dvc and metrics that aren't. With the recent name changes, we made it possible to have duplicate experiments, but it didn't seem so bad because duplicate stages would still be cached and skipped. With this PR, I will end up duplicating all the computation of this training stage. It also makes dvc much more reliant on dvc.lock. Before this PR, It's possible to have a workflow where you treat dvc.lock as ephemeral and don't commit or share it. As long as you share the run-cache, it's still easy to recover everything. After this PR, there's no way to recover the outputs from a stage with Should dvc be saving the git hash for objects that have What do you think? |
I think there are a lot of things to discuss there 😅 WDYT about the first commit/part of the P.R. (repro: Disable run-cache if --no-commit)? I think we can start by discussing/merging that and continue discussing |
b8f964d
to
eee8627
Compare
run-cache
usage for repro --no-commit
and cache: false
outputs.run-cache
usage for repro --no-commit
run-cache
usage for repro --no-commit
eee8627
to
47130ad
Compare
47130ad
to
a03270a
Compare
a03270a
to
656aaea
Compare
As described in #4428 (comment). Add `run_cache` argument to `Stage.save`. Set it to `False` if `no_commit`.
656aaea
to
a1729ac
Compare
No description provided.