Live update plots for non-checkpoint experiments which log with dvclive #1277
Conversation
…ere are no checkpoints and a plot has been updated
… update-non-checkpoint-plots
… update-non-checkpoint-plots
extension/src/cli/runner.ts
Outdated
| import { reportError } from '../vscode/reporting' | ||
|
|
||
| export const autoRegisteredCommands = { | ||
| EXPERIMENT_IS_RUNNING: 'isRunning', |
There was a problem hiding this comment.
[F] We need this because we don't want to call for updates when there are changes coming through the watchers, a checkpoints experiment is running and there are no missing revisions (all revisions are immutable).
There was a problem hiding this comment.
Maybe we could rename it something like EXPERIMENT_CHECK_IS_RUNNING: 'checkIsRunning'. It is longer, but it doesn't look out of place in the commands list.
| (await this.internalCommands.executeCommand<boolean>( | ||
| AvailableCommands.EXPERIMENT_IS_RUNNING | ||
| )) && | ||
| !definedAndNonEmpty(revisions) |
There was a problem hiding this comment.
[I] For follow up need to find a way to distinguish between checkpoint and non-checkpoint repositories to get rid of the workspace timing issue described in the description of this PR.
extension/src/cli/runner.ts
Outdated
| import { reportError } from '../vscode/reporting' | ||
|
|
||
| export const autoRegisteredCommands = { | ||
| EXPERIMENT_IS_RUNNING: 'isRunning', |
There was a problem hiding this comment.
Maybe we could rename it something like EXPERIMENT_CHECK_IS_RUNNING: 'checkIsRunning'. It is longer, but it doesn't look out of place in the commands list.
extension/src/plots/model/index.ts
Outdated
| return uniqueValues([ | ||
| ...this.getSelectedRevisions().filter( | ||
| rev => | ||
| !uniqueValues([ | ||
| ...Object.keys(this.comparisonData), | ||
| ...Object.keys(this.revisionData), | ||
| 'workspace' | ||
| ]).includes(rev) | ||
| ) | ||
| ]) |
There was a problem hiding this comment.
Is there a particular reason the array passed to uniqueValues needs to be duplicated? I don't think the Set constructor mutates its passed in array, and filter also makes a shallow copy.
Probably not a big deal though.
There was a problem hiding this comment.
Mostly because of this: treeverse/dvc#7265.
This logic does need cleaned up. I will follow up.
mattseddon
force-pushed
the
update-non-checkpoint-plots
branch
from
e5940b0 to
d37bf40
Compare
|
Code Climate has analyzed commit d37bf40 and detected 0 issues on this pull request. The test coverage on the diff in this pull request is 100.0% (85% is the threshold). This pull request will bring the total coverage in the repository to 96.4% (0.1% change). View more on Code Climate. |
3 participants
This PR provides live updates to static plots for repositories that are using dvclive without checkpoints and running experiments within the workspace (
--tempscenario not covered).Addresses part of #1256
Demo (non-checkpoints)
Screen.Recording.2022-02-02.at.4.06.33.pm.mov
The above highlights the difference in speed between
exp showandplots diff, we can see that the plots are updated well before the experiments data.Demo (checkpoints)
Screen.Recording.2022-02-02.at.4.18.35.pm.mov
For follow up:
dvc.yamland see if the repo has checkpoints, if it does not use the watcher whilst experiments are running).