Skip to content

Commit

Permalink
guide: begin Managing Experiments guide
Browse files Browse the repository at this point in the history
per #2690 (review)
  • Loading branch information
jorgeorpinel committed Aug 25, 2021
1 parent ec57601 commit 47b6c44
Show file tree
Hide file tree
Showing 2 changed files with 81 additions and 0 deletions.
13 changes: 13 additions & 0 deletions content/docs/user-guide/experiment-management/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,18 @@
# Experiment Management

<!--
Machine Learning and Data Science projects usually involve experimentation.
These experiments' goals can range from finding good hyperparameters to testing
for data and concept drift. DVC 2.0 introduced a new set of commands to manage
experiments with minimum boilerplate. It allows to run experiments defined by
pipelines, track their associated data and model files, set parameters for each,
push experiment parameters and code to Git remotes without committing them,
create branches and persist them in Git.
Each experiment represents a project variation based on the changes in your
current <abbr>workspace</abbr>.
-->

_New in DVC 2.0_

Data science and ML are iterative processes that require a large number of
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# Managing Experiments

Once you have defined and/or [run experiments] in your project, you can use
several features of DVC to see, compare, reproduce, share, or remove them.

[run experiments]: /doc/user-guide/experiment-management/running-experiments

## Experiment names

Experiments created with `dvc exp run` will have an auto-generated name like
`exp-bfe64` by default. It can be customized using the `--name` (`-n`) option:

```dvc
$ dvc exp run --name cnn-512 --set-param model.conv_units=512
```

When you create an experiment, DVC generates a Git-like SHA-1 hash from its
contents. This is shown when you [queue experiments] with `--queue`:

[queue experiments]:
/doc/user-guide/experiment-management/running-experiments#the-experiments-queue

```dvc
$ dvc exp run --queue -S model.conv_units=32
Queued experiment '6518f17' for future execution.
```

After running queued experiment, DVC uses the regular name mentioned earlier.

> Note that you can set a queued experiment's name in advance:
>
> ```dvc
> $ dvc exp run --queue --name cnn-512 -S model.conv_units=512
> Queued experiment '86bd8f9' for future execution.
> ```
You can refer to experiments in `dvc exp apply` or `dvc exp branch` either with
regular experiment names or by their SHA hashes.
## Listing experiments
Use `dvc exp show` to see both run and queued experiments:
```dvc
$ dvc exp show --no-pager --no-timestamp \
--include-metrics loss --include-params model.conv_units
```
```dvctable
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃ neutral:**Experiment** ┃ metric:**loss** ┃ param:**model.conv_units** ┃
┑━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
β”‚ workspace β”‚ 0.23534 β”‚ 64 β”‚
β”‚ 3973b6b β”‚ - β”‚ 16 β”‚
β”‚ β”œβ”€β”€ aeaabb0 [exp-cb13f] β”‚ 0.23534 β”‚ 64 β”‚
β”‚ β”œβ”€β”€ d0ee7ce [exp-5dccf] β”‚ 0.23818 β”‚ 32 β”‚
β”‚ β”œβ”€β”€ 1533e4d [exp-88874] β”‚ 0.24039 β”‚ 128 β”‚
β”‚ β”œβ”€β”€ b1f41d3 [cnn-256] β”‚ 0.23296 β”‚ 256 β”‚
β”‚ β”œβ”€β”€ 07e927f [exp-6c06d] β”‚ 0.23279 β”‚ 24 β”‚
β”‚ β”œβ”€β”€ b2b8586 [exp-2a1d5] β”‚ 0.25036 β”‚ 16 β”‚
β”‚ └── *86bd8f9 β”‚ - β”‚ 512 β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
```

When an experiment is not run yet, only the former hash is shown (marked with
`*`).

<!-- WIP -->

0 comments on commit 47b6c44

Please sign in to comment.