Skip to content

Commit

Permalink
docs: update docs for YAML 1.2 compatibility mitigation
Browse files Browse the repository at this point in the history
See iterative/dvc#5971

adds note to parameter values section metioning scientific notation for SEO
use ruamel.yaml in example instead of PyYAML and add note
  • Loading branch information
bobertlo committed May 21, 2022
1 parent 26fffa1 commit 8c503a3
Showing 1 changed file with 10 additions and 2 deletions.
12 changes: 10 additions & 2 deletions content/docs/command-reference/params/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,10 @@ as the tree path to find those values. Supported types are: string, integer,
float, and arrays (groups of params). Note that DVC does not ascribe any
specific meaning to these values.

> YAML 1.2 stores very large and very small numbers in scientific notation. Some
> popular libraries still use older versions of the format, and can introduce
> subtle bugs when used.

DVC saves parameter names and values to `dvc.lock` in order to track them over
time. They will be compared to the latest params files to determine if the stage
is outdated upon `dvc repro` (or `dvc status`).
Expand Down Expand Up @@ -115,16 +119,20 @@ The `train.py` script will have some code to parse and load the needed
parameters. For example:

```py
import yaml
from ruamel.yaml import YAML
with open("params.yaml", 'r') as fd:
params = yaml.safe_load(fd)
yaml = YAML()
params = yaml.load(fd)
lr = params['lr']
epochs = params['train']['epochs']
layers = params['train']['layers']
```

> Note that some popular libraries (like PyYAML) do not support YAML 1.2 and
> should be avoided whenever possible.

You can find that each parameter was defined in `dvc.yaml`, as well as saved to
`dvc.lock` along with the values. These are compared to the params files when
`dvc repro` is used, to determine if the parameter dependency has changed.
Expand Down

0 comments on commit 8c503a3

Please sign in to comment.