cmd ref: metrics diff precisions and examples

[jorgeorpinel]

mention that JSON is flattened, if we support arrays, how do we deal with strange values, etc?

• Currently we just link to dvc metric show for these details but it may not be enough.

• Add more examples? UPDATE: E.g. cmd ref: metrics diff precisions and examples #959 (comment) below

Other questions pending from #933:

how do actually metrics diff work for different file types? Especially JSON, but even for CSV – if some fields are missing in one of the references, will it freak out or show something?

• Need to ask engineering and/or try all the different options.

how does it (metrics show) actually work with figuring out the type by the extension. If raw is specified on the command line or in the DVC-file, will it still try to do extension analysis? Can I omit the type in the DVC-file?

• Play with it, figure out ^

[jorgeorpinel]

p.s. @efiop would be great to have some ideas on the questions above. Thanks

[efiop]

@jorgeorpinel Sorry for the delay. Here is an example:

#!/bin/bash

set -e
set -x

rm -rf myrepo
mkdir myrepo
cd myrepo

git init
dvc init
git add .
git commit -m "init"

dvc run -M metrics 'echo metrics_old > metrics'
dvc run -M metrics.json 'echo \{"a": \[1, 2, 3\]\} > metrics.json'
dvc run -M metrics.csv 'echo "a, 1, 2, 3" > metrics.csv'


git add .
git commit -m "old"

dvc run -M metrics 'echo metrics_new > metrics'
dvc run -M metrics.json 'echo \{"a": \[5, 2, 3\]\} > metrics.json'
dvc run -M metrics.csv 'echo "a, 5, 2, 3" > metrics.csv'

dvc metrics diff

will show:

    Path       Metric         Value                Change
metrics                 metrics_new          diff not supported
metrics.csv             a    5    2    3     diff not supported
metrics.json            {a: [5, 2, 3]}       diff not supported

So any wierd stuff will be considered non-diffable, but it will still try to print the new value.

Though I've found an issue with multiline raw metrics, will send a patch/file an issue soon.

[jorgeorpinel]

Thanks Ruslan!