YAML output formats whole number floats indistinguishable from integers [formerly: JSON to TOML changes an integer to a float]

[rafi]

JSON to TOML changes an integer to a float, this is not expected.

Example:

$ echo '{"foo.bar":"baz","qux":1}' | yj -jt
"foo.bar" = "baz"
qux = 1.0

[sclevine]

Thanks for pointing this out @rafi.

JSON does not distinguish between integer and floating point types:
https://en.m.wikipedia.org/wiki/JSON#Data_types,_syntax_and_example

I suspect that Go’s encoding/json package is faithful to that specification — hence the conversion. I’ll dig into this a bit further, but I believe the example you present is expected behavior.

[rafi]

On the flip side, providing a float produces inconsistent results:

$ echo 'qux: 1.0' | yj -yy
qux: 1
$ echo 'qux: 1.0' | yj -yt
qux = 1.0
$ echo 'qux: 1.0' | yj -yj
{"qux":1}
$ echo 'qux: 1.0' | yj -yc
"qux" = 1

[sclevine]

YAML and TOML both define distinct integer and float types, while JSON and HCL do not.

Therefore, these seem correct:

$ echo 'qux: 1.0' | yj -yt
qux = 1.0
$ echo 'qux: 1.0' | yj -yj
{"qux":1}
$ echo 'qux: 1.0' | yj -yc
"qux" = 1

But the reflective conversion should probably preserve the float, so this should be improved:

$ echo 'qux: 1.0' | yj -yy
qux: 1

I suspect there are other conversions with YAML and TOML that don't appropriately preserve floats. I'll take a closer look tonight.

[sclevine]

After more testing, this appears to be an upstream issue with the go-yaml YAML parser.

This snippet outputs:

f: 1

I'll open an issue upstream after I double-check the YAML spec to make sure 1 is not a valid representation of a float in YAML.

[sclevine]

go-yaml/yaml#430

[rafi]

Thanks @sclevine for the awesome response :)

[sclevine]

Happy to help :)

Given the response upstream, I'd like to:

1. Rename this issue to match the particular inconsistency you found in YAML output.
2. Leave it open until go-yaml v3 is released and adopted by yj.

Sound good?

[rafi]

Sounds great, thanks.

[sclevine]

Everything mentioned above should be addressed in v5.0.0, which uses gopkg.in/yaml.v3. Additionally, map order is now preserved across all formats.

[rafi]

Great stuff @sclevine!