encoding/json: encode precise floating point integers using %.0f, not %g

[joeshaw]

Go 1.1.2 on OS X 10.8.4, and on play.golang.org.

If you unmarshal a JSON numeric value into interface{}, it is encoded as a float64.  If
it is sufficiently large (>= 1e6, I believe) when you marshal it, it is encoded as a
string using exponent notation.  If the original value was an integer and you later try
to decode into an int64 value, an error is returned.

Example code is available here: http://play.golang.org/p/fYuB4tT7Ul

Using 'f' for the fmt byte to AppendFloat() instead of 'g' in
http://golang.org/src/pkg/encoding/json/encode.go#L334 would solve this problem, as
values which can be represented as integers will be encoded in a way that can be read
back in as integers.

(There are other issues as you get to very large values, since float64 can't encode a
whole int64... not sure what or if anything can be done about that.)

[joeshaw]

Comment 1:

Hmm, regarding the parenthetical I just noticed UseNumber() on the json.Decoder type, so
I can use that to decode the int64 cleanly, and when Number is used it just encodes the
string representation so you don't get the exponential notation.  Unfortunate that I
have to wrap my []byte blob in an io.Reader though.
Still, I think using 'f' in AppendFloat() would be good for the common case.

[rsc]

Comment 2:

Note that the int64 round trip cannot work for all int64 values, because the map to
float64 is not injective above 2^53 (try 9007199254740993). So you are better off
arranging to encode into int64 (or use UseNumber).
We could possibly print the floating point integers below 2^53 using %.0f instead of %g.
This won't happen for Go 1.2. It needs more thought.

Labels changed: added priority-later, go1.3maybe, removed priority-triage.

Status changed to Thinking.

[joeshaw]

Comment 3:

Yeah, the loss of precision became much more evident when I switched AppendFloat() to
'f'.
My thinking on this has changed since I first reported it.  Encoding integer values
below 2^53 as integers would probably help the common case but just push off the
surprise from 1e6 to a higher value.  Given json.Number, maybe the right way to go about
this is to always encode float64 with a decimal point so that JSON parsers never
interpret the values as integers.  It is slightly more inconvenient, but it would
involve the least surprise when your program happens to get to large integer values
(where "large" is either 1e6 or 2^53).
It would probably also be good to make this more explicit in the documentation.

[rsc]

Comment 4:

Labels changed: added release-none, removed go1.3maybe.

[rsc]

Comment 5:

Labels changed: added repo-main.

[parkr]

I'm using protobufs and in order to maintain compatibility with Node land, we can't use int64 in our messages. This breaks the deserialization of json ~> protobuf when a valid int32 is serialized into json, then deserialized into a protobuf.

[gopherbot]

CL https://golang.org/cl/30371 mentions this issue.

[integrii]

Jesus thank you @rsc - This has been a thorn for anyone encoding lat/lon with six points of precision (common for mobile app use). Though I would argue that we should go even further and use a decimal for even much larger numbers... perhaps even 1e-10.

Please someone merge this to a release branch!

[ianlancetaylor]

This is not the kind of change we put on a release branch. It will be in the Go 1.8 release.

[integrii]

In the mean time between now and release, is there a recommended way to force our json.Marshal output with six points of precision to come out as a decimal and not scientific notation?

[ianlancetaylor]

@integrii That is a great question for a forum rather than the issue tracker. See https://golang.org/wiki/Questions . Thanks.