strconv: FormatFloat rounding error

[cyberphone]

What version of Go are you using (go version)?

go version go1.11.4 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Anders\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\Users\Anders\go
set GOPROXY=
set GORACE=
set GOROOT=C:\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\Anders\AppData\Local\Temp\go-build2129905
62=/tmp/go-build -gno-record-gcc-switches

What did you do?

I'm trying to implement a JSON canonicalization scheme requiring numbers to be expressed in the shortest way. prec -1 seemed like a better alternative than porting V8's solution but unfortunately it seems that the -1 option is broken:

package main

import (
    "strconv"
    "math"
    "fmt"
)

func fpformat(hex string, format byte, precision int, expected string) {
    ieeeU64, err := strconv.ParseUint(hex, 16, 64)
    if err != nil {
        panic(hex)
    }
    ieeeF64 := math.Float64frombits(ieeeU64)
    shortestNumber := strconv.FormatFloat(ieeeF64, format, -1, 64)
    moreDecimals := strconv.FormatFloat(ieeeF64, format, precision, 64)
    plusOneDecimal := strconv.FormatFloat(ieeeF64, format, precision + 1, 64)
    fmt.Printf("\nfloat64hex: %s\n", hex)
    fmt.Printf("   actual using precision[-1]: %s\n", shortestNumber)  // INCORRECT in go1.11.4 windows/amd64
    fmt.Printf(" expected using precision[-1]: %s\n", expected)
    fmt.Printf("   actual using precision[%2d]: %s\n", precision, moreDecimals)
    fmt.Printf("   actual using precision[%2d]: %s\n", precision + 1, plusOneDecimal)
}

func main() {
    fpformat("439babe4b56e8a39", 'f', 0, "498484681984085570")
    fpformat("c4dee27bdef22651", 'g', 17, "-5.8339553793802237e+23")
}

What did you expect to see?

float64hex: 439babe4b56e8a39
   actual using precision[-1]: 498484681984085570
 expected using precision[-1]: 498484681984085570
   actual using precision[ 0]: 498484681984085568
   actual using precision[ 1]: 498484681984085568.0

float64hex: c4dee27bdef22651
   actual using precision[-1]: -5.8339553793802237e+23
 expected using precision[-1]: -5.8339553793802237e+23
   actual using precision[17]: -5.8339553793802237e+23
   actual using precision[18]: -5.83395537938022366e+23

What did you see instead?

float64hex: 439babe4b56e8a39
   actual using precision[-1]: 498484681984085560
 expected using precision[-1]: 498484681984085570
   actual using precision[ 0]: 498484681984085568
   actual using precision[ 1]: 498484681984085568.0

float64hex: c4dee27bdef22651
   actual using precision[-1]: -5.8339553793802236e+23
 expected using precision[-1]: -5.8339553793802237e+23
   actual using precision[17]: -5.8339553793802237e+23
   actual using precision[18]: -5.83395537938022366e+23

[ALTree]

I'm not sure I understand what issue you are reporting. The -1 option for prec is documented as

The special precision -1 uses the smallest number of digits necessary such that ParseFloat will return f exactly.

And this seems to hold for both your numbers:

package main

import (
	"fmt"
	"math"
	"strconv"
)

func test(num string) {
	n, err := strconv.ParseUint(num, 16, 64)
	if err != nil {
		panic("main")
	}

	f := math.Float64frombits(n)
	s := strconv.FormatFloat(f, 'f', -1, 64)
	g, err := strconv.ParseFloat(s, 64)
	if err != nil {
		panic("main")
	}

	fmt.Printf("%b\n", f)
	fmt.Printf("%b\n", g)
	fmt.Println(f == g)
}

func main() {
	test("439babe4b56e8a39")
	test("c4dee27bdef22651")
}

Note how f (the number I called FormatFloat on with prec -1) and g (the number returned by ParseFloat when called on the string from f) are bit-per-bit equal.

[cyberphone]

As you can see in my example using precision 0 for "integers" result in a value which after rounding should give another result while still being compatible with both ParseFloat and V8/EcmaScript.

This is a request rather than a bug. I did port 2000 lines to .NET but that wasn't a pleasure :(

[cyberphone]

Although a bit ugly the following workaround
https://github.com/cyberphone/json-canonicalization/blob/master/go/src/webpki.org/es6numfmt/es6numfmt.go
passes the 100 million test value suite:
https://github.com/cyberphone/json-canonicalization/tree/master/testdata

[bmkessler]

I'm not sure if this falls into a proposed change since (as noted above) the documentation only states that precision -1 outputs one of the shortest decimal representations that satisfies the round trip property, but I think the expectation is that the -1 precision output is also correctly rounded to the closest decimal representation to the input. Note that issue #2625 was resolved to produce the closest decimal. I believe that closest decimal is what other programming languages typically provide as well. The bug appears to be in this section of roundShortest:

// Okay to round up if upper has a different digit and either upper
// is inclusive or upper is bigger than the result of rounding up.
okup := m != u && (inclusive || m+1 < u || i+1 < upper.nd)

The round up comparison is byte-by-byte and so incorrectly fails for these inputs:

lower: 498484681984085536
d:     498484681984085568
upper: 498484681984085600

49848468198408557 < 49848468198408560 so should have okup == true
but '7' > '0' in the last digit

lower: 583395537938022332891136
d:     583395537938022366445568
upper: 583395537938022400000000

58339553793802237 < 58339553793802240 so should have okup == true
but '7' > '0' in the last digit

Adding @rsc as the author of the conversion routine.

[cyberphone]

The rounding issue also applies to a small set of floating point numbers:

Hex: c4af4eaaa5e19289

g -1: 
-7.3922251744542715e+22

g 17:
-7.3922251744542716e+22

[bmkessler]

That case is also the same issue with byte-wise comparison to '0'

lower: 73922251744542711611392
d:     73922251744542715805696
upper: 73922251744542720000000

73922251744542716 < 73922251744542720 so should have okup == true
but '6' > '0' in the last digit

[gopherbot]

Change https://golang.org/cl/157697 mentions this issue: strconv: fix rounding in FormatFloat fallback path

[cespare]

Though this bug has been around for a long time, I also happened to independently discover it a few days ago while implementing Ryu (see #15672 (comment)). I sent a CL though it will have to wait for Go 1.13.

Thanks @bmkessler for the very helpful analysis.

[cyberphone]

@cespare I tried the Java version of Ryu on x0000000000000001 and it returned 4.9e-324 which differs from the current Go implementation which returns 5e-324 (which also is compliant with JavaScript).