New floating-to-decimal formatting routine #24612

lifthrasiir · 2015-04-19T18:01:46Z

This is a direct port of my prior work on the float formatting. The detailed description is available here. In brief,

This adds a new hidden module core::num::flt2dec for testing from libcoretest. Why is it in core::num instead of core::fmt? Because I envision that the table used by flt2dec is directly applicable to dec2flt (cf. Float printing and/or parsing is inaccurate #24557) as well, which exceeds the realm of "formatting".
This contains both Dragon4 algorithm (exact, complete but slow) and Grisu3 algorithm (exact, fast but incomplete).
The code is accompanied with a large amount of self-tests and some exhaustive tests. In particular, libcoretest gets a new dependency on librand. For the external interface it relies on the existing test suite.
It is known that, in the best case, the entire formatting code has about 30 KBs of binary overhead (judged from strconv experiments). Not too bad but there might be a potential room for improvements.

This is rather large code. I did my best to comment and annotate the code, but you have been warned.

For the maximal availability the original code was licensed in CC0, but I've also dual-licensed it in MIT/Apache as well so there should be no licensing concern.

This is [breaking-change] as it changes the float output slightly (and it also affects the casing of inf and nan). I hope this is not a big deal though :)

Fixes #7030, #18038 and #24556. Also related to #6220 and #20870.

Known Issues

I've yet to finish make check-stage1. It does pass main test suites including run-pass but there might be some unknown edges on the doctests.
Figure out how this PR affects rustc.
Determine which internal routine is mapped to the formatting specifier. Depending on the decision, some internal routine can be safely removed (for instance, currently to_shortest_str is unused).

rust-highfive · 2015-04-19T18:01:51Z

r? @aturon

(rust_highfive has picked a reviewer for you, use r? to override)

pnkfelix · 2015-04-19T19:27:14Z

r? @pnkfelix

lifthrasiir · 2015-04-20T02:49:37Z

I've successfully finished make check-stage1. Also, the basic size benchmark done with fn main() { println!("hello, pi is {}", std::f64::consts::PI) }:

Options	2015-04-19 Nightly	This PR	Delta
`-C link-args=-s`	318,704	339,160	+20,456
`-C link-args=-s -O`	318,704	339,160	+20,456
`-C link-args=-s -O -C lto`	294,128	314,584	+20,456

(-C link-args=-s added because the binary from the current stage1 rustc seems to lack some debug sections.)

It seems that the original float formatting code had 10KB of overhead, so the best case overhead is somewhat amortized.

pnkfelix · 2015-04-20T15:07:26Z

Can you here summarize the effect of "it changes the float output slightly" beyond "affects the casing of inf and nan")?

In particular: There is some discussion in other tickets about things like:

whether to not to include the minus sign on -0.0,
or likewise, whether to include the .0 in +/-0.0.

I suspect that those sorts of changes are in some ways more breaking (or at least, more likely to catch developers unawares) than merely fixing our bugs with round-off error on the actual non-zero digits that we emit. And so it would be useful to know up front here whether this PR changes any of those behaviors, or if the changes to the float output is restricted to which (and how many) digits it chooses to emit on non-zero values.

lifthrasiir · 2015-04-20T15:25:05Z

@pnkfelix It is a bit hard to say, precisely because the old code was not well-behaved. I can however surely say that this does not change the intention (say, it doesn't make {:e} to print decimals only). More precisely, I think that exact changes are as follows:

When the precision is not given, all specifiers default to the shortest representation.
When the precision is given, all specifiers round precisely based on the exact (not shortest) decimal expansion. Unit tests have some examples, but namely, the exact value of 0.95f64 is slightly smaller than 0.95 and format!("{:.1}", 0.95f64) will give "0.9" instead of "1.0".
Given the enough precision (more than 800 fractional digits) it will (correctly) print trailling zeroes.
NaN is unsigned, i.e. format!("{:+?}", 0.0/0.0) is simply "NaN". (Previously it gave "+NaN".)

Edit 2015-04-22: I've changed the implementation so that it preserves the original behavior as much as possible. The changes specific to the original code (but subsequently reverted in this PR) are as follows:

For {} and {:?}, large (>=10^16) or small (<10^-4) value will rendered in the exponential notation.
Every letter inside the formatted number (e, inf, nan) is lowercased except for {:E} where it is uppercased instead. (Previously inf was always lowercased and NaN was always mixed-cased.)

Regarding the existing discussions, with an exception of the sign of nan, it behaves identically as before. It is easy to switch the behavior (e.g. in order to get a trailing .0, use frac_digits of 1), so I'm not too worried.

pnkfelix · 2015-04-20T15:30:38Z

src/libcore/num/flt2dec/mod.rs

+limitation `limit` (which determines the actual `n`), and we would like to get
+the representation `V = 0.d[0..n-1] * 10^k` such that:
+
+- `d[0]` is non-zero, unless `n` was zero in which case only `k` is returned.


I don't understand how the returned value for k is meaningful in the case where n is 0 ... is this comment implying that in such a scenario, the returned d[0] would have been non-zero if n had been positive?

No. For example, v = 3.14 and limit = 3 gives n = 0 and k = 3. The extrapolated digits would start with 00314....

Ah, re-reading your comment I understand what you really meant; yes, k <= limit holds if and only if n = 0. The exact value of k is thus unused in that case, but the code has a debugging assertion to check this identity.

There actually was one wrong comment (derived from the older faulty implementation) on to_exact_fixed_str implying that n cannot be zero (oops). I'll fix that soon.

No, I was and still am confused. In the comment above, n is the number of digits that we need to extract from the array d in the representation V = 0.d[0..n-1] * 10^k. My comments were based on that equation; it led me to ask the question: when can n be zero?

At the time when I wrote that question, I had not yet read the paragraph that starts with

When limit is given but not n, ...

which seems to describe scenarios that yield non-positive values for n. But this mostly just confused me further, probably because I do not yet actually understand the conversion that you have named "fixed mode conversion".

The text in that paragraph that makes it sound like the equation V = 0.d[0..n-1] * 10^k simply cannot be well-defined ... maybe that is the source of my confusion.

I will try reading the whole text here and see if I can suggest a way to rephrase it to be clearer.

(BTW I do recognize that the equation for V makes sense for n = 0 when V = 0 itself. But you yourself gave the example of v = 3.14; so clearly I am still not understanding something here.)

Probably I've introduced the concepts backwards.

The "exact mode conversion" is the mode that the caller demands the specified number of significant digits.

The "fixed mode conversion" is the mode that the caller demands the specified number of fractional digits.

The fixed mode can be implemented via the exact mode and the estimator, which calculates the required number of significant digits from the number of fractional digits and the original value. It turns out that the internal digit generation code already has to estimate the exponent (which is almost same to the difference between them), so we can reuse this logic.

In order to achieve this merger, the combined exact-fixed mode is implemented so that both the number of significant digits len and the number of fractional digits frac_digits are given. (The actual interface differs from this to be more flexible: len == buf.len() and frac_digits == -limit. This doesn't change the discussion however.) The digit generation code will finish whichever comes first, so it is possible that there are no digits generated. The n as I originally described is really determined from either len or frac_digits, so it might have been confusing.

By the way, I realized that my original example doesn't arise with the current external interface. So I'll give a better example: v = 0.000045, len = 3, frac_digits = 3 (so that limit = -3). Since v = 0.45 * 10^-4, k = -4, but this clearly triggers the condition k <= limit. This means that the resulting representation is all zeroes, i.e. V = 0.000 (note that k is unused here).

(The better description for this is always welcomed. 😭)

Oh wait! Are you actually saying:

we would like to get the representation V = 0.d[0..n-1] * 10^k such that d[0] is non-zero, unless n was zero in which case k digits are returned.

That I could understand, I think!

I will try re-reading this text, but substituting e.g. "we will only get k digits" in the spots where you wrote "we will only get k", and see if it all becomes clear to me then.

Uh, no. k is just the exponent which does not depend on the determined n (which is the number of digits actually returned). Say that, for certain v, the algorithm returned 0.123 * 10^-8 (which implies that n = 3, and also implies either len = 3 or limit = -11). Then for some other values for limit, it may return 0. * 10^-8 (with n = 0 and k = -8). 10^-8 part still remains here, therefore "we will only get k".

Oh sorry, somehow I didn't hit "reload" before I posted my last comment, so it may seem like I completely ignored you in my last comment.

I am re-reading the Steele+White Dragon paper now; I think your terminology matches theirs, so I just need to digest section 8 of their paper and figure out the right way to succinctly describe it.

Though to be honest the comment you wrote that points out the distinction between requesting a number of significant digits versus a number of fractional digits has helped me a lot.

Okay, back to reading, thanks.

You are welcome. Actually, I guess my comment above can directly go to the module documentation...

rprichard · 2015-04-21T04:32:48Z

The Display behavior prior to this change always used a decimal format (like %f). With this PR, it uses either exponent or decimal depending on magnitude (like %g). I think that's the right default, which #24556 argues for, but it's also a rather largish breaking change that I didn't see mentioned above. Display can format 1e300, which breaks the JSON serializer:

JSON:

            let s = v.to_string();
            if s.contains(".") {s} else {s + ".0"}

1e300.0 is not valid.

The CSV serializer works because Display in this PR reverts to decimal when a precision is specified:

CSV:

    let s: String = format!("{:.10}", v).trim_right_matches('0').into();
    if s.ends_with('.') { s + "0" } else { s }

lifthrasiir · 2015-04-21T04:40:57Z

@rprichard Aargh, I forgot to mention that! I have updated the earlier comment to include that breaking change. Thank you so much.

(As mentioned above, this too is very easy to change though. If the decision is reached I'll rebase this commit. I explicitly mentioned this kind of decision in the "known issues" to gather the future direction, as it seems that there are ongoing discussions about changing defaults.)

pnkfelix · 2015-04-21T08:20:54Z

@lifthrasiir hmm, it will probably be easier to land the most important parts of this PR (namely, the bug fixes) if you remove the big breaking change(s) like the one that @rprichard mentioned.

(Hopefully we could figure out a way for a user to opt-in to getting the %g-like behavior, either by adding a new format trait, or by changing the return type of the precision method and adding a new variant to its rule for precision in the grammar.)

But, I'll make sure we talk about this at the team meeting tonight. Maybe we can go ahead and change this without going through an RFC.

lifthrasiir · 2015-04-21T08:38:25Z

@pnkfelix Yeah, that was my oversight, I forgot to replicate the old behavior in some cases. I'll rebase this tonight (i.e. within 6 hours) to avoid any breaking changes except for fixing the rounding and inaccurate result.

rprichard · 2015-04-21T08:56:51Z

FWIW, the %g-like format is the default in many languages -- probably because it's non-lossy, concise, and convenient for values near zero in magnitude. It is awkward that it can take either of two different forms, though, and perhaps it's too late. I think the precision field for %g typically controls the total number of digits (before and after the decimal point).

pnkfelix · 2015-04-21T09:26:51Z

src/libcore/num/flt2dec/bignum.rs

+                    self.base[i] = 0;
+                }
+
+                // shift by `nbits` bits


nit: nbits must be referring to bits here, right?

pnkfelix · 2015-04-21T14:53:05Z

src/libcore/num/flt2dec/strategy/grisu.rs

+    /// The integer mantissa.
+    pub f: u64,
+    /// The exponent in base 2.
+    pub e: i16,


I haven't read the Grisu paper carefully, but in its discussion of the diy_fp struct, it points out

(1.) the paper's definition uses an unlimited range value for e_x to simplify the proofs,

(2.) in practice the exponent type must only have "slightly greater range than the input exponent" and

(3.) for IEEE doubles which have 11-bit exponents, a 32-bit signed integer is by far big enough.

The combination of 2. and 3. leads me to wonder whether we could get into trouble using only i16 here, which only has five extra bits. I know you exhaustively tested your algorithm for f32, but how confident are you that i16 suffices for f64 ?

That is more than enough. Grisu does have a stricter limitation on the mantissa size (Grisu1 needs 2 more extra bits, Grisu2 and Grisu3 needs 3), but they are very generous about the exponent range.

diy_fp is converted from the decoded float, then normalized, then multiplied by the cached power of 10 to get that within a certain interval (to be exact, [4, 2^32)). Since the normalization step can decrease the exponent by at most 63, and the original float has the minimal exponent of -1075, the exponent is no less than -1138. With the same argument, the exponent is no more than 971.

(In fact, this exponent type is enough for f80 if Rust had support for that. The problem with f80 is, however, that its mantissa is too large and u64 would not be sufficient.)

pnkfelix · 2015-04-21T15:57:00Z

(By the way this is all looking pretty fantastic so far. I just want to really make sure I or someone else besides you has a decent understanding of all of it, so please do bear with me while I continue to work my way through it.)

pnkfelix · 2015-05-09T07:49:27Z

@lifthrasiir okay, thanks for your patience with this.

I'm going to r+ this. I cannot claim that I understand the algorithms 100%, but I have reviewed all the papers and satisfied myself that this implementation is a good match for them, and that the goals of the papers are a match for what we want in Rust.

pnkfelix · 2015-05-09T07:50:14Z

@bors r+ 3d34e17 p=1

tshepang · 2015-05-09T09:04:43Z

epic!

bors · 2015-05-09T11:10:38Z

⌛ Testing commit 3d34e17 with merge d8b3a6a...

bors · 2015-05-09T11:54:14Z

💔 Test failed - auto-mac-64-opt

lifthrasiir · 2015-05-09T12:17:51Z

Whoops, I forgot to run make check-stage1 (not just make check-stage1-coretest). This failure is nothing to do with the implementation but only the tests, I'll shortly fix that.

lifthrasiir · 2015-05-09T13:53:35Z

I've confirmed this new commit passes make check-stage1. r? @pnkfelix

pnkfelix · 2015-05-09T14:33:00Z

@bors r+ 1aecd17 p=1

bors · 2015-05-09T14:56:57Z

⌛ Testing commit 1aecd17 with merge 67ba6dc...

This is a direct port of my prior work on the float formatting. The detailed description is available [here](https://github.com/lifthrasiir/rust-strconv#flt2dec). In brief, * This adds a new hidden module `core::num::flt2dec` for testing from `libcoretest`. Why is it in `core::num` instead of `core::fmt`? Because I envision that the table used by `flt2dec` is directly applicable to `dec2flt` (cf. #24557) as well, which exceeds the realm of "formatting". * This contains both Dragon4 algorithm (exact, complete but slow) and Grisu3 algorithm (exact, fast but incomplete). * The code is accompanied with a large amount of self-tests and some exhaustive tests. In particular, `libcoretest` gets a new dependency on `librand`. For the external interface it relies on the existing test suite. * It is known that, in the best case, the entire formatting code has about 30 KBs of binary overhead (judged from strconv experiments). Not too bad but there might be a potential room for improvements. This is rather large code. I did my best to comment and annotate the code, but you have been warned. For the maximal availability the original code was licensed in CC0, but I've also dual-licensed it in MIT/Apache as well so there should be no licensing concern. This is [breaking-change] as it changes the float output slightly (and it also affects the casing of `inf` and `nan`). I hope this is not a big deal though :) Fixes #7030, #18038 and #24556. Also related to #6220 and #20870. ## Known Issues - [x] I've yet to finish `make check-stage1`. It does pass main test suites including `run-pass` but there might be some unknown edges on the doctests. - [ ] Figure out how this PR affects rustc. - [ ] Determine which internal routine is mapped to the formatting specifier. Depending on the decision, some internal routine can be safely removed (for instance, currently `to_shortest_str` is unused).

bors · 2015-05-09T16:48:09Z

☀️ Test successful - auto-linux-32-nopt-t, auto-linux-32-opt, auto-linux-64-nopt-t, auto-linux-64-opt, auto-linux-64-x-android-t, auto-mac-32-opt, auto-mac-64-nopt-t, auto-mac-64-opt, auto-win-32-nopt-t, auto-win-32-opt, auto-win-64-nopt-t, auto-win-64-opt

emberian · 2015-05-10T14:07:41Z

Awesome! Epic review @pnkfelix, and thanks to @lifthrasiir for implementing these tricky algorithms.

Kimundi · 2015-05-20T08:12:17Z

Cool! Finally a proper implementation for this stuff :)

- Makes Num stringification 2x faster (tested with rand.Str) - Fixes RT#127184 https://rt.perl.org/Ticket/Display.html?id=127184 - Fixes RT#124796 https://rt.perl.org/Ticket/Display.html?id=124796 - Fixes RT#132330 https://rt.perl.org/Ticket/Display.html?id=132330 (fixes Num.WHICH and problems with set()s mentioned in that ticket) Grisu3[^1] is a 2010 algorithm that stringifies doubles to shortest possible representation that would still result in the same value if it's parsed and stringified again. This sort of strigification is what fixes the reported bugs. As a bonus, the algo is faster than snprintf('%.15g') we used to use for stringification. Grisu3 handles ~99.5% of cases. In the remaining 0.5%, it knows it won't produce the shortest possible result and so a fallback to slower algorithm is made, which in our current case is snprintf('%.17g'), which is a suboptimal fallback, and in the future Dragon4 algo[^2] could be used instead. The change from .15g to .17g in the fallback is intentional, as 15 is the number of guaranteed significant digits, but 17 is the number of maximum available significant digits when the IEEE doubles differ by at most one unit[^7]. Based on my research, an improved fallback would be Dragon4 algo[^2], and Grisu3+Dragon4 is the current state of the art. (some references may state that Errol[^8] algo superseded Grisu3+Dragon4 combo, but based on authors' corrections[^9], it appears there was a benchmarking error and Errol is not in fact faster). The Grisu3+Dragon4 is used[^5] by Rust (see also some trial impls in [^6]) and Grisu3's author's C++ version of the code[^3] indicates it's used by V8 JavaScript engine as well. There exist a C# impl[^4] of the algo as well, used in an efficient JSON encoder. [1] "Printing Floating-Point Numbers Quickly and Accurately with Integers" by Florian Loitsch https://goo.gl/cbvogg [2] "How to Print Floating-Point Numbers Accurately" by Steel & White: https://lists.nongnu.org/archive/html/gcl-devel/2012-10/pdfkieTlklRzN.pdf [3] https://github.com/google/double-conversion [4] https://github.com/kring/grisu.net [5] rust-lang/rust#24612 [6] https://github.com/lifthrasiir/rust-strconv [7] http://www2.open-std.org/JTC1/SC22/WG21/docs/papers/2005/n1822.pdf [8] https://github.com/marcandrysco/Errol [9] https://github.com/marcandrysco/Errol#performance-evaluation-correction

- Makes Num stringification 2x faster (tested with rand.Str) - Fixes RT#127184 https://rt.perl.org/Ticket/Display.html?id=127184 - Fixes RT#132330 https://rt.perl.org/Ticket/Display.html?id=132330 (fixes Num.WHICH and problems with set()s mentioned in that ticket) Grisu3[^1] is a 2010 algorithm that stringifies doubles to shortest possible representation that would still result in the same value if it's parsed and stringified again. This sort of strigification is what fixes the reported bugs. As a bonus, the algo is faster than snprintf('%.15g') we used to use for stringification. Grisu3 handles ~99.5% of cases. In the remaining 0.5%, it knows it won't produce the shortest possible result and so a fallback to slower algorithm is made, which in our current case is snprintf('%.17g'), which is a suboptimal fallback, and in the future Dragon4 algo[^2] could be used instead. The change from .15g to .17g in the fallback is intentional, as 15 is the number of guaranteed significant digits, but 17 is the number of maximum available significant digits when the IEEE doubles differ by at most one unit[^7]. Based on my research, an improved fallback would be Dragon4 algo[^2], and Grisu3+Dragon4 is the current state of the art. (some references may state that Errol[^8] algo superseded Grisu3+Dragon4 combo, but based on authors' corrections[^9], it appears there was a benchmarking error and Errol is not in fact faster). The Grisu3+Dragon4 is used[^5] by Rust (see also some trial impls in [^6]) and Grisu3's author's C++ version of the code[^3] indicates it's used by V8 JavaScript engine as well. There exist a C# impl[^4] of the algo as well, used in an efficient JSON encoder. [1] "Printing Floating-Point Numbers Quickly and Accurately with Integers" by Florian Loitsch https://goo.gl/cbvogg [2] "How to Print Floating-Point Numbers Accurately" by Steel & White: https://lists.nongnu.org/archive/html/gcl-devel/2012-10/pdfkieTlklRzN.pdf [3] https://github.com/google/double-conversion [4] https://github.com/kring/grisu.net [5] rust-lang/rust#24612 [6] https://github.com/lifthrasiir/rust-strconv [7] http://www2.open-std.org/JTC1/SC22/WG21/docs/papers/2005/n1822.pdf [8] https://github.com/marcandrysco/Errol [9] https://github.com/marcandrysco/Errol#performance-evaluation-correction

rust-highfive assigned aturon Apr 19, 2015

rust-highfive assigned pnkfelix and unassigned aturon Apr 19, 2015

lifthrasiir mentioned this pull request Apr 20, 2015

Float printing and/or parsing is inaccurate #24557

Closed

pnkfelix reviewed Apr 20, 2015
View reviewed changes

This was referenced Apr 20, 2015

Add i128 and u128 types rust-lang/rfcs#521

Closed

Printing of the largest and other very large floats is wrong #7030

Closed

pnkfelix reviewed Apr 21, 2015
View reviewed changes

lifthrasiir force-pushed the flt2dec branch 3 times, most recently from 97c76cb to dccacbc Compare April 21, 2015 12:27

pnkfelix reviewed Apr 21, 2015
View reviewed changes

test: fixed some tests affected by banker's rounding.

1aecd17

bors merged commit 1aecd17 into rust-lang:master May 9, 2015

This was referenced May 10, 2015

assert_eq! output is very unhelpful for floats #20870

Closed

largs floats print slightly wrong #18038

Closed

sfackler mentioned this pull request May 30, 2015

from_str_radix_float gives incorrect (/imprecise) results on beta and nightly #25895

Closed

hanna-kruppe mentioned this pull request Jul 26, 2015

Decimal to floating point conversion #27307

Merged

hanna-kruppe mentioned this pull request Oct 30, 2015

Pretty print isn't pretty for floats #29472

Closed

huonw mentioned this pull request Jan 5, 2016

Number to/from string API #6220

Closed

hanna-kruppe mentioned this pull request Jan 17, 2016

"{:?}" for 0. is 0 instead of 0. #30967

Closed

hanna-kruppe mentioned this pull request Feb 3, 2017

Escaping char in libcore adds 2k of static data for no_std cases #39492

Open

lifthrasiir mentioned this pull request Apr 9, 2017

Add support for hexadecimal float literals #1433

Closed

hanna-kruppe mentioned this pull request Aug 3, 2019

General floating point formatting in Debug with {:g?} rust-lang/rfcs#2729

Closed

lilyball mannequin mentioned this pull request Jan 3, 2016

[SR-447] Parsing strings into floating-point performs badly swiftlang/swift#43064

Closed

lilyball mannequin mentioned this pull request May 26, 2023

[SR-454] Reimplement float -> string with better algorithms swiftlang/swift#43071

Closed

New floating-to-decimal formatting routine #24612

New floating-to-decimal formatting routine #24612

Conversation

lifthrasiir commented Apr 19, 2015

Known Issues

rust-highfive commented Apr 19, 2015

pnkfelix commented Apr 19, 2015

lifthrasiir commented Apr 20, 2015

pnkfelix commented Apr 20, 2015

lifthrasiir commented Apr 20, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rprichard commented Apr 21, 2015

lifthrasiir commented Apr 21, 2015

pnkfelix commented Apr 21, 2015

lifthrasiir commented Apr 21, 2015

rprichard commented Apr 21, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pnkfelix commented Apr 21, 2015

pnkfelix commented May 9, 2015

pnkfelix commented May 9, 2015

tshepang commented May 9, 2015

bors commented May 9, 2015

bors commented May 9, 2015

lifthrasiir commented May 9, 2015

lifthrasiir commented May 9, 2015

pnkfelix commented May 9, 2015

bors commented May 9, 2015

bors commented May 9, 2015

emberian commented May 10, 2015

Kimundi commented May 20, 2015