Tracking issue for float_bits_conv feature #40470

est31 · 2017-03-13T09:19:52Z

Tracking issue for the conversion float_bits_conv feature, added by PR #39271.

The main topic during the PR's thread has been about how to handle signaling NaNs, and the final decision was to silently mask them, especially as LLVM behaviour regarding them is not well specified.

Another issue raised by @dwrensha was that sNaNs are not encoded consistently. IEEE 754-2008 (section 8.2.1 in the latest draft) specifies quiet NaNs to have their first bit of the trailing significand field set to 1, for signaling NaNs it should be set to 0. Some MIPS based platforms have it reversed. Should they be cfg'd out?

The text was updated successfully, but these errors were encountered:

Add functions to safely transmute float to int The safe subset of Rust tries to be as powerful as possible. While it is very powerful already, its currently impossible to safely transmute integers to floats. While crates exist that provide a safe interface, most prominently the `iee754` crate (which also inspired naming of the added functions), they themselves only use the unsafe `mem::transmute` function to accomplish this task. Also, including an entire crate for just two lines of unsafe code seems quite wasteful. That's why this PR adds functions to safely transmute integers to floats and vice versa, currently gated by the newly added `float_bits_conv` feature. The functions added are no niche case. Not just `ieee754` [currently implements](https://github.com/huonw/ieee754/blob/master/src/lib.rs#L441) float to int transmutation via unsafe code but also the [very popular `byteorder` crate](https://github.com/BurntSushi/byteorder/blob/1.0.0/src/lib.rs#L258). This functionality of byteorder is in turn used by higher level crates. I only give two examples out of many: [chor](https://github.com/pyfisch/cbor/blob/a7363ea9aaf372e3d24b52414b5c76552ecc91c8/src/ser.rs#L227) and [bincode](https://github.com/TyOverby/bincode/blob/f06a4cfcb5b194e54d4997c200c75b88b6c3fba4/src/serde/reader.rs#L218). One alternative would be to manually use functions like pow or multiplication by 1 to get a similar result, but they only work in the int -> float direction, and are not bit exact, and much slower (also, most likely the optimizer will never optimize it to a transmute because the conversion is not bit exact while the transmute is). Tracking issue: #40470

est31 · 2017-04-19T15:36:51Z

Hmm, seems this feature has broken the ieee754 crate:

error: use of unstable library feature 'float_bits_conv': recently added (see issue #40470)
cargo/registry/src/github.com-1ecc6299db9ec823/ieee754-0.2.1/src/lib.rs:458:17
    |
458 |                 Self::from_bits(
    |                 ^^^^^^^^^^^^^^^
...
610 | mk_impl!(f64, u64, i16, u16, u64, 11, 52);
    | ------------------------------------------ in this macro invocation

Apparently you can't shadow functions with a trait: https://is.gd/oKiVZH (the first assert works, but the second fails)

est31 · 2017-04-19T15:45:16Z

Is the issue above something we have to fix, or rather something the ieee754 crate has to?

hanna-kruppe · 2017-04-30T17:35:47Z

@est31 RFC 1105 laid out that adding inherent methods is a minor change so it's up to the team(s) to decide whether we're okay with the practical impact of the change.

est31 · 2017-07-03T14:57:43Z

cc #43025 --- with this merged, could we get this feature stabilized in Rust 1.20?

BurntSushi · 2017-07-04T18:33:38Z

#43055 has been submitted to stabilize this.

@rfcbot fcp merge

rfcbot · 2017-07-04T19:00:29Z

Team member @BurntSushi has proposed to merge this. The next step is review by the rest of the tagged teams:

No concerns currently listed.

Once these reviewers reach consensus, this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

alexcrichton · 2017-07-04T20:02:48Z

@rfcbot reviewed

rfcbot · 2017-07-06T20:20:45Z

🔔 This is now entering its final comment period, as per the review above. 🔔

rfcbot · 2017-07-06T20:28:37Z

🔔 This is now entering its final comment period, as per the review above. 🔔

rfcbot · 2017-07-16T20:32:41Z

The final comment period is now complete.

rfcbot · 2017-07-16T20:32:41Z

The final comment period is now complete.

@BurntSushi

Stabilize float_bits_conv for Rust 1.21 Stabilizes the `float_bits_conv` lib feature for the 1.20 release of Rust. I've initially implemented the feature in #39271 and later made PR #43025 to output quiet NaNs even on platforms with different encodings, which seems to have been the only unresolved issue of the API. Due to PR #43025 being only applied to master this stabilisation can't happen for Rust 1.19 through the usual "stabilisation on beta" system that is being done for library APIs. r? @BurntSushi closes #40470.

Gankra · 2017-11-13T03:52:04Z

Really late to this party. As I note here, the NaN masking is unnecessary from LLVM's perspective.

I'm not totally clear on the platform-specific-NaN-reprs issue, but at least on x86/x64, we can just make from_bits into transmute safely. (probably also aarch64? At least its SIMD units are IEEE 754-2008 compliant, which specs NaN repr). The documentation has been written sufficiently vaguely to allow us to change the implementation to a transmute.

If we wish to do so, this just leaves the following question: do we ignore the existence of non-2008-compliant hardware (transmute on all platforms) or do we allow behaviour to be platform-specific (transmute only on definitely-2008-compliant hardware)?

sfackler · 2017-11-13T04:01:26Z

I'm certainly on board with just transmuting on architectures we know are okay.

Amanieu · 2017-11-13T04:16:21Z

Transmuting should be safe on ARM & AArch64.

The only issue is whether a sNaN can cause a trap when it is used. By default floating-point exception are masked on ARM/AArch64, but can be enabled by setting some bits in the fpscr/fpcr register.

These bits also allow enabling exceptions for situations like inexact floating-point rounding, float division by zero, etc. These operations are all supported in safe Rust, so I suggest that the official Rust policy be something along the lines: "Rust assumes that all floating-point operations do not trap."

sfackler · 2017-11-13T04:21:12Z

I don't think we really care if the operation traps, right? It's not unsafe to do so.

Amanieu · 2017-11-13T04:24:44Z

We do care if the trap is considered an "observable side-effect" of the operation, since this affects optimization. This would impact dead code elimination for example: you would still have to trigger the trap even if the result of the operation is not used.

sfackler · 2017-11-13T04:27:37Z

Ah sure that makes sense.

est31 · 2017-11-13T05:44:06Z

@gankro

As I note here, the NaN masking is unnecessary from LLVM's perspective.

very interesting! We need more people like you :).

I am personally okay with anything (the documentation was made vague purposefully!), but before we commit to anything that we can't reverse (e.g. making docs super clear that it is only a transmute), I'd love to get @gankro 's LLVM patch merged upstream so that it has been reviewed by LLVM reviewers and deemed okay.

sunfishcode · 2017-11-13T15:21:09Z

Unfortunately, it is still necessary from LLVM's perspective; see https://github.com/llvm-mirror/llvm/blob/master/lib/Analysis/InstructionSimplify.cpp#L4142 for example. I support changing LLVM's behavior here, but this isn't just a case of fixing mistaken documentation; it's changing the policy, so we need to be careful.

Gankra · 2017-11-13T16:36:24Z

Interesting. That transformation is justified by sNaN being bad, but in fact doesn't look for sNaN. Rather it looks for undef, and up to this point we've fairly conservatively considered doing ~anything with undef to be UB for Rust (because undef is poorly defined and we don't want to couple with LLVM semantics anyway).

As such, the existence of that transformation shouldn't actually be a barrier to us making this change? (we should still fix llvm as well)

jrmuizel · 2017-11-13T18:33:49Z

FWIW, that change originally came from here: llvm-mirror/llvm@50b2ca4

sunfishcode · 2017-11-15T19:38:50Z

@gankro That's a good point. Allowing Rust to produce sNaNs in more places doesn't break those kinds of optimizations. And I'm not aware of any optimizations that explicitly check for sNaNs, and I've checked all the usual suspects. So this change seems ok.

Make float::from_bits transmute See commit message for details. See also this discussion here: #40470 (comment) (may require libs team discussion before merging)

est31 · 2017-11-24T20:55:33Z

cc #46246

Lower the priority of unstable methods when picking a candidate. Previously, when searching for the impl of a method, we do not consider the stability of the impl. This leads to lots of insta-inference-regressions due to method ambiguity when a popular name is chosen. This has happened multiple times in Rust's history e.g. * `f64::from_bits` #40470 * `Ord::{min, max}` #42496 * `Ord::clamp` #44095 (eventually got reverted due to these breakages) * `Iterator::flatten` #48115 (recently added) This PR changes the probing order so that unstable items are considered last. If a stable item is found, the unstable items will not be considered (but a future-incompatible warning will still be emitted), thus allowing stable code continue to function without using qualified names. Once the unstable feature is stabilized, the ambiguity error will still be emitted, but the user can also use newly stable std methods, while the current situation is that downstream user is forced to update the code without any immediate benefit. (I hope that we could bring back `Ord::clamp` if this PR is merged.)

est31 mentioned this issue Mar 13, 2017

Add functions to safely transmute float to int #39271

Merged

alexcrichton added B-unstable Blocker: Implemented in the nightly compiler and unstable. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. labels Mar 13, 2017

DenialAdams mentioned this issue Apr 22, 2017

Make font_from_bytes public ggez/ggez#62

Merged

sphinxc0re mentioned this issue Apr 26, 2017

Build fails on current beta huonw/ieee754#2

Closed

brson mentioned this issue May 6, 2017

Rust 1.18 regression - ieee754-0.2.1, from_bits #41793

Closed

NikVolf mentioned this issue May 31, 2017

Avoid unsafe transmuting ints to floats paritytech/parity-wasm#18

Closed

sunfishcode mentioned this issue Jun 6, 2017

Remove use of unsafe transmute in read_f32 and read_f64 bytecodealliance/wasmparser#14

Closed

jdm mentioned this issue Jun 12, 2017

cargo build failed pcwalton/rust-media#36

Closed

This was referenced Jun 26, 2017

Can't compile due to use of unstable library feature 'float_bits_conv' RustAudio/lewton#21

Closed

Fix on nightly. huonw/ieee754#4

Merged

est31 mentioned this issue Jul 4, 2017

Stabilize float_bits_conv for Rust 1.20 #43055

Merged

rfcbot added the proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. label Jul 4, 2017

rfcbot added final-comment-period In the final comment period and will be merged soon unless new substantive objections are raised. and removed proposed-final-comment-period Proposed to merge/close by relevant subteam, see T-<team> label. Will enter FCP once signed off. labels Jul 6, 2017

bors closed this as completed in #43055 Jul 17, 2017

tomaka mentioned this issue Aug 9, 2017

Runtime VertexAttributes vulkano-rs/vulkano#736

Closed

Gankra mentioned this issue Nov 15, 2017

Make float::from_bits transmute #46012

Merged

bors added a commit that referenced this issue Nov 24, 2017

Auto merge of #46012 - Gankro:float-conv-transmute, r=sfackler

078fbe4

Make float::from_bits transmute See commit message for details. See also this discussion here: #40470 (comment) (may require libs team discussion before merging)

bors added a commit that referenced this issue Nov 24, 2017

Auto merge of #46012 - Gankro:float-conv-transmute, r=sfackler

85d50ce

Make float::from_bits transmute See commit message for details. See also this discussion here: #40470 (comment) (may require libs team discussion before merging)

kennytm mentioned this issue Feb 26, 2018

Lower the priority of unstable methods when picking a candidate. #48552

Merged

hanna-kruppe mentioned this issue Aug 24, 2018

Area proposal: Representation and validity invariants rust-lang/unsafe-code-guidelines#5

Merged

nikomatsakis mentioned this issue Aug 30, 2018

Representation of bool, integers and floating points rust-lang/unsafe-code-guidelines#9

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tracking issue for float_bits_conv feature #40470

Tracking issue for float_bits_conv feature #40470

est31 commented Mar 13, 2017 •

edited

Loading

est31 commented Apr 19, 2017

est31 commented Apr 19, 2017

hanna-kruppe commented Apr 30, 2017

est31 commented Jul 3, 2017

BurntSushi commented Jul 4, 2017

rfcbot commented Jul 4, 2017 •

edited by aturon

Loading

alexcrichton commented Jul 4, 2017

rfcbot commented Jul 6, 2017

rfcbot commented Jul 6, 2017

rfcbot commented Jul 16, 2017

rfcbot commented Jul 16, 2017

Gankra commented Nov 13, 2017

sfackler commented Nov 13, 2017

Amanieu commented Nov 13, 2017

sfackler commented Nov 13, 2017

Amanieu commented Nov 13, 2017

sfackler commented Nov 13, 2017

est31 commented Nov 13, 2017

sunfishcode commented Nov 13, 2017

Gankra commented Nov 13, 2017

jrmuizel commented Nov 13, 2017

sunfishcode commented Nov 15, 2017

est31 commented Nov 24, 2017

Tracking issue for float_bits_conv feature #40470

Tracking issue for float_bits_conv feature #40470

Comments

est31 commented Mar 13, 2017 • edited Loading

est31 commented Apr 19, 2017

est31 commented Apr 19, 2017

hanna-kruppe commented Apr 30, 2017

est31 commented Jul 3, 2017

BurntSushi commented Jul 4, 2017

rfcbot commented Jul 4, 2017 • edited by aturon Loading

alexcrichton commented Jul 4, 2017

rfcbot commented Jul 6, 2017

rfcbot commented Jul 6, 2017

rfcbot commented Jul 16, 2017

rfcbot commented Jul 16, 2017

Gankra commented Nov 13, 2017

sfackler commented Nov 13, 2017

Amanieu commented Nov 13, 2017

sfackler commented Nov 13, 2017

Amanieu commented Nov 13, 2017

sfackler commented Nov 13, 2017

est31 commented Nov 13, 2017

sunfishcode commented Nov 13, 2017

Gankra commented Nov 13, 2017

jrmuizel commented Nov 13, 2017

sunfishcode commented Nov 15, 2017

est31 commented Nov 24, 2017

est31 commented Mar 13, 2017 •

edited

Loading

rfcbot commented Jul 4, 2017 •

edited by aturon

Loading