-
Notifications
You must be signed in to change notification settings - Fork 12.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add functions to safely transmute float to int #39271
Conversation
r? @BurntSushi (rust_highfive has picked a reviewer for you, use r? to override) |
@alexcrichton I think we briefly talked about this at one point and you mentioned there were possible safety issues related to |
My only worry here would be related to "signaling nans" where some bit patterns of nan may cause hardware traps in some floating point operations (I believe). IIRC we don't generate them normally, so I don't think we've brushed up against them yet (I may be wrong). That being said I think it has to do with weird processor mode features so I'd be fine having a feature like this. |
Signaling NaNs are only a concern when going from int to float. So I'd be totally down with safe bitcasting from float to int, but I'm totally opposed to safe bitcasting from int to float. |
@retep998 Can you explain the issue in more detail for people unfamiliar with signaling NaNs and under what circumstances they violate safety? |
A signalling NaN is a floating point number with a certain bit pattern that causes the CPU to trap when attempting to perform any operations with it, which means it'll raise an interrupt or signal or exception (similar to what division by zero does). As a result, LLVM defines operations on signalling NaNs to be undefined behavior, so if you can create a signalling NaN in safe code then you can create undefined behavior which definitely violates Rust's rules. Being able to bitcast an int to a float allows you to get any bit pattern of float you desire, including signalling NaNs, hence it should not be allowed in safe code. |
Hmm, what about making |
@BurntSushi In principle code like this
should cause this "Invalid Operation" exception
If segfault is bad enough to be considered unsafety, then sigfpe is bad too? ... I guess? |
Well, I don't think segfaults directly imply unsafety, they are just highly correlated, no? For example, if using a signaling NaN wasn't undefined behavior and always caused your program to abort, then I wouldn't consider that a violation of safety. (Similar to, say, a |
Making it panic seems like a fine solution to me (if signaling NaN's are indeed unsafe). |
I think returning Err is better because the most likely use case is in deserialisation code, and here panicing on well crafted input is not a very good thing to do. You can still unwrap or expect if you want to panic in such a case, but if the code paniced instead of returning an Err, you'd have to duplicate checks. |
src/libstd/f32.rs
Outdated
/// | ||
/// Note that this function is distinct from casting. | ||
/// | ||
/// ``` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: The examples should be under an # Examples
heading
Looks good to me! |
What are the primary uses of floats transmuted from integers, by the way? I can imagine two scenarios:
|
To restate my concerns here:
|
Duplicating my comment from reddit b/c I don't expect people to read reddit C11 standard says the behavior of hard/soft The ISO/IEC 60559 states [2]
So I believe panicking on [1] ISO/IEC 9899/201X Standard: Link Section: 5.2.4.2.2 Bullet Point 3, annotation 22 [2] ISO/IEC 60559 Standard: Link Section 12 first-ish paragraph. |
@valarauca Can Rust diverge from C11 and make it well defined? (Is that even a coherent question?) |
@BurntSushi I think Rust diverging from C to make undefined behavior well defined is one of the core missions of Rust (?) My recommendation is panicking is fine. |
Sure, I just don't know the details between "make it well defined" and "make sure this is consistent with how we're using LLVM." |
I've updated my signaling NaN example in #39271 (comment). Now it outputs
on Linux. A possible solution:
If FP exceptions are disabled, then signaling NaNs are converted into quiet NaNs by arithmetic operations, so this is completely safe. The behavior is controlled by IEEE 754-2008, not implementations. |
What ever is done will have to be encoded into the Rust standard library and fixing various bugs will be a continuous effort. The LLVM doesn't handle The real issue is platforms. For GPU's, ARMv7, ARMv8, and NEON subnormals are flushed to zero (which isn't IEEE754 or ISO/IEC 60559 conformant) so there really isn't any The easiest way to do this is have a something like: fn is_nan(&self) -> bool {
*self != Nanf32 //or Nanf64 for 64bit
} This is what IEEE and ISO/IEC suggest.
This doesn't 100% work. If I guess the proper way to catch both Quiet and Signal NAN, IEEE and ISO/IEC encode Quiet/Signal NAN's the same, but IDFK all platforms do. @petrochenkov example seems to imply that x86/x87/x64 treats anything with an 0xFF or 0x7FF exponent as a NaN which is non-standard... So a quick (untested) example: const F32_NAN: u32 = 0x7F800000u32;
const F32_NAN_MASK: u32 = 0x7FFFFFFFu32;
unsafe fn is_signal_nan(x: *const f32) -> bool {
let ptr: *const u32 = mem::transmute(x);
let mut val: u32 = read_volatile(ptr);
(val & F32_NAN_MASK) == F32_NAN
}
const F64NAN: 7FF0000000000000u64;
const F64_NAN_MASK: 7FFFFFFFFFFFFFFFu64;
unsafe fn is_signal_nan(x: *const f64) -> bool {
let ptr: *const u64 = mem::transmute(x);
let mut val: u64 = read_volatile(ptr);
(val & F64_NAN_MASK) == F64NAN
} The type cast/volatile read is to avoid the compiler/platform doing any magic even in This of course can still fail on some platforms is unsafe fn is_nan(x: f32) -> bool {
let val: u32 = mem::transmute(x);
....
} One can hit a register to register copies and incur the same magic. There is no idiomatic way to do this. I think the best approach is the stdlib will have to implement a solid attempt at NAN checking, and this function will grow and get more specialized as bug reports come in. Even well tested implementation will likely have a lot of holes in it as different processor models/modes will have different semantics. |
[Comment is moved to https://github.com/rust-lang/rust/pull/39271#issuecomment-277264729] |
Discussed during libs triage today our conclusions were:
|
src/libstd/f32.rs
Outdated
/// | ||
/// ``` | ||
/// #![feature(float_bits_conv)] | ||
/// assert!((1f32).to_bits() != 1f32 as u32); // to_bits() is not casting! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you can use assert_ne
here
Tidy is fixed now. Had to rebase. r? @BurntSushi |
@bors r=BurntSushi |
📌 Commit 0c14815 has been approved by |
⌛ Testing commit 0c14815 with merge 1ceb5ad... |
💔 Test failed - status-appveyor |
⌛ Testing commit 0c14815 with merge f55afe5... |
Add functions to safely transmute float to int The safe subset of Rust tries to be as powerful as possible. While it is very powerful already, its currently impossible to safely transmute integers to floats. While crates exist that provide a safe interface, most prominently the `iee754` crate (which also inspired naming of the added functions), they themselves only use the unsafe `mem::transmute` function to accomplish this task. Also, including an entire crate for just two lines of unsafe code seems quite wasteful. That's why this PR adds functions to safely transmute integers to floats and vice versa, currently gated by the newly added `float_bits_conv` feature. The functions added are no niche case. Not just `ieee754` [currently implements](https://github.com/huonw/ieee754/blob/master/src/lib.rs#L441) float to int transmutation via unsafe code but also the [very popular `byteorder` crate](https://github.com/BurntSushi/byteorder/blob/1.0.0/src/lib.rs#L258). This functionality of byteorder is in turn used by higher level crates. I only give two examples out of many: [chor](https://github.com/pyfisch/cbor/blob/a7363ea9aaf372e3d24b52414b5c76552ecc91c8/src/ser.rs#L227) and [bincode](https://github.com/TyOverby/bincode/blob/f06a4cfcb5b194e54d4997c200c75b88b6c3fba4/src/serde/reader.rs#L218). One alternative would be to manually use functions like pow or multiplication by 1 to get a similar result, but they only work in the int -> float direction, and are not bit exact, and much slower (also, most likely the optimizer will never optimize it to a transmute because the conversion is not bit exact while the transmute is). Tracking issue: #40470
💔 Test failed - status-travis |
@bors retry
|
Add functions to safely transmute float to int The safe subset of Rust tries to be as powerful as possible. While it is very powerful already, its currently impossible to safely transmute integers to floats. While crates exist that provide a safe interface, most prominently the `iee754` crate (which also inspired naming of the added functions), they themselves only use the unsafe `mem::transmute` function to accomplish this task. Also, including an entire crate for just two lines of unsafe code seems quite wasteful. That's why this PR adds functions to safely transmute integers to floats and vice versa, currently gated by the newly added `float_bits_conv` feature. The functions added are no niche case. Not just `ieee754` [currently implements](https://github.com/huonw/ieee754/blob/master/src/lib.rs#L441) float to int transmutation via unsafe code but also the [very popular `byteorder` crate](https://github.com/BurntSushi/byteorder/blob/1.0.0/src/lib.rs#L258). This functionality of byteorder is in turn used by higher level crates. I only give two examples out of many: [chor](https://github.com/pyfisch/cbor/blob/a7363ea9aaf372e3d24b52414b5c76552ecc91c8/src/ser.rs#L227) and [bincode](https://github.com/TyOverby/bincode/blob/f06a4cfcb5b194e54d4997c200c75b88b6c3fba4/src/serde/reader.rs#L218). One alternative would be to manually use functions like pow or multiplication by 1 to get a similar result, but they only work in the int -> float direction, and are not bit exact, and much slower (also, most likely the optimizer will never optimize it to a transmute because the conversion is not bit exact while the transmute is). Tracking issue: #40470
☀️ Test successful - status-appveyor, status-travis |
Stabilize float_bits_conv for Rust 1.21 Stabilizes the `float_bits_conv` lib feature for the 1.20 release of Rust. I've initially implemented the feature in #39271 and later made PR #43025 to output quiet NaNs even on platforms with different encodings, which seems to have been the only unresolved issue of the API. Due to PR #43025 being only applied to master this stabilisation can't happen for Rust 1.19 through the usual "stabilisation on beta" system that is being done for library APIs. r? @BurntSushi closes #40470.
The safe subset of Rust tries to be as powerful as possible. While it is very powerful already, its currently impossible to safely transmute integers to floats. While crates exist that provide a safe interface, most prominently the
iee754
crate (which also inspired naming of the added functions), they themselves only use the unsafemem::transmute
function to accomplish this task.Also, including an entire crate for just two lines of unsafe code seems quite wasteful.
That's why this PR adds functions to safely transmute integers to floats and vice versa, currently gated by the newly added
float_bits_conv
feature.The functions added are no niche case. Not just
ieee754
currently implements float to int transmutation via unsafe code but also the very popularbyteorder
crate. This functionality of byteorder is in turn used by higher level crates. I only give two examples out of many: chor and bincode.One alternative would be to manually use functions like pow or multiplication by 1 to get a similar result, but they only work in the int -> float direction, and are not bit exact, and much slower (also, most likely the optimizer will never optimize it to a transmute because the conversion is not bit exact while the transmute is).
Tracking issue: #40470