Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

where available use AtomicU{64,128} instead of mutex for Instant backsliding protection #83093

Merged
merged 6 commits into from
Aug 20, 2021

Conversation

the8472
Copy link
Member

@the8472 the8472 commented Mar 13, 2021

This decreases the overhead of backsliding protection on x86 systems with unreliable TSC, e.g. windows. And on aarch64 systems where 128bit atomics are available.

The following benchmarks were taken on x86_64 linux though by overriding actually_monotonic(), the numbers may look different on other platforms

# actually_monotonic() == true
test time::tests::instant_contention_01_threads                   ... bench:          44 ns/iter (+/- 0)
test time::tests::instant_contention_02_threads                   ... bench:          44 ns/iter (+/- 0)
test time::tests::instant_contention_04_threads                   ... bench:          44 ns/iter (+/- 0)
test time::tests::instant_contention_08_threads                   ... bench:          44 ns/iter (+/- 0)
test time::tests::instant_contention_16_threads                   ... bench:          44 ns/iter (+/- 0)

# 1x AtomicU64
test time::tests::instant_contention_01_threads                   ... bench:          65 ns/iter (+/- 0)
test time::tests::instant_contention_02_threads                   ... bench:         157 ns/iter (+/- 20)
test time::tests::instant_contention_04_threads                   ... bench:         281 ns/iter (+/- 53)
test time::tests::instant_contention_08_threads                   ... bench:         555 ns/iter (+/- 77)
test time::tests::instant_contention_16_threads                   ... bench:         883 ns/iter (+/- 107)

# mutex
test time::tests::instant_contention_01_threads                   ... bench:          60 ns/iter (+/- 2)
test time::tests::instant_contention_02_threads                   ... bench:         770 ns/iter (+/- 231)
test time::tests::instant_contention_04_threads                   ... bench:       1,347 ns/iter (+/- 45)
test time::tests::instant_contention_08_threads                   ... bench:       2,693 ns/iter (+/- 114)
test time::tests::instant_contention_16_threads                   ... bench:       5,244 ns/iter (+/- 487)

Since I don't have an arm machine with 128bit atomics I wasn't able to benchmark the AtomicU128 implementation.

@rustbot rustbot added S-blocked Status: Marked as blocked ❌ on something else such as an RFC or other implementation work. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Mar 13, 2021
@rust-highfive
Copy link
Collaborator

r? @sfackler

(rust-highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Mar 13, 2021
@the8472 the8472 marked this pull request as draft March 13, 2021 20:11
@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@joshtriplett
Copy link
Member

Very nice! Thank you for working on this.

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@the8472 the8472 changed the title x86: use cmpxchg16b instead of mutex for Instant backsliding protection where available use cmpxchg16b or AtomicU128 instead of mutex for Instant backsliding protection Mar 24, 2021
@rust-log-analyzer

This comment has been minimized.

@the8472 the8472 marked this pull request as ready for review April 14, 2021 20:06
@the8472
Copy link
Member Author

the8472 commented Apr 14, 2021

since #83776 landed is_x86_feature_detected! is now usable within std.

@rustbot label -S-blocked

@rustbot rustbot removed the S-blocked Status: Marked as blocked ❌ on something else such as an RFC or other implementation work. label Apr 14, 2021
@the8472
Copy link
Member Author

the8472 commented Apr 14, 2021

It should be ready for review now. I have only tested on x86_64 linux but touched code for other platforms too, so there's a chance that it'll break during a full build.

@jgarvin
Copy link

jgarvin commented Apr 15, 2021

Not sure if this is the right place to ask, but what does this protect against? Why is the tsc less reliable on Windows? I thought rdtsc behavior was only determined by the CPU, not the OS. Is the use of atomics just to facilitate only needing to store a single value regardless of number of threads, or is this somehow trying to make comparing Instants collected on different threads more safe?

@the8472
Copy link
Member Author

the8472 commented Apr 15, 2021

I thought rdtsc behavior was only determined by the CPU, not the OS.

Well, it's complicated. The OS plays some role, such as a synchronizing the counters on all cores. Additionally they might inject additional adjustments (e.g. NTP corrections) into some of their time APIs.
But the primary issue is which hardware time source the OS chooses to expose through its time APIs. Linux puts in quite some effort to ensure that TSC is only used when it is reliable and regularly receives updates that mark certain system configurations as unreliable and fall back to different time sources. Historically windows has been less reliable, hence the actually_monotonic() -> bool {false} implementation which results in the Instant being monotonized with a Mutex.

but what does this protect against?

Observed time going backwards when Instants obtained from different cores are compared despite sequential ordering between the Instants, either by happening on the same thread that gets moved between cores or via some synchronization establishing sequential ordering.

Is the use of atomics just to facilitate only needing to store a single value regardless of number of threads or is this somehow trying to make comparing Instants collected on different threads more safe?

The latter. The OS time source is still queried on each Instant::now() call. The obtained value is then checked against the static var and whichever is newer is returned and put back into the static. I.e. it guarantees monotonically increasing values between ordered instants.

There already exists a default implementation of this protection which is based on a Mutex. This PR just offers a faster alternative on platforms that have the appropriate atomics.

@bjorn3
Copy link
Member

bjorn3 commented Apr 15, 2021

but what does this protect against?

Observed time going backwards when Instants obtained from different cores are compared despite sequential ordering between the Instants, either by happening on the same thread that gets moved between cores or via some synchronization establishing sequential ordering.

To elaborate on this: The following would panic without backsliding protection if the time moves backwards:

let a = Instant::now();
// some work
let b = Instant::now();
let duration = b - a; // panic: supplied instant is later than self

This panic would need to happen as Duration can only store positive durations.

@ChrisDenton
Copy link
Member

But the primary issue is which hardware time source the OS chooses to expose through its time APIs. Linux puts in quite some effort to ensure that TSC is only used when it is reliable and regularly receives updates that mark certain system configurations as unreliable and fall back to different time sources. Historically windows has been less reliable, hence the actually_monotonic() -> bool {false} implementation which results in the Instant being monotonized with a Mutex.

Hm. Is this still an issue with Windows 7+? I'm reading Guidance for acquiring time stamps. The only outstanding issue it mentions is that multiple threads using QPC have ambiguous ordering if the counters only differ by ± 1 "tick".

@the8472
Copy link
Member Author

the8472 commented Apr 15, 2021

See #56560 for an example involving windows 10 in a virtualized environment.

@bors
Copy link
Contributor

bors commented Aug 20, 2021

📌 Commit 17ceb386ee21cfc76ecd00d567df96c06dddc0b8 has been approved by Amanieu

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 20, 2021
@the8472
Copy link
Member Author

the8472 commented Aug 20, 2021

Thanks, fixed. That the sys module and runner share a name but are different platforms is a bit confusing. I should have paid more attention to the module documentation.

@bors
Copy link
Contributor

bors commented Aug 20, 2021

⌛ Testing commit 17ceb386ee21cfc76ecd00d567df96c06dddc0b8 with merge 87078bd63304a678e0c178410426c1397940a408...

@rust-log-analyzer

This comment has been minimized.

@bors
Copy link
Contributor

bors commented Aug 20, 2021

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 20, 2021
@Amanieu
Copy link
Member

Amanieu commented Aug 20, 2021

@bors r+

@bors
Copy link
Contributor

bors commented Aug 20, 2021

📌 Commit cd82b42 has been approved by Amanieu

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 20, 2021
@bors
Copy link
Contributor

bors commented Aug 20, 2021

⌛ Testing commit cd82b42 with merge a003591...

@bors
Copy link
Contributor

bors commented Aug 20, 2021

☀️ Test successful - checks-actions
Approved by: Amanieu
Pushing a003591 to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Aug 20, 2021
@bors bors merged commit a003591 into rust-lang:master Aug 20, 2021
@rustbot rustbot added this to the 1.56.0 milestone Aug 20, 2021
JohnTitor added a commit to JohnTitor/rust that referenced this pull request Sep 19, 2021
…=kennytm

fix potential race in AtomicU64 time monotonizer

The AtomicU64-based monotonizer introduced in rust-lang#83093 is incorrect because several threads could try to update the value concurrently and a thread which doesn't have the newest value among all the updates could win.

That bug probably has little real world impact since it doesn't make observed time worse than hardware clocks. The worst case would probably be a thread which has a clock that is behind by several cycles observing several inconsistent fixups, which should be similar to observing the unfiltered backslide in the first place.

New benchmarks, they don't look as good as the original PR but still an improvement compared to the mutex.
I don't know why the contended mutex case is faster now than in the previous benchmarks.

```
actually_monotonic() == true:
test time::tests::instant_contention_01_threads                   ... bench:          44 ns/iter (+/- 0)
test time::tests::instant_contention_02_threads                   ... bench:          45 ns/iter (+/- 0)
test time::tests::instant_contention_04_threads                   ... bench:          45 ns/iter (+/- 0)
test time::tests::instant_contention_08_threads                   ... bench:          45 ns/iter (+/- 0)
test time::tests::instant_contention_16_threads                   ... bench:          46 ns/iter (+/- 0)

atomic u64:
test time::tests::instant_contention_01_threads                   ... bench:          66 ns/iter (+/- 0)
test time::tests::instant_contention_02_threads                   ... bench:         287 ns/iter (+/- 14)
test time::tests::instant_contention_04_threads                   ... bench:         296 ns/iter (+/- 43)
test time::tests::instant_contention_08_threads                   ... bench:         604 ns/iter (+/- 163)
test time::tests::instant_contention_16_threads                   ... bench:       1,147 ns/iter (+/- 29)

mutex:
test time::tests::instant_contention_01_threads                   ... bench:          78 ns/iter (+/- 0)
test time::tests::instant_contention_02_threads                   ... bench:         652 ns/iter (+/- 275)
test time::tests::instant_contention_04_threads                   ... bench:         900 ns/iter (+/- 32)
test time::tests::instant_contention_08_threads                   ... bench:       1,927 ns/iter (+/- 62)
test time::tests::instant_contention_16_threads                   ... bench:       3,748 ns/iter (+/- 146)
```
JohnTitor added a commit to JohnTitor/rust that referenced this pull request Sep 19, 2021
…=kennytm

fix potential race in AtomicU64 time monotonizer

The AtomicU64-based monotonizer introduced in rust-lang#83093 is incorrect because several threads could try to update the value concurrently and a thread which doesn't have the newest value among all the updates could win.

That bug probably has little real world impact since it doesn't make observed time worse than hardware clocks. The worst case would probably be a thread which has a clock that is behind by several cycles observing several inconsistent fixups, which should be similar to observing the unfiltered backslide in the first place.

New benchmarks, they don't look as good as the original PR but still an improvement compared to the mutex.
I don't know why the contended mutex case is faster now than in the previous benchmarks.

```
actually_monotonic() == true:
test time::tests::instant_contention_01_threads                   ... bench:          44 ns/iter (+/- 0)
test time::tests::instant_contention_02_threads                   ... bench:          45 ns/iter (+/- 0)
test time::tests::instant_contention_04_threads                   ... bench:          45 ns/iter (+/- 0)
test time::tests::instant_contention_08_threads                   ... bench:          45 ns/iter (+/- 0)
test time::tests::instant_contention_16_threads                   ... bench:          46 ns/iter (+/- 0)

atomic u64:
test time::tests::instant_contention_01_threads                   ... bench:          66 ns/iter (+/- 0)
test time::tests::instant_contention_02_threads                   ... bench:         287 ns/iter (+/- 14)
test time::tests::instant_contention_04_threads                   ... bench:         296 ns/iter (+/- 43)
test time::tests::instant_contention_08_threads                   ... bench:         604 ns/iter (+/- 163)
test time::tests::instant_contention_16_threads                   ... bench:       1,147 ns/iter (+/- 29)

mutex:
test time::tests::instant_contention_01_threads                   ... bench:          78 ns/iter (+/- 0)
test time::tests::instant_contention_02_threads                   ... bench:         652 ns/iter (+/- 275)
test time::tests::instant_contention_04_threads                   ... bench:         900 ns/iter (+/- 32)
test time::tests::instant_contention_08_threads                   ... bench:       1,927 ns/iter (+/- 62)
test time::tests::instant_contention_16_threads                   ... bench:       3,748 ns/iter (+/- 146)
```
wip-sync pushed a commit to NetBSD/pkgsrc-wip that referenced this pull request Oct 22, 2021
Pkgsrc changes:
 * Remove one now-longer-applicable patch, adjust a few others
 * Bump bootstrap requirements to 1.55.0.

Upstream changes:

Version 1.56.0 (2021-10-21)
========================

Language
--------

- [The 2021 Edition is now stable.][rust#88100]
  See [the edition guide][rust-2021-edition-guide] for more details.
- [The pattern in `binding @ pattern` can now also introduce new bindings.]
  [rust#85305]
- [Union field access is permitted in `const fn`.][rust#85769]

[rust-2021-edition-guide]: https://doc.rust-lang.org/nightly/edition-guide/rust-2021/index.html

Compiler
--------

- [Upgrade to LLVM 13.][rust#87570]
- [Support memory, address, and thread sanitizers on
  aarch64-unknown-freebsd.][rust#88023]
- [Allow specifying a deployment target version for all iOS targets][rust#87699]
- [Warnings can be forced on with `--force-warn`.][rust#87472]
  This feature is primarily intended for usage by `cargo fix`,
  rather than end users.
- [Promote `aarch64-apple-ios-sim` to Tier 2\*.][rust#87760]
- [Add `powerpc-unknown-freebsd` at Tier 3\*.][rust#87370]
- [Add `riscv32imc-esp-espidf` at Tier 3\*.][rust#87666]

\* Refer to Rust's [platform support page][platform-support-doc] for more
information on Rust's tiered platform support.

Libraries
---------

- [Allow writing of incomplete UTF-8 sequences via stdout/stderr on Windows.]
  [rust#83342]
  The Windows console still requires valid Unicode, but this change allows
  splitting a UTF-8 character across multiple write calls. This allows, for
  instance, programs that just read and write data buffers (e.g. copying a file
  to stdout) without regard for Unicode or character boundaries.
- [Prefer `AtomicU{64,128}` over Mutex for Instant backsliding protection.]
  [rust#83093]
  For this use case, atomics scale much better under contention.
- [Implement `Extend<(A, B)>` for `(Extend<A>, Extend<B>)`][rust#85835]
- [impl Default, Copy, Clone for std::io::Sink and std::io::Empty][rust#86744]
- [`impl From<[(K, V); N]>` for all collections.][rust#84111]
- [Remove `P: Unpin` bound on impl Future for Pin.][rust#81363]
- [Treat invalid environment variable names as non-existent.][rust#86183]
  Previously, the environment functions would panic if given a
  variable name with an internal null character or equal sign (`=`).
  Now, these functions will just treat such names as non-existent
  variables, since the OS cannot represent the existence of a
  variable with such a name.

Stabilised APIs
---------------

- [`std::os::unix::fs::chroot`]
- [`UnsafeCell::raw_get`]
- [`BufWriter::into_parts`]
- [`core::panic::{UnwindSafe, RefUnwindSafe, AssertUnwindSafe}`]
  These APIs were previously stable in `std`, but are now also available
  in `core`.
- [`Vec::shrink_to`]
- [`String::shrink_to`]
- [`OsString::shrink_to`]
- [`PathBuf::shrink_to`]
- [`BinaryHeap::shrink_to`]
- [`VecDeque::shrink_to`]
- [`HashMap::shrink_to`]
- [`HashSet::shrink_to`]

These APIs are now usable in const contexts:

- [`std::mem::transmute`]
- [`[T]::first`][`slice::first`]
- [`[T]::split_first`][`slice::split_first`]
- [`[T]::last`][`slice::last`]
- [`[T]::split_last`][`slice::split_last`]

Cargo
-----

- [Cargo supports specifying a minimum supported Rust version in Cargo.toml.]
  [`rust-version`] This has no effect at present on dependency
  version selection.  We encourage crates to specify their minimum
  supported Rust version, and we encourage CI systems that support
  Rust code to include a crate's specified minimum version in the
  text matrix for that crate by default.

Compatibility notes
-------------------

- [Update to new argument parsing rules on Windows.][rust#87580]
  This adjusts Rust's standard library to match the behavior of the standard
  libraries for C/C++. The rules have changed slightly over time, and this PR
  brings us to the latest set of rules (changed in 2008).
- [Disallow the aapcs calling convention on aarch64][rust#88399]
  This was already not supported by LLVM; this change surfaces this lack of
  support with a better error message.
- [Make `SEMICOLON_IN_EXPRESSIONS_FROM_MACROS` warn by default][rust#87385]
- [Warn when an escaped newline skips multiple lines.][rust#87671]
- [Calls to `libc::getpid` / `std::process::id` from `Command::pre_exec`
  may return different values on glibc <= 2.24.][rust#81825] Rust
  now invokes the `clone3` system call directly, when available,
  to use new functionality available via that system call. Older
  versions of glibc cache the result of `getpid`, and only update
  that cache when calling glibc's clone/fork functions, so a direct
  system call bypasses that cache update. glibc 2.25 and newer no
  longer cache `getpid` for exactly this reason.

Internal changes
----------------
These changes provide no direct user facing benefits, but represent
significant improvements to the internals and overall performance
of rustc and related tools.

- [LLVM is compiled with PGO in published x86_64-unknown-linux-gnu
  artifacts.][rust#88069] This improves the performance of most Rust builds.

- [Unify representation of macros in internal data structures.][rust#88019]
  This change fixes a host of bugs with the handling of macros by the compiler,
  as well as rustdoc.

[`std::os::unix::fs::chroot`]: https://doc.rust-lang.org/stable/std/os/unix/fs/fn.chroot.html
[`Iterator::intersperse`]: https://doc.rust-lang.org/stable/std/iter/trait.Iterator.html#method.intersperse
[`Iterator::intersperse_with`]: https://doc.rust-lang.org/stable/std/iter/trait.Iterator.html#method.intersperse
[`UnsafeCell::raw_get`]: https://doc.rust-lang.org/stable/std/cell/struct.UnsafeCell.html#method.raw_get
[`BufWriter::into_parts`]: https://doc.rust-lang.org/stable/std/io/struct.BufWriter.html#method.into_parts
[`core::panic::{UnwindSafe, RefUnwindSafe, AssertUnwindSafe}`]: rust-lang/rust#84662
[`Vec::shrink_to`]: https://doc.rust-lang.org/stable/std/vec/struct.Vec.html#method.shrink_to
[`String::shrink_to`]: https://doc.rust-lang.org/stable/std/string/struct.String.html#method.shrink_to
[`OsString::shrink_to`]: https://doc.rust-lang.org/stable/std/ffi/struct.OsString.html#method.shrink_to
[`PathBuf::shrink_to`]: https://doc.rust-lang.org/stable/std/path/struct.PathBuf.html#method.shrink_to
[`BinaryHeap::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/struct.BinaryHeap.html#method.shrink_to
[`VecDeque::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/struct.VecDeque.html#method.shrink_to
[`HashMap::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/hash_map/struct.HashMap.html#method.shrink_to
[`HashSet::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/hash_set/struct.HashSet.html#method.shrink_to
[`std::mem::transmute`]: https://doc.rust-lang.org/stable/std/mem/fn.transmute.html
[`slice::first`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.first
[`slice::split_first`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_first
[`slice::last`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.last
[`slice::split_last`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_last
[`rust-version`]: https://doc.rust-lang.org/nightly/cargo/reference/manifest.html#the-rust-version-field
[rust#87671]: rust-lang/rust#87671
[rust#86183]: rust-lang/rust#86183
[rust#87385]: rust-lang/rust#87385
[rust#88100]: rust-lang/rust#88100
[rust#86860]: rust-lang/rust#86860
[rust#84039]: rust-lang/rust#84039
[rust#86492]: rust-lang/rust#86492
[rust#88363]: rust-lang/rust#88363
[rust#85305]: rust-lang/rust#85305
[rust#87832]: rust-lang/rust#87832
[rust#88069]: rust-lang/rust#88069
[rust#87472]: rust-lang/rust#87472
[rust#87699]: rust-lang/rust#87699
[rust#87570]: rust-lang/rust#87570
[rust#88023]: rust-lang/rust#88023
[rust#87760]: rust-lang/rust#87760
[rust#87370]: rust-lang/rust#87370
[rust#87580]: rust-lang/rust#87580
[rust#83342]: rust-lang/rust#83342
[rust#83093]: rust-lang/rust#83093
[rust#88177]: rust-lang/rust#88177
[rust#88548]: rust-lang/rust#88548
[rust#88551]: rust-lang/rust#88551
[rust#88299]: rust-lang/rust#88299
[rust#88220]: rust-lang/rust#88220
[rust#85835]: rust-lang/rust#85835
[rust#86879]: rust-lang/rust#86879
[rust#86744]: rust-lang/rust#86744
[rust#84662]: rust-lang/rust#84662
[rust#86593]: rust-lang/rust#86593
[rust#81050]: rust-lang/rust#81050
[rust#81363]: rust-lang/rust#81363
[rust#84111]: rust-lang/rust#84111
[rust#85769]: rust-lang/rust#85769 (comment)
[rust#88490]: rust-lang/rust#88490
[rust#88269]: rust-lang/rust#88269
[rust#84176]: rust-lang/rust#84176
[rust#88399]: rust-lang/rust#88399
[rust#88227]: rust-lang/rust#88227
[rust#88200]: rust-lang/rust#88200
[rust#82776]: rust-lang/rust#82776
[rust#88077]: rust-lang/rust#88077
[rust#87728]: rust-lang/rust#87728
[rust#87050]: rust-lang/rust#87050
[rust#87619]: rust-lang/rust#87619
[rust#81825]: rust-lang/rust#81825 (comment)
[rust#88019]: rust-lang/rust#88019
[rust#87666]: rust-lang/rust#87666
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request Nov 20, 2021
Pkgsrc changes:
 * Bump bootstrap kit version to 1.55.0.
 * Adjust patches as needed, some no longer apply (so removed)
 * Update checksum adjustments.
 * Avoid rust-llvm on SunOS
 * Optionally build docs
 * Remove reference to closed/old PR#54621

Upstream changes:

Version 1.56.1 (2021-11-01)
===========================

- New lints to detect the presence of bidirectional-override Unicode
  codepoints in the compiled source code ([CVE-2021-42574])

[CVE-2021-42574]: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-42574

Version 1.56.0 (2021-10-21)
========================

Language
--------

- [The 2021 Edition is now stable.][rust#88100]
  See [the edition guide][rust-2021-edition-guide] for more details.
- [The pattern in `binding @ pattern` can now also introduce new bindings.]
  [rust#85305]
- [Union field access is permitted in `const fn`.][rust#85769]

[rust-2021-edition-guide]:
  https://doc.rust-lang.org/nightly/edition-guide/rust-2021/index.html

Compiler
--------

- [Upgrade to LLVM 13.][rust#87570]
- [Support memory, address, and thread sanitizers on aarch64-unknown-freebsd.]
  [rust#88023]
- [Allow specifying a deployment target version for all iOS targets][rust#87699]
- [Warnings can be forced on with `--force-warn`.][rust#87472]
  This feature is primarily intended for usage by `cargo fix`, rather than
  end users.
- [Promote `aarch64-apple-ios-sim` to Tier 2\*.][rust#87760]
- [Add `powerpc-unknown-freebsd` at Tier 3\*.][rust#87370]
- [Add `riscv32imc-esp-espidf` at Tier 3\*.][rust#87666]

\* Refer to Rust's [platform support page][platform-support-doc] for more
information on Rust's tiered platform support.

Libraries
---------

- [Allow writing of incomplete UTF-8 sequences via stdout/stderr on Windows.]
  [rust#83342]
  The Windows console still requires valid Unicode, but this change allows
  splitting a UTF-8 character across multiple write calls. This allows, for
  instance, programs that just read and write data buffers (e.g. copying a file
  to stdout) without regard for Unicode or character boundaries.
- [Prefer `AtomicU{64,128}` over Mutex for Instant backsliding protection.]
  [rust#83093]
  For this use case, atomics scale much better under contention.
- [Implement `Extend<(A, B)>` for `(Extend<A>, Extend<B>)`][rust#85835]
- [impl Default, Copy, Clone for std::io::Sink and std::io::Empty][rust#86744]
- [`impl From<[(K, V); N]>` for all collections.][rust#84111]
- [Remove `P: Unpin` bound on impl Future for Pin.][rust#81363]
- [Treat invalid environment variable names as non-existent.][rust#86183]
  Previously, the environment functions would panic if given a
  variable name with an internal null character or equal sign (`=`).
  Now, these functions will just treat such names as non-existent
  variables, since the OS cannot represent the existence of a
  variable with such a name.

Stabilised APIs
---------------

- [`std::os::unix::fs::chroot`]
- [`UnsafeCell::raw_get`]
- [`BufWriter::into_parts`]
- [`core::panic::{UnwindSafe, RefUnwindSafe, AssertUnwindSafe}`]
  These APIs were previously stable in `std`, but are now also available
  in `core`.
- [`Vec::shrink_to`]
- [`String::shrink_to`]
- [`OsString::shrink_to`]
- [`PathBuf::shrink_to`]
- [`BinaryHeap::shrink_to`]
- [`VecDeque::shrink_to`]
- [`HashMap::shrink_to`]
- [`HashSet::shrink_to`]

These APIs are now usable in const contexts:

- [`std::mem::transmute`]
- [`[T]::first`][`slice::first`]
- [`[T]::split_first`][`slice::split_first`]
- [`[T]::last`][`slice::last`]
- [`[T]::split_last`][`slice::split_last`]

Cargo
-----

- [Cargo supports specifying a minimum supported Rust version in Cargo.toml.]
  [`rust-version`]
  This has no effect at present on dependency version selection.
  We encourage crates to specify their minimum supported Rust
  version, and we encourage CI systems that support Rust code to
  include a crate's specified minimum version in the text matrix
  for that crate by default.

Compatibility notes
-------------------

- [Update to new argument parsing rules on Windows.][rust#87580]
  This adjusts Rust's standard library to match the behavior of the standard
  libraries for C/C++. The rules have changed slightly over time, and this PR
  brings us to the latest set of rules (changed in 2008).
- [Disallow the aapcs calling convention on aarch64][rust#88399]
  This was already not supported by LLVM; this change surfaces this lack of
  support with a better error message.
- [Make `SEMICOLON_IN_EXPRESSIONS_FROM_MACROS` warn by default][rust#87385]
- [Warn when an escaped newline skips multiple lines.][rust#87671]
- [Calls to `libc::getpid` / `std::process::id` from `Command::pre_exec`
  may return different values on glibc <= 2.24.][rust#81825]
  Rust now invokes the `clone3` system call directly, when available,
  to use new functionality available via that system call. Older
  versions of glibc cache the result of `getpid`, and only update
  that cache when calling glibc's clone/fork functions, so a direct
  system call bypasses that cache update. glibc 2.25 and newer no
  longer cache `getpid` for exactly this reason.

Internal changes
----------------
These changes provide no direct user facing benefits, but represent
significant improvements to the internals and overall performance
of rustc and related tools.

- [LLVM is compiled with PGO in published x86_64-unknown-linux-gnu artifacts.]
  [rust#88069]
  This improves the performance of most Rust builds.
- [Unify representation of macros in internal data structures.][rust#88019]
  This change fixes a host of bugs with the handling of macros by the compiler,
  as well as rustdoc.

[`std::os::unix::fs::chroot`]: https://doc.rust-lang.org/stable/std/os/unix/fs/fn.chroot.html
[`Iterator::intersperse`]: https://doc.rust-lang.org/stable/std/iter/trait.Iterator.html#method.intersperse
[`Iterator::intersperse_with`]: https://doc.rust-lang.org/stable/std/iter/trait.Iterator.html#method.intersperse
[`UnsafeCell::raw_get`]: https://doc.rust-lang.org/stable/std/cell/struct.UnsafeCell.html#method.raw_get
[`BufWriter::into_parts`]: https://doc.rust-lang.org/stable/std/io/struct.BufWriter.html#method.into_parts
[`core::panic::{UnwindSafe, RefUnwindSafe, AssertUnwindSafe}`]: rust-lang/rust#84662
[`Vec::shrink_to`]: https://doc.rust-lang.org/stable/std/vec/struct.Vec.html#method.shrink_to
[`String::shrink_to`]: https://doc.rust-lang.org/stable/std/string/struct.String.html#method.shrink_to
[`OsString::shrink_to`]: https://doc.rust-lang.org/stable/std/ffi/struct.OsString.html#method.shrink_to
[`PathBuf::shrink_to`]: https://doc.rust-lang.org/stable/std/path/struct.PathBuf.html#method.shrink_to
[`BinaryHeap::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/struct.BinaryHeap.html#method.shrink_to
[`VecDeque::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/struct.VecDeque.html#method.shrink_to
[`HashMap::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/hash_map/struct.HashMap.html#method.shrink_to
[`HashSet::shrink_to`]: https://doc.rust-lang.org/stable/std/collections/hash_set/struct.HashSet.html#method.shrink_to
[`std::mem::transmute`]: https://doc.rust-lang.org/stable/std/mem/fn.transmute.html
[`slice::first`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.first
[`slice::split_first`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_first
[`slice::last`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.last
[`slice::split_last`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.split_last
[`rust-version`]: https://doc.rust-lang.org/nightly/cargo/reference/manifest.html#the-rust-version-field
[rust#87671]: rust-lang/rust#87671
[rust#86183]: rust-lang/rust#86183
[rust#87385]: rust-lang/rust#87385
[rust#88100]: rust-lang/rust#88100
[rust#86860]: rust-lang/rust#86860
[rust#84039]: rust-lang/rust#84039
[rust#86492]: rust-lang/rust#86492
[rust#88363]: rust-lang/rust#88363
[rust#85305]: rust-lang/rust#85305
[rust#87832]: rust-lang/rust#87832
[rust#88069]: rust-lang/rust#88069
[rust#87472]: rust-lang/rust#87472
[rust#87699]: rust-lang/rust#87699
[rust#87570]: rust-lang/rust#87570
[rust#88023]: rust-lang/rust#88023
[rust#87760]: rust-lang/rust#87760
[rust#87370]: rust-lang/rust#87370
[rust#87580]: rust-lang/rust#87580
[rust#83342]: rust-lang/rust#83342
[rust#83093]: rust-lang/rust#83093
[rust#88177]: rust-lang/rust#88177
[rust#88548]: rust-lang/rust#88548
[rust#88551]: rust-lang/rust#88551
[rust#88299]: rust-lang/rust#88299
[rust#88220]: rust-lang/rust#88220
[rust#85835]: rust-lang/rust#85835
[rust#86879]: rust-lang/rust#86879
[rust#86744]: rust-lang/rust#86744
[rust#84662]: rust-lang/rust#84662
[rust#86593]: rust-lang/rust#86593
[rust#81050]: rust-lang/rust#81050
[rust#81363]: rust-lang/rust#81363
[rust#84111]: rust-lang/rust#84111
[rust#85769]: rust-lang/rust#85769 (comment)
[rust#88490]: rust-lang/rust#88490
[rust#88269]: rust-lang/rust#88269
[rust#84176]: rust-lang/rust#84176
[rust#88399]: rust-lang/rust#88399
[rust#88227]: rust-lang/rust#88227
[rust#88200]: rust-lang/rust#88200
[rust#82776]: rust-lang/rust#82776
[rust#88077]: rust-lang/rust#88077
[rust#87728]: rust-lang/rust#87728
[rust#87050]: rust-lang/rust#87050
[rust#87619]: rust-lang/rust#87619
[rust#81825]: rust-lang/rust#81825 (comment)
[rust#88019]: rust-lang/rust#88019
[rust#87666]: rust-lang/rust#87666

Version 1.55.0 (2021-09-09)
============================

Language
--------
- [You can now write open "from" range patterns (`X..`), which will start
  at `X` and will end at the maximum value of the integer.][83918]
- [You can now explicitly import the prelude of different editions
  through `std::prelude` (e.g. `use std::prelude::rust_2021::*;`).][86294]

Compiler
--------
- [Added tier 3\* support for `powerpc64le-unknown-freebsd`.][83572]

\* Refer to Rust's [platform support page][platform-support-doc] for more
   information on Rust's tiered platform support.

Libraries
---------

- [Updated std's float parsing to use the Eisel-Lemire algorithm.][86761]
  These improvements should in general provide faster string parsing of floats,
  no longer reject certain valid floating point values, and reduce
  the produced code size for non-stripped artifacts.
- [`string::Drain` now implements `AsRef<str>` and `AsRef<[u8]>`.][86858]

Stabilised APIs
---------------

- [`Bound::cloned`]
- [`Drain::as_str`]
- [`IntoInnerError::into_error`]
- [`IntoInnerError::into_parts`]
- [`MaybeUninit::assume_init_mut`]
- [`MaybeUninit::assume_init_ref`]
- [`MaybeUninit::write`]
- [`array::map`]
- [`ops::ControlFlow`]
- [`x86::_bittest`]
- [`x86::_bittestandcomplement`]
- [`x86::_bittestandreset`]
- [`x86::_bittestandset`]
- [`x86_64::_bittest64`]
- [`x86_64::_bittestandcomplement64`]
- [`x86_64::_bittestandreset64`]
- [`x86_64::_bittestandset64`]

The following previously stable functions are now `const`.

- [`str::from_utf8_unchecked`]


Cargo
-----
- [Cargo will now deduplicate compiler diagnostics to the terminal when invoking
  rustc in parallel such as when using `cargo test`.][cargo/9675]
- [The package definition in `cargo metadata` now includes the `"default_run"`
  field from the manifest.][cargo/9550]
- [Added `cargo d` as an alias for `cargo doc`.][cargo/9680]
- [Added `{lib}` as formatting option for `cargo tree` to print the `"lib_name"`
  of packages.][cargo/9663]

Rustdoc
-------
- [Added "Go to item on exact match" search option.][85876]
- [The "Implementors" section on traits no longer shows redundant
  method definitions.][85970]
- [Trait implementations are toggled open by default.][86260] This should
  make the implementations more searchable by tools like `CTRL+F` in
  your browser.
- [Intra-doc links should now correctly resolve associated items (e.g. methods)
  through type aliases.][86334]
- [Traits which are marked with `#[doc(hidden)]` will no longer appear in the
  "Trait Implementations" section.][86513]


Compatibility Notes
-------------------
- [std functions that return an `io::Error` will no longer use the
  `ErrorKind::Other` variant.][85746] This is to better reflect that these
  kinds of errors could be categorised [into newer more specific `ErrorKind`
  variants][79965], and that they do not represent a user error.
- [Using environment variable names with `process::Command` on Windows now
  behaves as expected.][85270] Previously using envionment variables with
  `Command` would cause them to be ASCII-uppercased.
- [Rustdoc will now warn on using rustdoc lints that aren't prefixed
  with `rustdoc::`][86849]

[86849]: rust-lang/rust#86849
[86513]: rust-lang/rust#86513
[86334]: rust-lang/rust#86334
[86260]: rust-lang/rust#86260
[85970]: rust-lang/rust#85970
[85876]: rust-lang/rust#85876
[83572]: rust-lang/rust#83572
[86294]: rust-lang/rust#86294
[86858]: rust-lang/rust#86858
[86761]: rust-lang/rust#86761
[85769]: rust-lang/rust#85769
[85746]: rust-lang/rust#85746
[85305]: rust-lang/rust#85305
[85270]: rust-lang/rust#85270
[84111]: rust-lang/rust#84111
[83918]: rust-lang/rust#83918
[79965]: rust-lang/rust#79965
[87370]: rust-lang/rust#87370
[87298]: rust-lang/rust#87298
[cargo/9663]: rust-lang/cargo#9663
[cargo/9675]: rust-lang/cargo#9675
[cargo/9550]: rust-lang/cargo#9550
[cargo/9680]: rust-lang/cargo#9680
[cargo/9663]: rust-lang/cargo#9663
[`array::map`]: https://doc.rust-lang.org/stable/std/primitive.array.html#method.map
[`Bound::cloned`]: https://doc.rust-lang.org/stable/std/ops/enum.Bound.html#method.cloned
[`Drain::as_str`]: https://doc.rust-lang.org/stable/std/string/struct.Drain.html#method.as_str
[`IntoInnerError::into_error`]: https://doc.rust-lang.org/stable/std/io/struct.IntoInnerError.html#method.into_error
[`IntoInnerError::into_parts`]: https://doc.rust-lang.org/stable/std/io/struct.IntoInnerError.html#method.into_parts
[`MaybeUninit::assume_init_mut`]: https://doc.rust-lang.org/stable/std/mem/union.MaybeUninit.html#method.assume_init_mut
[`MaybeUninit::assume_init_ref`]: https://doc.rust-lang.org/stable/std/mem/union.MaybeUninit.html#method.assume_init_ref
[`MaybeUninit::write`]: https://doc.rust-lang.org/stable/std/mem/union.MaybeUninit.html#method.write
[`Seek::rewind`]: https://doc.rust-lang.org/stable/std/io/trait.Seek.html#method.rewind
[`ops::ControlFlow`]: https://doc.rust-lang.org/stable/std/ops/enum.ControlFlow.html
[`str::from_utf8_unchecked`]: https://doc.rust-lang.org/stable/std/str/fn.from_utf8_unchecked.html
[`x86::_bittest`]: https://doc.rust-lang.org/stable/core/arch/x86/fn._bittest.html
[`x86::_bittestandcomplement`]: https://doc.rust-lang.org/stable/core/arch/x86/fn._bittestandcomplement.html
[`x86::_bittestandreset`]: https://doc.rust-lang.org/stable/core/arch/x86/fn._bittestandreset.html
[`x86::_bittestandset`]: https://doc.rust-lang.org/stable/core/arch/x86/fn._bittestandset.html
[`x86_64::_bittest64`]: https://doc.rust-lang.org/stable/core/arch/x86_64/fn._bittest64.html
[`x86_64::_bittestandcomplement64`]: https://doc.rust-lang.org/stable/core/arch/x86_64/fn._bittestandcomplement64.html
[`x86_64::_bittestandreset64`]: https://doc.rust-lang.org/stable/core/arch/x86_64/fn._bittestandreset64.html
[`x86_64::_bittestandset64`]: https://doc.rust-lang.org/stable/core/arch/x86_64/fn._bittestandset64.html
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Feb 13, 2022
…mulacrum

make `Instant::{duration_since, elapsed, sub}` saturating and remove workarounds

This removes all mutex/atomic-based workarounds for non-monotonic clocks and makes the previously panicking methods saturating instead. Additionally `saturating_duration_since` becomes deprecated since `duration_since` now fills that role.

Effectively this moves the fixup from `Instant` construction to the comparisons.

This has some observable effects, especially on platforms without monotonic clocks:

* Incorrectly ordered Instant comparisons no longer panic in release mode. This could hide some programming errors, but since debug mode still panics tests can still catch them.
* `checked_duration_since` will now return `None` in more cases. Previously it only happened when one compared instants obtained in the wrong order or manually created ones. Now it also does on backslides.
* non-monotonic intervals will not be transitive, i.e. `b.duration_since(a) + c.duration_since(b) != c.duration_since(a)`

The upsides are reduced complexity and lower overhead of `Instant::now`.

## Motivation

Currently we must choose between two poisons. One is high worst-case latency and jitter of `Instant::now()` due to explicit synchronization; see rust-lang#83093 for benchmarks, the worst-case overhead is > 100x. The other is sporadic panics on specific, rare combinations of CPU/hypervisor/operating system due to platform bugs.

Use-cases where low-overhead, fine-grained timestamps are needed - such as syscall tracing, performance profiles or sensor data acquisition (drone flight controllers were mentioned in a libs meeting) in multi-threaded programs - are negatively impacted by the synchronization.

The panics are user-visible (program crashes), hard to reproduce and can be triggered by any dependency that might be using Instants for any reason.

A solution that is fast _and_ doesn't panic is desirable.

----

closes rust-lang#84448
closes rust-lang#86470
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-time Area: Time merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.