Skip to content

optimize slice::Iter::fold #106343

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jun 15, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions library/core/benches/slice.rs
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
use core::ptr::NonNull;
use test::black_box;
use test::Bencher;

Expand Down Expand Up @@ -162,3 +163,11 @@ fn fill_byte_sized(b: &mut Bencher) {
black_box(slice.fill(black_box(NewType(42))));
});
}

// Tests the ability of the compiler to recognize that only the last slice item is needed
// based on issue #106288
#[bench]
fn fold_to_last(b: &mut Bencher) {
let slice: &[i32] = &[0; 1024];
b.iter(|| black_box(slice).iter().fold(None, |_, r| Some(NonNull::from(r))));
}
33 changes: 33 additions & 0 deletions library/core/src/slice/iter/macros.rs
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,39 @@ macro_rules! iterator {
self.next_back()
}

#[inline]
fn fold<B, F>(self, init: B, mut f: F) -> B
where
F: FnMut(B, Self::Item) -> B,
{
// this implementation consists of the following optimizations compared to the
// default implementation:
// - do-while loop, as is llvm's preferred loop shape,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's very weird to me that doing this manually is needed, since LLVM has a loop rotation pass to do this. But I guess it's fine.

// see https://releases.llvm.org/16.0.0/docs/LoopTerminology.html#more-canonical-loops
// - bumps an index instead of a pointer since the latter case inhibits
// some optimizations, see #111603
// - avoids Option wrapping/matching
if is_empty!(self) {
return init;
}
let mut acc = init;
let mut i = 0;
let len = len!(self);
loop {
// SAFETY: the loop iterates `i in 0..len`, which always is in bounds of
// the slice allocation
acc = f(acc, unsafe { & $( $mut_ )? *self.ptr.add(i).as_ptr() });
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How much of the duplication here is essential?

Avoiding the Option loop seems entirely reasonable to me, but how does this approach compare to something much shorter? For example, something like

for _ in 0..len!(self) {
    acc = f(acc, next_unchecked!(self));
}

Or the same with a while (n --> 0) loop or something, if the Range is not ok.

And if that's not sufficient, it would be nice to have some comments here about why it's written this way.

(Also, if overloading fold is worth it, I expect rfold should be overridden too.)

@rustbot author

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you look at the PR history you'll see that I tried several approaches. A for-in-range loop was one of them. I even used IndexRange instead of Range.

And if that's not sufficient, it would be nice to have some comments here about why it's written this way.

Ok, will do.

(Also, if overloading fold is worth it, I expect rfold should be overridden too.)

Maybe, but that'd be optimizing two things at once which makes assessing perf more complicated.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment.

@rustbot ready

// SAFETY: `i` can't overflow since it'll only reach usize::MAX if the
// slice had that length, in which case we'll break out of the loop
// after the increment
i = unsafe { i.unchecked_add(1) };
if i == len {
break;
}
}
acc
}

// We override the default implementation, which uses `try_fold`,
// because this simple implementation generates less LLVM IR and is
// faster to compile.
Expand Down
14 changes: 14 additions & 0 deletions tests/codegen/slice-iter-fold.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
// ignore-debug: the debug assertions get in the way
// compile-flags: -O
// min-llvm-version: 16
#![crate_type = "lib"]

// CHECK-LABEL: @slice_fold_to_last
#[no_mangle]
pub fn slice_fold_to_last(slice: &[i32]) -> Option<&i32> {
// CHECK-NOT: loop
// CHECK-NOT: br
// CHECK-NOT: call
// CHECK: ret
slice.iter().fold(None, |_, i| Some(i))
}
8 changes: 0 additions & 8 deletions tests/codegen/vec-shrink-panik.rs
Original file line number Diff line number Diff line change
Expand Up @@ -37,14 +37,6 @@ pub fn issue71861(vec: Vec<u32>) -> Box<[u32]> {
// CHECK-LABEL: @issue75636
#[no_mangle]
pub fn issue75636<'a>(iter: &[&'a str]) -> Box<[&'a str]> {
// CHECK-NOT: panic

// Call to panic_cannot_unwind in case of double-panic is expected,
// on LLVM 16 and older, but other panics are not.
// old: filter
// old-NEXT: ; call core::panicking::panic_cannot_unwind
// old-NEXT: panic_cannot_unwind

Comment on lines -40 to -47
Copy link
Contributor

@klensy klensy Jun 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now there panic? Or why this was killed.
Ohh, i see it lower, my bad.

// CHECK-NOT: panic
iter.iter().copied().collect()
}
Expand Down