Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Vec::retain leaks items when predicate panics #52267

Closed
stephaneyfx opened this issue Jul 11, 2018 · 5 comments
Closed

Vec::retain leaks items when predicate panics #52267

stephaneyfx opened this issue Jul 11, 2018 · 5 comments
Labels
A-collections Area: `std::collection` C-bug Category: This is a bug. regression-from-stable-to-stable Performance or correctness regression from one stable version to another. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.

Comments

@stephaneyfx
Copy link
Contributor

stephaneyfx commented Jul 11, 2018

Since the implementation of Vec::retain changed after Rust 1.25, items may not be dropped when the predicate panics.

Reproduction example

#[derive(Debug)]
struct Foo(i32);

impl Drop for Foo {
    fn drop(&mut self) {
        println!("Dropping Foo({})", self.0);
    }
}

fn main() {
    let mut v = (0..3).map(Foo).collect::<Vec<_>>();
    println!("Before retain");
    v.retain(|&Foo(n)| match n {
        0 => false,
        2 => panic!("Boo"),
        _ => true,
    });
    println!("After retain");
}

Expected result

All 3 Foo instances are dropped. The output should look like (with Rust 1.25):

Before retain
thread 'main' panicked at 'Boo', a.rs:15:14
note: Run with `RUST_BACKTRACE=1` for a backtrace.
Dropping Foo(1)
Dropping Foo(0)
Dropping Foo(2)

Actual result

Only one Foo instance is dropped. The output since Rust 1.26 (and with 1.27.1) is:

Before retain
Dropping Foo(0)
thread 'main' panicked at 'Boo', a.rs:15:14
note: Run with `RUST_BACKTRACE=1` for a backtrace.

Edited to clarify item leak (i.e. not dropped).

@stephaneyfx stephaneyfx changed the title Vec::retain leaks memory when predicate panics Vec::retain leaks items when predicate panics Jul 11, 2018
@Mark-Simulacrum Mark-Simulacrum added regression-from-stable-to-stable Performance or correctness regression from one stable version to another. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. labels Jul 11, 2018
@Mark-Simulacrum
Copy link
Member

cc @rust-lang/libs -- I believe this is allowed per stability, but we should discuss at least

@stephaneyfx
Copy link
Contributor Author

I know that leaking memory is not considered unsafe, but I assume that the standard library strives to not leak.

@dtolnay
Copy link
Member

dtolnay commented Jul 11, 2018

Looks like the relevant change in 1.26.0 was #48065.

@alexcrichton
Copy link
Member

Thanks for narrowing this down @dtolnay, that makes sense in that I believe the drain iterator (or similar ones) have always performed a little oddly in the face of panics in the middle. I'd personally consider this fine and would consider this ok to not consider P-high and fix in a backport.

That being said it'd of course be great to solve this!

@jonas-schievink
Copy link
Contributor

Fixed by #61224 (which includes a test). Closing.

bors added a commit to rust-lang-ci/rust that referenced this issue Feb 11, 2021
Optimize Vec::retain

Use `copy_non_overlapping` instead of `swap` to reduce memory writes, like what we've done in rust-lang#44355 and `String::retain`.
rust-lang#48065 already tried to do this optimization but it is reverted in rust-lang#67300 due to bad codegen of `DrainFilter::drop`.

This PR re-implement the drop-then-move approach. I did a [benchmark](https://gist.github.com/oxalica/3360eec9376f22533fcecff02798b698) on small-no-drop, small-need-drop, large-no-drop elements with different predicate functions. It turns out that the new implementation is >20% faster in average for almost all cases. Only 2/24 cases are slower by 3% and 5%. See the link above for more detail.

I think regression in may-panic cases is due to drop-guard preventing some optimization. If it's permitted to leak elements when predicate function of element's `drop` panic, the new implementation should be almost always faster than current one.
I'm not sure if we should leak on panic, since there is indeed an issue (rust-lang#52267) complains about it before.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-collections Area: `std::collection` C-bug Category: This is a bug. regression-from-stable-to-stable Performance or correctness regression from one stable version to another. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

5 participants