Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dropping NonSend resources is unsound #3310

Closed
alice-i-cecile opened this issue Dec 12, 2021 · 4 comments
Closed

Dropping NonSend resources is unsound #3310

alice-i-cecile opened this issue Dec 12, 2021 · 4 comments
Labels
A-ECS Entities, components, systems, and events C-Bug An unexpected or incorrect behavior P-High This is particularly urgent, and deserves immediate attention P-Unsound A bug that results in undefined compiler behavior S-Needs-Design This issue requires design work to think about how it would best be accomplished

Comments

@alice-i-cecile
Copy link
Member

alice-i-cecile commented Dec 12, 2021

Bevy version

0.5 (and present main)

What you did

use bevy::prelude::*;
use std::cell::UnsafeCell;
struct MyNonSend(UnsafeCell<()>);

impl Drop for MyNonSend {
    fn drop(&mut self) {
        dbg!("dropped");
    }
}

fn main() {
    let mut world = World::new();

    world.insert_non_send(MyNonSend(UnsafeCell::new(())));

    std::thread::spawn(move || {
        drop(world);
    })
    .join()
    .unwrap();
}

What you expected to happen

The NonSend resource is dropped correctly.

What actually happened

"Dropped" is printed.

Additional information

Full credit to @TheRawMeatball, @jakobhellermann @BoxyUwU and @DJMcNab. Blame them; I'm just the ticketmaster!

@alice-i-cecile alice-i-cecile added C-Bug An unexpected or incorrect behavior A-ECS Entities, components, systems, and events P-High This is particularly urgent, and deserves immediate attention S-Needs-Design This issue requires design work to think about how it would best be accomplished labels Dec 12, 2021
@Guvante
Copy link
Contributor

Guvante commented Dec 14, 2021

Ideas of how to resolve "you can't print "dropped" from a different thread".

  1. Make World non-send
    • Remove unsafe impl Send for World{} to prevent move || { drop(world); } from being valid
    • I feel like it wouldn't be a ticket if this was wanted
    • Never mind my thought that this wouldn't work was correct, I just messed up testing it. World has to be Send.
  2. Implement drop to panic if not on the main thread
    • impl Drop for World { fn drop(&mut self) { if !self.main_thread_validator.is_main_thread()) { panic!("attempted to drop World off of the main thread"); } }
    • This feels even worse than making World non-send
  3. Implement drop and check for a non-send and if so panic
    • False positives could certainly be a thing here unless mechanisms were specifically added to track dropability
  4. Implement drop and mem::forget anything that is non-send
    • Technically correct and safe but not exactly intuitive
  5. Move the drop of the things that are non-send to the main thread
    • impl Drop would be required of course
    • Would require a queue of abstract work that is pumped every frame, I know Unity has one but don't know about Bevy.

@DJMcNab
Copy link
Member

DJMcNab commented Dec 14, 2021

So importantly, NonSend is not only for the main thread

It can be any thread, so long as it is only 'accessed' from that single thread.

Which makes 1 unviable, since it might be needed that things are added on different threads. 2 would be incorrect by the same token.

5 is not possible in general, and unviable anyway because of the same reason.

@Guvante
Copy link
Contributor

Guvante commented Dec 15, 2021

Each World object has a main thread that is set on creation. Sure it doesn't have to be the original thread of the program but it also isn't any thread. There is certainly a main thread for the World.

Allowing N threads to create Worlds and queue destructors would require a mechanism to dequeue on each of those threads but could totally be done. Similarly you could try to queue the destructor and panic if you can not due to there being no queue for the main thread of the World.

Honestly though unless the intention is to be ambiguous it would be nice to clarify if this means mem::forget or not.

@DJMcNab
Copy link
Member

DJMcNab commented Dec 15, 2021

Hmm yeah, sorry

I misunderstood how NonSend's validation works.

@james7132 james7132 added the P-Unsound A bug that results in undefined compiler behavior label Nov 10, 2022
@bors bors bot closed this as completed in aaf384a Jan 9, 2023
james7132 added a commit to james7132/bevy that referenced this issue Jan 21, 2023
# Objective

Fixes bevyengine#3310. Fixes bevyengine#6282. Fixes bevyengine#6278. Fixes bevyengine#3666.

## Solution
Split out `!Send` resources into `NonSendResources`. Add a `origin_thread_id` to all `!Send` Resources, check it on dropping `NonSendResourceData`, if there's a mismatch, panic. Moved all of the checks that `MainThreadValidator` would do into `NonSendResources` instead.

All `!Send` resources now individually track which thread they were inserted from. This is validated against for every access, mutation, and drop that could be done against the value. 

A regression test using an altered version of the example from bevyengine#3310 has been added.

This is a stopgap solution for the current status quo. A full solution may involve fully removing `!Send` resources/components from `World`, which will likely require a much more thorough design on how to handle the existing in-engine and ecosystem use cases.

This PR also introduces another breaking change:

```rust
    use bevy_ecs::prelude::*;

    #[derive(Resource)]
    struct Resource(u32);

    fn main() {
        let mut world = World::new();
        world.insert_resource(Resource(1));
        world.insert_non_send_resource(Resource(2));
        let res = world.get_resource_mut::<Resource>().unwrap();
        assert_eq!(res.0, 2);
    }
```

This code will run correctly on 0.9.1 but not with this PR, since NonSend resources and normal resources have become actual distinct concepts storage wise.

## Changelog
Changed: Fix soundness bug with `World: Send`. Dropping a `World` that contains a `!Send` resource on the wrong thread will now panic.

## Migration Guide
Normal resources and `NonSend` resources no longer share the same backing storage. If `R: Resource`, then `NonSend<R>` and `Res<R>` will return different instances from each other. If you are using both `Res<T>` and `NonSend<T>` (or their mutable variants), to fetch the same resources, it's strongly advised to use `Res<T>`.
alradish pushed a commit to alradish/bevy that referenced this issue Jan 22, 2023
# Objective

Fixes bevyengine#3310. Fixes bevyengine#6282. Fixes bevyengine#6278. Fixes bevyengine#3666.

## Solution
Split out `!Send` resources into `NonSendResources`. Add a `origin_thread_id` to all `!Send` Resources, check it on dropping `NonSendResourceData`, if there's a mismatch, panic. Moved all of the checks that `MainThreadValidator` would do into `NonSendResources` instead.

All `!Send` resources now individually track which thread they were inserted from. This is validated against for every access, mutation, and drop that could be done against the value. 

A regression test using an altered version of the example from bevyengine#3310 has been added.

This is a stopgap solution for the current status quo. A full solution may involve fully removing `!Send` resources/components from `World`, which will likely require a much more thorough design on how to handle the existing in-engine and ecosystem use cases.

This PR also introduces another breaking change:

```rust
    use bevy_ecs::prelude::*;

    #[derive(Resource)]
    struct Resource(u32);

    fn main() {
        let mut world = World::new();
        world.insert_resource(Resource(1));
        world.insert_non_send_resource(Resource(2));
        let res = world.get_resource_mut::<Resource>().unwrap();
        assert_eq!(res.0, 2);
    }
```

This code will run correctly on 0.9.1 but not with this PR, since NonSend resources and normal resources have become actual distinct concepts storage wise.

## Changelog
Changed: Fix soundness bug with `World: Send`. Dropping a `World` that contains a `!Send` resource on the wrong thread will now panic.

## Migration Guide
Normal resources and `NonSend` resources no longer share the same backing storage. If `R: Resource`, then `NonSend<R>` and `Res<R>` will return different instances from each other. If you are using both `Res<T>` and `NonSend<T>` (or their mutable variants), to fetch the same resources, it's strongly advised to use `Res<T>`.
ItsDoot pushed a commit to ItsDoot/bevy that referenced this issue Feb 1, 2023
# Objective

Fixes bevyengine#3310. Fixes bevyengine#6282. Fixes bevyengine#6278. Fixes bevyengine#3666.

## Solution
Split out `!Send` resources into `NonSendResources`. Add a `origin_thread_id` to all `!Send` Resources, check it on dropping `NonSendResourceData`, if there's a mismatch, panic. Moved all of the checks that `MainThreadValidator` would do into `NonSendResources` instead.

All `!Send` resources now individually track which thread they were inserted from. This is validated against for every access, mutation, and drop that could be done against the value. 

A regression test using an altered version of the example from bevyengine#3310 has been added.

This is a stopgap solution for the current status quo. A full solution may involve fully removing `!Send` resources/components from `World`, which will likely require a much more thorough design on how to handle the existing in-engine and ecosystem use cases.

This PR also introduces another breaking change:

```rust
    use bevy_ecs::prelude::*;

    #[derive(Resource)]
    struct Resource(u32);

    fn main() {
        let mut world = World::new();
        world.insert_resource(Resource(1));
        world.insert_non_send_resource(Resource(2));
        let res = world.get_resource_mut::<Resource>().unwrap();
        assert_eq!(res.0, 2);
    }
```

This code will run correctly on 0.9.1 but not with this PR, since NonSend resources and normal resources have become actual distinct concepts storage wise.

## Changelog
Changed: Fix soundness bug with `World: Send`. Dropping a `World` that contains a `!Send` resource on the wrong thread will now panic.

## Migration Guide
Normal resources and `NonSend` resources no longer share the same backing storage. If `R: Resource`, then `NonSend<R>` and `Res<R>` will return different instances from each other. If you are using both `Res<T>` and `NonSend<T>` (or their mutable variants), to fetch the same resources, it's strongly advised to use `Res<T>`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-ECS Entities, components, systems, and events C-Bug An unexpected or incorrect behavior P-High This is particularly urgent, and deserves immediate attention P-Unsound A bug that results in undefined compiler behavior S-Needs-Design This issue requires design work to think about how it would best be accomplished
Projects
None yet
4 participants