From 98e58ee6e065a33d15bfc10e4518ba051319c46a Mon Sep 17 00:00:00 2001 From: Christian Legnitto Date: Tue, 17 May 2022 00:14:42 -0400 Subject: [PATCH 01/15] RFC for `Entropy` --- rfcs/entropy.md | 135 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 135 insertions(+) create mode 100644 rfcs/entropy.md diff --git a/rfcs/entropy.md b/rfcs/entropy.md new file mode 100644 index 00000000..9b4788c7 --- /dev/null +++ b/rfcs/entropy.md @@ -0,0 +1,135 @@ +# Feature Name: `Entropy` + +## Summary + +Include a source of entropy to (optionally) enable deterministic random number generation. + +## Motivation + +Bevy games / applications often need to use randomness[1](#1) but doing so makes execution non-deterministic. This is problematic for automated testing as well as deterministic execution. Deterministic execution is important for those wishing to create games with [deterministic lockstep](https://gafferongames.com/post/deterministic_lockstep/) as well as high-fidelity simulations. + +Currently there is no official way to introduce randomness in bevy, a plugin, or an app. Other engines such as [Godot](https://docs.godotengine.org/en/stable/tutorials/math/random_number_generation.html), [Unity](https://docs.unity3d.com/ScriptReference/Random.html), and [Unreal](https://docs.unrealengine.com/4.27/en-US/BlueprintAPI/Math/Random/) include engine support for randomness. + + +1 For example, 16 of bevy's examples currently use `rand` +
+
+examples/games/contributors.rs
+examples/games/alien_cake_addict.rs
+examples/async_tasks/external_source_external_thread.rs
+examples/async_tasks/async_compute.rs
+examples/ecs/iter_combinations.rs
+examples/stress_tests/transform_hierarchy.rs
+examples/stress_tests/many_lights.rs
+examples/animation/custom_skinned_mesh.rs
+examples/stress_tests/bevymark.rs
+examples/stress_tests/many_sprites.rs
+examples/ecs/parallel_query.rs
+examples/ecs/component_change_detection.rs
+examples/ecs/ecs_guide.rs
+examples/app/random.rs
+crates/bevy_ecs/examples/resources.rs
+crates/bevy_ecs/examples/change_detection.rs
+
+
+ + +## User-facing explanation + +### Overview + +Games often use randomness as a core mechanic. For example, card games generate a random deck for each game and killing monsters in an RPG often rewards players with a random item. While randomness makes games more interesting and increases replayability, it also makes games harder to test and prevents advanced techniques such as [deterministic lockstep](https://gafferongames.com/post/deterministic_lockstep/). + +Let's pretend you are creating a poker game where a human player can play against the computer. The computer's poker logic is very simple--when the computer has a good hand, it bets all of its money. To make sure the behavior works, you write a test to first check the computer's hand and if it is good confirm that all its money is bet. If the test passes does it ensure the computer behaves as intended? Sadly, no. + +Because the deck is randomly shuffled for each game--without doing so the player would already know the card order from the previous game--it is not guaranteed that the computer player gets a good hand and thus the betting logic goes unchecked. While there are ways around this--a fake deck that is not shuffled, running the test many times to increase confidence, breaking the logic into units and testing those--it would be very helpful to have randomness as well as way to make it _less_ random. + +Luckily, when a computer needs a random number it doesn't use real randomness and instead uses a [pseudorandom number generator](https://en.wikipedia.org/wiki/Pseudorandom_number_generator). Popular Rust libraries containing pseudorandom number generators are [`rand`](https://crates.io/crates/rand) and [`fastrand`](https://crates.io/crates/fastrand). + +Pseudorandom number generators require a source of [entropy](https://en.wikipedia.org/wiki/Entropy) called a [random seed](https://en.wikipedia.org/wiki/Random_seed). The random seed is used as input to generate numbers that _appear_ random but are instead in a specific and deterministic order. For the same random seed, a pseudorandom number generator always returns the same numbers in the same order. + +For example, let's say you seed a pseudorandom number generator with `1234`. You then ask for a random number between `10` and `99` and the pseudorandom number generator returns `12`. If you run the program again with the same seed (`1234`) and ask for another random number between `1` and `99`, you will again get `12`. If you then change the seed to `4567` and run the program, more than likely the result will not be `12` and will instead be a different number. If you run the program again with the `4567` seed, you should see the same number from the previous `4567`-seeded run. + +There are many types of pseudorandom number generators each with their own strengths and weakenesses. Because of this, Bevy does not include a pseudorandom number generator. Instead, the `bevy_entropy` plugin includes a source of entropy to use as a random seed for your pseudorandom number generator. The plugin can be completely disabled if no source of entropy is required, the default entropy from the OS can be used if randomness is needed but deterministic execution is not, or a world seed can be specified for deterministic random number generation. + +Note that Bevy currently has [other sources of non-determinism](https://github.com/bevyengine/bevy/discussions/2480) unrealted to pseudorandom number generators. + +### Usage + +The `bevy_entropy` plugin ships with Bevy and is enabled by default. If you do not need randomness, you may [disable the plugin](https://docs.rs/bevy/latest/bevy/app/struct.PluginGroupBuilder.html#method.disable) or use Bevy's [minimal set of plugins](https://docs.rs/bevy/latest/bevy/struct.MinimalPlugins.html). + +When enabled, `bevy_entropy` provides one world resource: `Entropy`. `Entropy` is then used as the seed for the pseudorandom number generator of your choosing. A complete example leveraging the [`StdRng`](https://docs.rs/rand/latest/rand/rngs/struct.StdRng.html) pseudorandom number generator from the [`rand`](https://crates.io/crates/rand) crate can be found in https://github.com/bevyengine/bevy/tree/main/examples. + +The default source of world entropy [`Entropy::default()`] is non-deterministic and seeded from the operating system. It is guarenteed to be suitable for [cryptographic applications](https://en.wikipedia.org/wiki/Pseudorandom_number_generator#Cryptographic_PRNGs) but the actual seeding mechanism is an implementation detail that will change over time. + +You may choose to determinstically seed your own world entropy via [`Entropy::from`]. The seed you choose may have security implications or influence the distribution of the resulting random numbers. See https://rust-random.github.io/book/guide-seeding.html for more details about how to pick a "good" random seed for your needs. + +Depending on your game and the type of randomness you require, when specifying a seed you would normally do one of the following: + +1. Get a good random seed out-of-band and hardcode it in the source. +2. Dynamically call to the OS and print the seed so the user can rerun deterministically. In games like [Factorio](https://www.factorio.com/) sharing random seeds is encouraged and supported. +3. Dynamically call to the OS and share the seed with a server so the client and server deterministically execute together. +4. Load the seed from a server so the client and server deterministically execute together. + + +## Implementation strategy + +https://github.com/bevyengine/bevy/pull/2504 + +- The PR includes `rand` as a dependency but it is not in public API. We might be able to slim it down to [`getrandom`](https://crates.io/crates/getrandom). + + +## Drawbacks + +- This may not be general enough to include in Bevy. +- This may not be general enough to be on by default. +- Includes `rand` as a dependency + - But not in public API + +## Rationale and alternatives + +- Why is this design the best in the space of possible designs? + - It gives flexibility for PRNG crates while also enforcing standards on the ecosystem. + - It defaults to what someone would resonably expect if they wrote it themselves. + - It defaults to something safe (suitable for cryptographic functions), removing a possible footgun. +- What other designs have been considered and what is the rationale for not choosing them? + - We could go higher and expose an API closer to `rand`. This is what [Godot](https://docs.godotengine.org/en/stable/tutorials/math/random_number_generation.html), [Unity](https://docs.unity3d.com/ScriptReference/Random.html), and [Unreal](https://docs.unrealengine.com/4.27/en-US/BlueprintAPI/Math/Random/) do. We are not experts in PRNG api design, and while `rand` is clearly the most popular random crate in the Rust ecosystem we don't currently want to tie bevy to any particluar API in case something better emerges. + - We could go lower and merely expose a static `WorldSeed`. I'm worried about what the default would be and forcing a seed at world creation feels heavyweight. + - We could default to a faster PRNG rather than a safer one. I wanted folks to fall into the pit of success. +- What objections immediately spring to mind? How have you addressed them? + - `rand` is too heavy of a dependency. + - This should not be in core. +- What is the impact of not doing this? + - There is no chance of the Bevy ecosystem supporting deterministic execution. +- Why is this important to implement as a feature of Bevy itself, rather than an ecosystem crate? + - There needs to be one true way to keep the ecosystem coherant. + +## Prior art + +* https://github.com/bevyengine/bevy/discussions/2480 +* https://github.com/bevyengine/bevy/discussions/1678 + +Other engines such as [Godot](https://docs.godotengine.org/en/stable/tutorials/math/random_number_generation.html), [Unity](https://docs.unity3d.com/ScriptReference/Random.html), and [Unreal](https://docs.unrealengine.com/4.27/en-US/BlueprintAPI/Math/Random/) include engine support for randomness (with a higher-level API). + +* https://towardsdatascience.com/how-to-use-random-seeds-effectively-54a4cd855a79 +* https://www.whatgamesare.com/determinism.html +* https://gafferongames.com/post/deterministic_lockstep/ +* https://ruoyusun.com/2019/03/29/game-networking-2.html +* https://arxiv.org/abs/2104.06262 + +## Unresolved questions + +- What parts of the design do you expect to resolve through the RFC process before this gets merged? + - Do we want this? + - What about `rand`? + - Is `Entropy` naming too obscure? + - How do we deal with security vs speed tradeoff here? + - Is this the right level of abstraction? Lower? Higher? +- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? + - Built-in higher-level randomness APIs + - Entropy per-system rather than per-world + +## Future possibilities + +- Higher-level randomness APIs +- Entropy per-system rather than per-world From 31081ad324a683eecbc56e0a7fcea1ef28f1e4ec Mon Sep 17 00:00:00 2001 From: Christian Legnitto Date: Tue, 17 May 2022 00:16:52 -0400 Subject: [PATCH 02/15] Rename with PR number --- rfcs/{entropy.md => 55-entropy.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename rfcs/{entropy.md => 55-entropy.md} (100%) diff --git a/rfcs/entropy.md b/rfcs/55-entropy.md similarity index 100% rename from rfcs/entropy.md rename to rfcs/55-entropy.md From 1753a7162bb0556dbad5e98d8672a6d59dc195b9 Mon Sep 17 00:00:00 2001 From: Christian Legnitto Date: Tue, 17 May 2022 00:24:20 -0400 Subject: [PATCH 03/15] Fix typos --- rfcs/55-entropy.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/rfcs/55-entropy.md b/rfcs/55-entropy.md index 9b4788c7..3ff98c74 100644 --- a/rfcs/55-entropy.md +++ b/rfcs/55-entropy.md @@ -50,7 +50,7 @@ Pseudorandom number generators require a source of [entropy](https://en.wikipedi For example, let's say you seed a pseudorandom number generator with `1234`. You then ask for a random number between `10` and `99` and the pseudorandom number generator returns `12`. If you run the program again with the same seed (`1234`) and ask for another random number between `1` and `99`, you will again get `12`. If you then change the seed to `4567` and run the program, more than likely the result will not be `12` and will instead be a different number. If you run the program again with the `4567` seed, you should see the same number from the previous `4567`-seeded run. -There are many types of pseudorandom number generators each with their own strengths and weakenesses. Because of this, Bevy does not include a pseudorandom number generator. Instead, the `bevy_entropy` plugin includes a source of entropy to use as a random seed for your pseudorandom number generator. The plugin can be completely disabled if no source of entropy is required, the default entropy from the OS can be used if randomness is needed but deterministic execution is not, or a world seed can be specified for deterministic random number generation. +There are many types of pseudorandom number generators each with their own strengths and weaknesses. Because of this, Bevy does not include a pseudorandom number generator. Instead, the `bevy_entropy` plugin includes a source of entropy to use as a random seed for your pseudorandom number generator. The plugin can be completely disabled if no source of entropy is required, the default entropy from the OS can be used if randomness is needed but deterministic execution is not, or a world seed can be specified for deterministic random number generation. Note that Bevy currently has [other sources of non-determinism](https://github.com/bevyengine/bevy/discussions/2480) unrealted to pseudorandom number generators. @@ -84,7 +84,7 @@ https://github.com/bevyengine/bevy/pull/2504 - This may not be general enough to include in Bevy. - This may not be general enough to be on by default. - Includes `rand` as a dependency - - But not in public API + - But not in public API. ## Rationale and alternatives @@ -93,7 +93,7 @@ https://github.com/bevyengine/bevy/pull/2504 - It defaults to what someone would resonably expect if they wrote it themselves. - It defaults to something safe (suitable for cryptographic functions), removing a possible footgun. - What other designs have been considered and what is the rationale for not choosing them? - - We could go higher and expose an API closer to `rand`. This is what [Godot](https://docs.godotengine.org/en/stable/tutorials/math/random_number_generation.html), [Unity](https://docs.unity3d.com/ScriptReference/Random.html), and [Unreal](https://docs.unrealengine.com/4.27/en-US/BlueprintAPI/Math/Random/) do. We are not experts in PRNG api design, and while `rand` is clearly the most popular random crate in the Rust ecosystem we don't currently want to tie bevy to any particluar API in case something better emerges. + - We could go higher and expose an API closer to `rand`. This is what [Godot](https://docs.godotengine.org/en/stable/tutorials/math/random_number_generation.html), [Unity](https://docs.unity3d.com/ScriptReference/Random.html), and [Unreal](https://docs.unrealengine.com/4.27/en-US/BlueprintAPI/Math/Random/) do. We are not experts in PRNG API design, and while `rand` is clearly the most popular random crate in the Rust ecosystem we don't currently want to tie bevy to any particluar API in case something better emerges. - We could go lower and merely expose a static `WorldSeed`. I'm worried about what the default would be and forcing a seed at world creation feels heavyweight. - We could default to a faster PRNG rather than a safer one. I wanted folks to fall into the pit of success. - What objections immediately spring to mind? How have you addressed them? From 2db25dc7619153e1e487b53d32556cf372b9f408 Mon Sep 17 00:00:00 2001 From: Christian Legnitto Date: Tue, 17 May 2022 00:26:03 -0400 Subject: [PATCH 04/15] Fix typo Co-authored-by: Alice Cecile --- rfcs/55-entropy.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/55-entropy.md b/rfcs/55-entropy.md index 3ff98c74..6630b3a9 100644 --- a/rfcs/55-entropy.md +++ b/rfcs/55-entropy.md @@ -93,7 +93,7 @@ https://github.com/bevyengine/bevy/pull/2504 - It defaults to what someone would resonably expect if they wrote it themselves. - It defaults to something safe (suitable for cryptographic functions), removing a possible footgun. - What other designs have been considered and what is the rationale for not choosing them? - - We could go higher and expose an API closer to `rand`. This is what [Godot](https://docs.godotengine.org/en/stable/tutorials/math/random_number_generation.html), [Unity](https://docs.unity3d.com/ScriptReference/Random.html), and [Unreal](https://docs.unrealengine.com/4.27/en-US/BlueprintAPI/Math/Random/) do. We are not experts in PRNG API design, and while `rand` is clearly the most popular random crate in the Rust ecosystem we don't currently want to tie bevy to any particluar API in case something better emerges. + - We could go higher and expose an API closer to `rand`. This is what [Godot](https://docs.godotengine.org/en/stable/tutorials/math/random_number_generation.html), [Unity](https://docs.unity3d.com/ScriptReference/Random.html), and [Unreal](https://docs.unrealengine.com/4.27/en-US/BlueprintAPI/Math/Random/) do. We are not experts in PRNG API design, and while `rand` is clearly the most popular random crate in the Rust ecosystem we don't currently want to tie bevy to any particular API in case something better emerges. - We could go lower and merely expose a static `WorldSeed`. I'm worried about what the default would be and forcing a seed at world creation feels heavyweight. - We could default to a faster PRNG rather than a safer one. I wanted folks to fall into the pit of success. - What objections immediately spring to mind? How have you addressed them? From 97278d6a8cbcc68d37050b55ded141a3ad9af854 Mon Sep 17 00:00:00 2001 From: Christian Legnitto Date: Tue, 17 May 2022 00:39:58 -0400 Subject: [PATCH 05/15] Rename feature from `Entropy` to `Deterministic RNG` --- rfcs/{55-entropy.md => 55-deterministic_rng.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename rfcs/{55-entropy.md => 55-deterministic_rng.md} (99%) diff --git a/rfcs/55-entropy.md b/rfcs/55-deterministic_rng.md similarity index 99% rename from rfcs/55-entropy.md rename to rfcs/55-deterministic_rng.md index 6630b3a9..ec54e112 100644 --- a/rfcs/55-entropy.md +++ b/rfcs/55-deterministic_rng.md @@ -1,4 +1,4 @@ -# Feature Name: `Entropy` +# Feature Name: `Deterministic RNG` ## Summary From 4309d80af5c08d1e139963d1350684967f6cfbbb Mon Sep 17 00:00:00 2001 From: Christian Legnitto Date: Tue, 17 May 2022 00:42:45 -0400 Subject: [PATCH 06/15] Typo, missing "a" --- rfcs/55-entropy.md | 135 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 135 insertions(+) create mode 100644 rfcs/55-entropy.md diff --git a/rfcs/55-entropy.md b/rfcs/55-entropy.md new file mode 100644 index 00000000..94be61a1 --- /dev/null +++ b/rfcs/55-entropy.md @@ -0,0 +1,135 @@ +# Feature Name: `Deterministic RNG` + +## Summary + +Include a source of entropy to (optionally) enable deterministic random number generation. + +## Motivation + +Bevy games / applications often need to use randomness[1](#1) but doing so makes execution non-deterministic. This is problematic for automated testing as well as deterministic execution. Deterministic execution is important for those wishing to create games with [deterministic lockstep](https://gafferongames.com/post/deterministic_lockstep/) as well as high-fidelity simulations. + +Currently there is no official way to introduce randomness in bevy, a plugin, or an app. Other engines such as [Godot](https://docs.godotengine.org/en/stable/tutorials/math/random_number_generation.html), [Unity](https://docs.unity3d.com/ScriptReference/Random.html), and [Unreal](https://docs.unrealengine.com/4.27/en-US/BlueprintAPI/Math/Random/) include engine support for randomness. + + +1 For example, 16 of bevy's examples currently use `rand` +
+
+examples/games/contributors.rs
+examples/games/alien_cake_addict.rs
+examples/async_tasks/external_source_external_thread.rs
+examples/async_tasks/async_compute.rs
+examples/ecs/iter_combinations.rs
+examples/stress_tests/transform_hierarchy.rs
+examples/stress_tests/many_lights.rs
+examples/animation/custom_skinned_mesh.rs
+examples/stress_tests/bevymark.rs
+examples/stress_tests/many_sprites.rs
+examples/ecs/parallel_query.rs
+examples/ecs/component_change_detection.rs
+examples/ecs/ecs_guide.rs
+examples/app/random.rs
+crates/bevy_ecs/examples/resources.rs
+crates/bevy_ecs/examples/change_detection.rs
+
+
+ + +## User-facing explanation + +### Overview + +Games often use randomness as a core mechanic. For example, card games generate a random deck for each game and killing monsters in an RPG often rewards players with a random item. While randomness makes games more interesting and increases replayability, it also makes games harder to test and prevents advanced techniques such as [deterministic lockstep](https://gafferongames.com/post/deterministic_lockstep/). + +Let's pretend you are creating a poker game where a human player can play against the computer. The computer's poker logic is very simple--when the computer has a good hand, it bets all of its money. To make sure the behavior works, you write a test to first check the computer's hand and if it is good confirm that all its money is bet. If the test passes does it ensure the computer behaves as intended? Sadly, no. + +Because the deck is randomly shuffled for each game--without doing so the player would already know the card order from the previous game--it is not guaranteed that the computer player gets a good hand and thus the betting logic goes unchecked. While there are ways around this--a fake deck that is not shuffled, running the test many times to increase confidence, breaking the logic into units and testing those--it would be very helpful to have randomness as well as a way to make it _less_ random. + +Luckily, when a computer needs a random number it doesn't use real randomness and instead uses a [pseudorandom number generator](https://en.wikipedia.org/wiki/Pseudorandom_number_generator). Popular Rust libraries containing pseudorandom number generators are [`rand`](https://crates.io/crates/rand) and [`fastrand`](https://crates.io/crates/fastrand). + +Pseudorandom number generators require a source of [entropy](https://en.wikipedia.org/wiki/Entropy) called a [random seed](https://en.wikipedia.org/wiki/Random_seed). The random seed is used as input to generate numbers that _appear_ random but are instead in a specific and deterministic order. For the same random seed, a pseudorandom number generator always returns the same numbers in the same order. + +For example, let's say you seed a pseudorandom number generator with `1234`. You then ask for a random number between `10` and `99` and the pseudorandom number generator returns `12`. If you run the program again with the same seed (`1234`) and ask for another random number between `1` and `99`, you will again get `12`. If you then change the seed to `4567` and run the program, more than likely the result will not be `12` and will instead be a different number. If you run the program again with the `4567` seed, you should see the same number from the previous `4567`-seeded run. + +There are many types of pseudorandom number generators each with their own strengths and weaknesses. Because of this, Bevy does not include a pseudorandom number generator. Instead, the `bevy_entropy` plugin includes a source of entropy to use as a random seed for your pseudorandom number generator. The plugin can be completely disabled if no source of entropy is required, the default entropy from the OS can be used if randomness is needed but deterministic execution is not, or a world seed can be specified for deterministic random number generation. + +Note that Bevy currently has [other sources of non-determinism](https://github.com/bevyengine/bevy/discussions/2480) unrealted to pseudorandom number generators. + +### Usage + +The `bevy_entropy` plugin ships with Bevy and is enabled by default. If you do not need randomness, you may [disable the plugin](https://docs.rs/bevy/latest/bevy/app/struct.PluginGroupBuilder.html#method.disable) or use Bevy's [minimal set of plugins](https://docs.rs/bevy/latest/bevy/struct.MinimalPlugins.html). + +When enabled, `bevy_entropy` provides one world resource: `Entropy`. `Entropy` is then used as the seed for the pseudorandom number generator of your choosing. A complete example leveraging the [`StdRng`](https://docs.rs/rand/latest/rand/rngs/struct.StdRng.html) pseudorandom number generator from the [`rand`](https://crates.io/crates/rand) crate can be found in https://github.com/bevyengine/bevy/tree/main/examples. + +The default source of world entropy [`Entropy::default()`] is non-deterministic and seeded from the operating system. It is guarenteed to be suitable for [cryptographic applications](https://en.wikipedia.org/wiki/Pseudorandom_number_generator#Cryptographic_PRNGs) but the actual seeding mechanism is an implementation detail that will change over time. + +You may choose to determinstically seed your own world entropy via [`Entropy::from`]. The seed you choose may have security implications or influence the distribution of the resulting random numbers. See https://rust-random.github.io/book/guide-seeding.html for more details about how to pick a "good" random seed for your needs. + +Depending on your game and the type of randomness you require, when specifying a seed you would normally do one of the following: + +1. Get a good random seed out-of-band and hardcode it in the source. +2. Dynamically call to the OS and print the seed so the user can rerun deterministically. In games like [Factorio](https://www.factorio.com/) sharing random seeds is encouraged and supported. +3. Dynamically call to the OS and share the seed with a server so the client and server deterministically execute together. +4. Load the seed from a server so the client and server deterministically execute together. + + +## Implementation strategy + +https://github.com/bevyengine/bevy/pull/2504 + +- The PR includes `rand` as a dependency but it is not in public API. We might be able to slim it down to [`getrandom`](https://crates.io/crates/getrandom). + + +## Drawbacks + +- This may not be general enough to include in Bevy. +- This may not be general enough to be on by default. +- Includes `rand` as a dependency + - But not in public API. + +## Rationale and alternatives + +- Why is this design the best in the space of possible designs? + - It gives flexibility for PRNG crates while also enforcing standards on the ecosystem. + - It defaults to what someone would resonably expect if they wrote it themselves. + - It defaults to something safe (suitable for cryptographic functions), removing a possible footgun. +- What other designs have been considered and what is the rationale for not choosing them? + - We could go higher and expose an API closer to `rand`. This is what [Godot](https://docs.godotengine.org/en/stable/tutorials/math/random_number_generation.html), [Unity](https://docs.unity3d.com/ScriptReference/Random.html), and [Unreal](https://docs.unrealengine.com/4.27/en-US/BlueprintAPI/Math/Random/) do. We are not experts in PRNG API design, and while `rand` is clearly the most popular random crate in the Rust ecosystem we don't currently want to tie bevy to any particular API in case something better emerges. + - We could go lower and merely expose a static `WorldSeed`. I'm worried about what the default would be and forcing a seed at world creation feels heavyweight. + - We could default to a faster PRNG rather than a safer one. I wanted folks to fall into the pit of success. +- What objections immediately spring to mind? How have you addressed them? + - `rand` is too heavy of a dependency. + - This should not be in core. +- What is the impact of not doing this? + - There is no chance of the Bevy ecosystem supporting deterministic execution. +- Why is this important to implement as a feature of Bevy itself, rather than an ecosystem crate? + - There needs to be one true way to keep the ecosystem coherant. + +## Prior art + +* https://github.com/bevyengine/bevy/discussions/2480 +* https://github.com/bevyengine/bevy/discussions/1678 + +Other engines such as [Godot](https://docs.godotengine.org/en/stable/tutorials/math/random_number_generation.html), [Unity](https://docs.unity3d.com/ScriptReference/Random.html), and [Unreal](https://docs.unrealengine.com/4.27/en-US/BlueprintAPI/Math/Random/) include engine support for randomness (with a higher-level API). + +* https://towardsdatascience.com/how-to-use-random-seeds-effectively-54a4cd855a79 +* https://www.whatgamesare.com/determinism.html +* https://gafferongames.com/post/deterministic_lockstep/ +* https://ruoyusun.com/2019/03/29/game-networking-2.html +* https://arxiv.org/abs/2104.06262 + +## Unresolved questions + +- What parts of the design do you expect to resolve through the RFC process before this gets merged? + - Do we want this? + - What about `rand`? + - Is `Entropy` naming too obscure? + - How do we deal with security vs speed tradeoff here? + - Is this the right level of abstraction? Lower? Higher? +- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? + - Built-in higher-level randomness APIs + - Entropy per-system rather than per-world + +## Future possibilities + +- Higher-level randomness APIs +- Entropy per-system rather than per-world From f926a6adea5cf808ff541b6e5802482af52b8cfe Mon Sep 17 00:00:00 2001 From: Christian Legnitto Date: Tue, 17 May 2022 00:44:17 -0400 Subject: [PATCH 07/15] Reword --- rfcs/55-entropy.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/55-entropy.md b/rfcs/55-entropy.md index 94be61a1..bc13b972 100644 --- a/rfcs/55-entropy.md +++ b/rfcs/55-entropy.md @@ -50,7 +50,7 @@ Pseudorandom number generators require a source of [entropy](https://en.wikipedi For example, let's say you seed a pseudorandom number generator with `1234`. You then ask for a random number between `10` and `99` and the pseudorandom number generator returns `12`. If you run the program again with the same seed (`1234`) and ask for another random number between `1` and `99`, you will again get `12`. If you then change the seed to `4567` and run the program, more than likely the result will not be `12` and will instead be a different number. If you run the program again with the `4567` seed, you should see the same number from the previous `4567`-seeded run. -There are many types of pseudorandom number generators each with their own strengths and weaknesses. Because of this, Bevy does not include a pseudorandom number generator. Instead, the `bevy_entropy` plugin includes a source of entropy to use as a random seed for your pseudorandom number generator. The plugin can be completely disabled if no source of entropy is required, the default entropy from the OS can be used if randomness is needed but deterministic execution is not, or a world seed can be specified for deterministic random number generation. +There are many types of pseudorandom number generators each with their own strengths and weaknesses. Because of this, Bevy does not include a pseudorandom number generator. Instead, the `bevy_entropy` plugin includes a source of entropy to use as a random seed for your chosen pseudorandom number generator. The plugin can be completely disabled if no source of entropy is required, the default entropy from the OS can be used if randomness is needed but deterministic execution is not, or a world seed can be specified for deterministic random number generation. Note that Bevy currently has [other sources of non-determinism](https://github.com/bevyengine/bevy/discussions/2480) unrealted to pseudorandom number generators. From c7811bec28c2783eaafdd4c045395bf154cc19fb Mon Sep 17 00:00:00 2001 From: Christian Legnitto Date: Tue, 17 May 2022 00:45:48 -0400 Subject: [PATCH 08/15] Spelling --- rfcs/55-entropy.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/55-entropy.md b/rfcs/55-entropy.md index bc13b972..ed3adc68 100644 --- a/rfcs/55-entropy.md +++ b/rfcs/55-entropy.md @@ -52,7 +52,7 @@ For example, let's say you seed a pseudorandom number generator with `1234`. You There are many types of pseudorandom number generators each with their own strengths and weaknesses. Because of this, Bevy does not include a pseudorandom number generator. Instead, the `bevy_entropy` plugin includes a source of entropy to use as a random seed for your chosen pseudorandom number generator. The plugin can be completely disabled if no source of entropy is required, the default entropy from the OS can be used if randomness is needed but deterministic execution is not, or a world seed can be specified for deterministic random number generation. -Note that Bevy currently has [other sources of non-determinism](https://github.com/bevyengine/bevy/discussions/2480) unrealted to pseudorandom number generators. +Note that Bevy currently has [other sources of non-determinism](https://github.com/bevyengine/bevy/discussions/2480) unrelated to pseudorandom number generators. ### Usage From 99a3be6cbbaffdb1bce283370931e7f862c011f1 Mon Sep 17 00:00:00 2001 From: Christian Legnitto Date: Tue, 17 May 2022 00:47:21 -0400 Subject: [PATCH 09/15] Reword --- rfcs/55-entropy.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/55-entropy.md b/rfcs/55-entropy.md index ed3adc68..7de6248e 100644 --- a/rfcs/55-entropy.md +++ b/rfcs/55-entropy.md @@ -64,7 +64,7 @@ The default source of world entropy [`Entropy::default()`] is non-deterministic You may choose to determinstically seed your own world entropy via [`Entropy::from`]. The seed you choose may have security implications or influence the distribution of the resulting random numbers. See https://rust-random.github.io/book/guide-seeding.html for more details about how to pick a "good" random seed for your needs. -Depending on your game and the type of randomness you require, when specifying a seed you would normally do one of the following: +Depending on your game and the type of randomness you require, when specifying a seed you will normally do one of the following: 1. Get a good random seed out-of-band and hardcode it in the source. 2. Dynamically call to the OS and print the seed so the user can rerun deterministically. In games like [Factorio](https://www.factorio.com/) sharing random seeds is encouraged and supported. From b2a814e2a14e8a78735231b513137f9a0a138c4b Mon Sep 17 00:00:00 2001 From: Christian Legnitto Date: Tue, 17 May 2022 02:08:09 -0400 Subject: [PATCH 10/15] Delete old file --- rfcs/55-deterministic_rng.md | 8 +-- rfcs/55-entropy.md | 135 ----------------------------------- 2 files changed, 4 insertions(+), 139 deletions(-) delete mode 100644 rfcs/55-entropy.md diff --git a/rfcs/55-deterministic_rng.md b/rfcs/55-deterministic_rng.md index ec54e112..7de6248e 100644 --- a/rfcs/55-deterministic_rng.md +++ b/rfcs/55-deterministic_rng.md @@ -42,7 +42,7 @@ Games often use randomness as a core mechanic. For example, card games generate Let's pretend you are creating a poker game where a human player can play against the computer. The computer's poker logic is very simple--when the computer has a good hand, it bets all of its money. To make sure the behavior works, you write a test to first check the computer's hand and if it is good confirm that all its money is bet. If the test passes does it ensure the computer behaves as intended? Sadly, no. -Because the deck is randomly shuffled for each game--without doing so the player would already know the card order from the previous game--it is not guaranteed that the computer player gets a good hand and thus the betting logic goes unchecked. While there are ways around this--a fake deck that is not shuffled, running the test many times to increase confidence, breaking the logic into units and testing those--it would be very helpful to have randomness as well as way to make it _less_ random. +Because the deck is randomly shuffled for each game--without doing so the player would already know the card order from the previous game--it is not guaranteed that the computer player gets a good hand and thus the betting logic goes unchecked. While there are ways around this--a fake deck that is not shuffled, running the test many times to increase confidence, breaking the logic into units and testing those--it would be very helpful to have randomness as well as a way to make it _less_ random. Luckily, when a computer needs a random number it doesn't use real randomness and instead uses a [pseudorandom number generator](https://en.wikipedia.org/wiki/Pseudorandom_number_generator). Popular Rust libraries containing pseudorandom number generators are [`rand`](https://crates.io/crates/rand) and [`fastrand`](https://crates.io/crates/fastrand). @@ -50,9 +50,9 @@ Pseudorandom number generators require a source of [entropy](https://en.wikipedi For example, let's say you seed a pseudorandom number generator with `1234`. You then ask for a random number between `10` and `99` and the pseudorandom number generator returns `12`. If you run the program again with the same seed (`1234`) and ask for another random number between `1` and `99`, you will again get `12`. If you then change the seed to `4567` and run the program, more than likely the result will not be `12` and will instead be a different number. If you run the program again with the `4567` seed, you should see the same number from the previous `4567`-seeded run. -There are many types of pseudorandom number generators each with their own strengths and weaknesses. Because of this, Bevy does not include a pseudorandom number generator. Instead, the `bevy_entropy` plugin includes a source of entropy to use as a random seed for your pseudorandom number generator. The plugin can be completely disabled if no source of entropy is required, the default entropy from the OS can be used if randomness is needed but deterministic execution is not, or a world seed can be specified for deterministic random number generation. +There are many types of pseudorandom number generators each with their own strengths and weaknesses. Because of this, Bevy does not include a pseudorandom number generator. Instead, the `bevy_entropy` plugin includes a source of entropy to use as a random seed for your chosen pseudorandom number generator. The plugin can be completely disabled if no source of entropy is required, the default entropy from the OS can be used if randomness is needed but deterministic execution is not, or a world seed can be specified for deterministic random number generation. -Note that Bevy currently has [other sources of non-determinism](https://github.com/bevyengine/bevy/discussions/2480) unrealted to pseudorandom number generators. +Note that Bevy currently has [other sources of non-determinism](https://github.com/bevyengine/bevy/discussions/2480) unrelated to pseudorandom number generators. ### Usage @@ -64,7 +64,7 @@ The default source of world entropy [`Entropy::default()`] is non-deterministic You may choose to determinstically seed your own world entropy via [`Entropy::from`]. The seed you choose may have security implications or influence the distribution of the resulting random numbers. See https://rust-random.github.io/book/guide-seeding.html for more details about how to pick a "good" random seed for your needs. -Depending on your game and the type of randomness you require, when specifying a seed you would normally do one of the following: +Depending on your game and the type of randomness you require, when specifying a seed you will normally do one of the following: 1. Get a good random seed out-of-band and hardcode it in the source. 2. Dynamically call to the OS and print the seed so the user can rerun deterministically. In games like [Factorio](https://www.factorio.com/) sharing random seeds is encouraged and supported. diff --git a/rfcs/55-entropy.md b/rfcs/55-entropy.md deleted file mode 100644 index 7de6248e..00000000 --- a/rfcs/55-entropy.md +++ /dev/null @@ -1,135 +0,0 @@ -# Feature Name: `Deterministic RNG` - -## Summary - -Include a source of entropy to (optionally) enable deterministic random number generation. - -## Motivation - -Bevy games / applications often need to use randomness[1](#1) but doing so makes execution non-deterministic. This is problematic for automated testing as well as deterministic execution. Deterministic execution is important for those wishing to create games with [deterministic lockstep](https://gafferongames.com/post/deterministic_lockstep/) as well as high-fidelity simulations. - -Currently there is no official way to introduce randomness in bevy, a plugin, or an app. Other engines such as [Godot](https://docs.godotengine.org/en/stable/tutorials/math/random_number_generation.html), [Unity](https://docs.unity3d.com/ScriptReference/Random.html), and [Unreal](https://docs.unrealengine.com/4.27/en-US/BlueprintAPI/Math/Random/) include engine support for randomness. - - -1 For example, 16 of bevy's examples currently use `rand` -
-
-examples/games/contributors.rs
-examples/games/alien_cake_addict.rs
-examples/async_tasks/external_source_external_thread.rs
-examples/async_tasks/async_compute.rs
-examples/ecs/iter_combinations.rs
-examples/stress_tests/transform_hierarchy.rs
-examples/stress_tests/many_lights.rs
-examples/animation/custom_skinned_mesh.rs
-examples/stress_tests/bevymark.rs
-examples/stress_tests/many_sprites.rs
-examples/ecs/parallel_query.rs
-examples/ecs/component_change_detection.rs
-examples/ecs/ecs_guide.rs
-examples/app/random.rs
-crates/bevy_ecs/examples/resources.rs
-crates/bevy_ecs/examples/change_detection.rs
-
-
- - -## User-facing explanation - -### Overview - -Games often use randomness as a core mechanic. For example, card games generate a random deck for each game and killing monsters in an RPG often rewards players with a random item. While randomness makes games more interesting and increases replayability, it also makes games harder to test and prevents advanced techniques such as [deterministic lockstep](https://gafferongames.com/post/deterministic_lockstep/). - -Let's pretend you are creating a poker game where a human player can play against the computer. The computer's poker logic is very simple--when the computer has a good hand, it bets all of its money. To make sure the behavior works, you write a test to first check the computer's hand and if it is good confirm that all its money is bet. If the test passes does it ensure the computer behaves as intended? Sadly, no. - -Because the deck is randomly shuffled for each game--without doing so the player would already know the card order from the previous game--it is not guaranteed that the computer player gets a good hand and thus the betting logic goes unchecked. While there are ways around this--a fake deck that is not shuffled, running the test many times to increase confidence, breaking the logic into units and testing those--it would be very helpful to have randomness as well as a way to make it _less_ random. - -Luckily, when a computer needs a random number it doesn't use real randomness and instead uses a [pseudorandom number generator](https://en.wikipedia.org/wiki/Pseudorandom_number_generator). Popular Rust libraries containing pseudorandom number generators are [`rand`](https://crates.io/crates/rand) and [`fastrand`](https://crates.io/crates/fastrand). - -Pseudorandom number generators require a source of [entropy](https://en.wikipedia.org/wiki/Entropy) called a [random seed](https://en.wikipedia.org/wiki/Random_seed). The random seed is used as input to generate numbers that _appear_ random but are instead in a specific and deterministic order. For the same random seed, a pseudorandom number generator always returns the same numbers in the same order. - -For example, let's say you seed a pseudorandom number generator with `1234`. You then ask for a random number between `10` and `99` and the pseudorandom number generator returns `12`. If you run the program again with the same seed (`1234`) and ask for another random number between `1` and `99`, you will again get `12`. If you then change the seed to `4567` and run the program, more than likely the result will not be `12` and will instead be a different number. If you run the program again with the `4567` seed, you should see the same number from the previous `4567`-seeded run. - -There are many types of pseudorandom number generators each with their own strengths and weaknesses. Because of this, Bevy does not include a pseudorandom number generator. Instead, the `bevy_entropy` plugin includes a source of entropy to use as a random seed for your chosen pseudorandom number generator. The plugin can be completely disabled if no source of entropy is required, the default entropy from the OS can be used if randomness is needed but deterministic execution is not, or a world seed can be specified for deterministic random number generation. - -Note that Bevy currently has [other sources of non-determinism](https://github.com/bevyengine/bevy/discussions/2480) unrelated to pseudorandom number generators. - -### Usage - -The `bevy_entropy` plugin ships with Bevy and is enabled by default. If you do not need randomness, you may [disable the plugin](https://docs.rs/bevy/latest/bevy/app/struct.PluginGroupBuilder.html#method.disable) or use Bevy's [minimal set of plugins](https://docs.rs/bevy/latest/bevy/struct.MinimalPlugins.html). - -When enabled, `bevy_entropy` provides one world resource: `Entropy`. `Entropy` is then used as the seed for the pseudorandom number generator of your choosing. A complete example leveraging the [`StdRng`](https://docs.rs/rand/latest/rand/rngs/struct.StdRng.html) pseudorandom number generator from the [`rand`](https://crates.io/crates/rand) crate can be found in https://github.com/bevyengine/bevy/tree/main/examples. - -The default source of world entropy [`Entropy::default()`] is non-deterministic and seeded from the operating system. It is guarenteed to be suitable for [cryptographic applications](https://en.wikipedia.org/wiki/Pseudorandom_number_generator#Cryptographic_PRNGs) but the actual seeding mechanism is an implementation detail that will change over time. - -You may choose to determinstically seed your own world entropy via [`Entropy::from`]. The seed you choose may have security implications or influence the distribution of the resulting random numbers. See https://rust-random.github.io/book/guide-seeding.html for more details about how to pick a "good" random seed for your needs. - -Depending on your game and the type of randomness you require, when specifying a seed you will normally do one of the following: - -1. Get a good random seed out-of-band and hardcode it in the source. -2. Dynamically call to the OS and print the seed so the user can rerun deterministically. In games like [Factorio](https://www.factorio.com/) sharing random seeds is encouraged and supported. -3. Dynamically call to the OS and share the seed with a server so the client and server deterministically execute together. -4. Load the seed from a server so the client and server deterministically execute together. - - -## Implementation strategy - -https://github.com/bevyengine/bevy/pull/2504 - -- The PR includes `rand` as a dependency but it is not in public API. We might be able to slim it down to [`getrandom`](https://crates.io/crates/getrandom). - - -## Drawbacks - -- This may not be general enough to include in Bevy. -- This may not be general enough to be on by default. -- Includes `rand` as a dependency - - But not in public API. - -## Rationale and alternatives - -- Why is this design the best in the space of possible designs? - - It gives flexibility for PRNG crates while also enforcing standards on the ecosystem. - - It defaults to what someone would resonably expect if they wrote it themselves. - - It defaults to something safe (suitable for cryptographic functions), removing a possible footgun. -- What other designs have been considered and what is the rationale for not choosing them? - - We could go higher and expose an API closer to `rand`. This is what [Godot](https://docs.godotengine.org/en/stable/tutorials/math/random_number_generation.html), [Unity](https://docs.unity3d.com/ScriptReference/Random.html), and [Unreal](https://docs.unrealengine.com/4.27/en-US/BlueprintAPI/Math/Random/) do. We are not experts in PRNG API design, and while `rand` is clearly the most popular random crate in the Rust ecosystem we don't currently want to tie bevy to any particular API in case something better emerges. - - We could go lower and merely expose a static `WorldSeed`. I'm worried about what the default would be and forcing a seed at world creation feels heavyweight. - - We could default to a faster PRNG rather than a safer one. I wanted folks to fall into the pit of success. -- What objections immediately spring to mind? How have you addressed them? - - `rand` is too heavy of a dependency. - - This should not be in core. -- What is the impact of not doing this? - - There is no chance of the Bevy ecosystem supporting deterministic execution. -- Why is this important to implement as a feature of Bevy itself, rather than an ecosystem crate? - - There needs to be one true way to keep the ecosystem coherant. - -## Prior art - -* https://github.com/bevyengine/bevy/discussions/2480 -* https://github.com/bevyengine/bevy/discussions/1678 - -Other engines such as [Godot](https://docs.godotengine.org/en/stable/tutorials/math/random_number_generation.html), [Unity](https://docs.unity3d.com/ScriptReference/Random.html), and [Unreal](https://docs.unrealengine.com/4.27/en-US/BlueprintAPI/Math/Random/) include engine support for randomness (with a higher-level API). - -* https://towardsdatascience.com/how-to-use-random-seeds-effectively-54a4cd855a79 -* https://www.whatgamesare.com/determinism.html -* https://gafferongames.com/post/deterministic_lockstep/ -* https://ruoyusun.com/2019/03/29/game-networking-2.html -* https://arxiv.org/abs/2104.06262 - -## Unresolved questions - -- What parts of the design do you expect to resolve through the RFC process before this gets merged? - - Do we want this? - - What about `rand`? - - Is `Entropy` naming too obscure? - - How do we deal with security vs speed tradeoff here? - - Is this the right level of abstraction? Lower? Higher? -- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? - - Built-in higher-level randomness APIs - - Entropy per-system rather than per-world - -## Future possibilities - -- Higher-level randomness APIs -- Entropy per-system rather than per-world From 05a3cfb534132f644c7de4b63a1074d5900e08c7 Mon Sep 17 00:00:00 2001 From: Christian Legnitto Date: Tue, 17 May 2022 13:07:06 -0400 Subject: [PATCH 11/15] Review suggestion Co-authored-by: Alice Cecile --- rfcs/55-deterministic_rng.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfcs/55-deterministic_rng.md b/rfcs/55-deterministic_rng.md index 7de6248e..fcd82000 100644 --- a/rfcs/55-deterministic_rng.md +++ b/rfcs/55-deterministic_rng.md @@ -50,7 +50,7 @@ Pseudorandom number generators require a source of [entropy](https://en.wikipedi For example, let's say you seed a pseudorandom number generator with `1234`. You then ask for a random number between `10` and `99` and the pseudorandom number generator returns `12`. If you run the program again with the same seed (`1234`) and ask for another random number between `1` and `99`, you will again get `12`. If you then change the seed to `4567` and run the program, more than likely the result will not be `12` and will instead be a different number. If you run the program again with the `4567` seed, you should see the same number from the previous `4567`-seeded run. -There are many types of pseudorandom number generators each with their own strengths and weaknesses. Because of this, Bevy does not include a pseudorandom number generator. Instead, the `bevy_entropy` plugin includes a source of entropy to use as a random seed for your chosen pseudorandom number generator. The plugin can be completely disabled if no source of entropy is required, the default entropy from the OS can be used if randomness is needed but deterministic execution is not, or a world seed can be specified for deterministic random number generation. +There are many types of pseudorandom number generators each with their own strengths and weaknesses. Because of this, Bevy does not include a pseudorandom number generator. Instead, the `bevy_entropy` plugin includes a source of entropy to use as a random seed for your chosen pseudorandom number generator. The plugin can be completely disabled if no source of entropy is required, the default entropy from the OS can be used if randomness is needed but deterministic execution is not, or a **world seed** can be specified for deterministic random number generation. Note that Bevy currently has [other sources of non-determinism](https://github.com/bevyengine/bevy/discussions/2480) unrelated to pseudorandom number generators. From 3f42c8661470b002016d4b49a1a19889b22c1ec6 Mon Sep 17 00:00:00 2001 From: James Liu Date: Mon, 30 May 2022 14:12:47 -0700 Subject: [PATCH 12/15] Start #54: Global Task Pools (#54) --- rfcs/54-global-task-pools.md | 118 +++++++++++++++++++++++++++++++++++ 1 file changed, 118 insertions(+) create mode 100644 rfcs/54-global-task-pools.md diff --git a/rfcs/54-global-task-pools.md b/rfcs/54-global-task-pools.md new file mode 100644 index 00000000..984ad60b --- /dev/null +++ b/rfcs/54-global-task-pools.md @@ -0,0 +1,118 @@ +# Feature Name: `global-task-pools` + +## Summary +Globally initialize the newtyped TaskPools (ComputeTaskPool, +AsyncComputeTaskPool, IoTaskPool) as statics and provide static accessors to +them. + +## Motivation +Bevy's TaskPools are currently passed around as ref-counted shared references, +and accessed primarily as ECS resources. This presents an ergonomics challenge +when integrating third-party crates without direct access to the ECS systems or +World. An example of this [Backroll's use of the TaskPools][backroll] +where the references must be passed and clone repeatedly. + +This also presents an ergonomics issue for existing Bevy APIs, like +`par_for_each(_mut)`. All internally parallel systems must remember to add +a `Res` parameter. This also presents a (rather small) footgun +for new users as to which task pool resource they should use, and choosing the +wrong one will give the wrong performance characteristics due to improper thread +allocation, or higher thread contention from the threads already in the wrong +pool. + +Multitenant projects (processes with multiple Bevy apps running concurrently) +currently spin up new task pools for every top-level App in the process. The +number of threads per CPU core scales with the number of Apps in the process, +which may cause OS level thrashing from the context switches. This is most +commonly seen in server environments where processes may spin up and spin down +Apps in response to players starting and ending multiplayer matches. + +## User-facing explanation +Instead of being publicly constructable, the newtyped TaskPools will become +static singletons, initialized on their first access. + +Users can get a static reference to a task pool via the `get` associated +function (i.e. `ComputeTaskPool::get`), which will initialize the TaskPool with +the default configuration if it hasn't already been initialized. + +For more explicit control over initialization, a corresponding `init` associated +function will take a task pool configuration and initialize the task pool, or +panic if the pool is already initialized. By convention, libraries, including +Bevy itself, should just use `get` and defer configuration to the final end +user's configuration. + +## Implementation strategy +Prototype implementation: https://github.com/bevyengine/bevy/pull/2250 + +This implementation is rather simple and can be broken down into the following +smaller tasks: + + - The public construction of the newtyped TaskPools should be disabled/removed. + - New static variables holding [`OnceCell`][once-cell] wrappers around each + task pool will be added. + - Move the default task pool initialization configuration from `bevy_core` into + `bevy_tasks`. + - Implement `get` and `init` functions as described above for each type. + - Replace `Res<*TaskPool>` accesses with `*TaskPool::get` calls. + - Optional: Remove the internal `Arc`s as the task pools to remove one level of + indirection. + - Optional: Remove the `TaskPool` parameters from `par_for_each(_mut)` on Query + +## Drawbacks +Reconfiguring the task pools is now impossible under this initial design. +Calling `init` a second time will panic. This could be remedied by +moving reconfiguration into the TaskPool itself. The current way of +reconfiguring the task pools is to construct a new task pool and replace the +resource. However, this does not properly clean up the TaskPools if they +internally are self-referential: a long-lived, or indefinitely looping scheduled +task that contains a reference to the same task pool will keep the TaskPool +alive until all tasks terminate. Moving this reconfiguration as a first class +operation on TaskPool may be required to support these use cases. + +Multitenant processes now share the same threads for all apps. Highly active or +long running apps may starve out others. Whether or not this is preferable over +OS-level thread thrashing is likely use case dependent. + +## Rationale and alternatives +This is a significant improvement in terms of ergonomics over the status quo. +This API design removes the need to have a task pool parameter on systems, +`Query`, and `QueryState` types. It also removes integration friction for third +party crates that aren't directly integrating with the ECS state. + +### Alternative: One Global Task Pool, Enum Tasks +This alternative proposes to remove the newtyped task pools entirely and just +have one global task pool instead. To get the same scheduling guarantee, tasks +would internally state which of the three task types it is. This would allow the +executors to independently schedule and prioritize tasks as they currently are, +but without forcing the user to make that decision, or at least allow users to +choose a common default. + +This is still technically possible, but would require much more effort to +implement. + +## Prior art +Other async executors also provide global versions of their executors: + + - [tokio][tokio]'s runtime is global by default + - [async-global-executor][async-global-executor] + +Other language's async runtimes also largely use globals as well: + + - Python's [asyncio event loop][asyncio-event-loop] is thread-local. Fetched + via `asyncio.get_event_loop()`. + - C#'s [managed-thread-pool][csharp-pool] is used for it's async runtime + - Go spins up a 1:1 threadpool that all of it's async goroutines run on, + scheduling is global and requires no reference. + +Other game engines have a built in global threadpool used for their form of +multithreading: + + - Unity has dedicated threads for rendering, audio, and a dedicated worker + thread pool for running multihreaded jobs. + - Unreal: ??? + - Godot: ??? + +[backroll]: https://github.com/HouraiTeahouse/backroll-rs/blob/219f4b6fda27250a7c4f7928a381418faebd5544/backroll/src/backend/p2p.rs +[once-cell]: https://docs.rs/once_cell/latest/once_cell/sync/struct.OnceCell.html +[asyncio-event-loop]: https://docs.python.org/3/library/asyncio-eventloop.html +[csharp-pool]: https://docs.microsoft.com/en-us/dotnet/standard/threading/the-managed-thread-pool From fb26c3d0a84fb3d3fc717089c8a17c4aa3f9450a Mon Sep 17 00:00:00 2001 From: James Liu Date: Mon, 30 May 2022 15:14:47 -0700 Subject: [PATCH 13/15] Consistent Hierarchies (#53) --- rfcs/53-consistent-hierarchy.md | 453 ++++++++++++++++++++++++++++++++ 1 file changed, 453 insertions(+) create mode 100644 rfcs/53-consistent-hierarchy.md diff --git a/rfcs/53-consistent-hierarchy.md b/rfcs/53-consistent-hierarchy.md new file mode 100644 index 00000000..c939cec1 --- /dev/null +++ b/rfcs/53-consistent-hierarchy.md @@ -0,0 +1,453 @@ +# Feature Name: `consistent-hierarchy` + +## Summary +The current transform hierarchy operations currently allow for it to remain in a +inconsistent state. This RFC proposes a few changes to make it globally +consistent at any given time, as well as make it easier to support non-Tranform +use cases. + +## Motivation +Hierarchies are present in all parts of game development. The one that +immediately comes to mind is the Transform hierarchy, where children exist in the +local space of their parents. However, this concept is can be more general than +transform hierarchies: + + - UI elements have their own parent-sibling-child relationships that are not + strictly expressible in 3D affine transforms. + - Animations are usually bound to a bone hierarchy that may or may not involve + Transforms, and is typically queried based on paths within the hierarchy. + - Certain attributes intuitively should be inherited from parent to child. For + example, hiding a parent via the `Visibility` component typically means the + children of that parent (and all of their descendants) are also hidden. Many + of these properties are not directly tied to transforms or their propagation. + However, many of these use cases also share a similar pattern of detecting + changes and propagating them either down to their descendants or bubbling up a + message through their ancestors. + +All of these cases involve mutating, querying, and traversing these large trees +or forests of linked entities. Throughout each of these operations, it's +imperative that Bevy provides a globally consistent view of the +hierarchy/hiearchies. + +Many of these operations are both abundantly common and moderately expensive. +Queries and traversals can be linear in the number of children and involve heavy +random access patterns which can be detrimental to game performance. + +## Background +In this RFC, we will be using the following terms repeatedly: + + - **Parent:** this refers to the upper level Entity relative to their children. Each + Entity can either have zero or one parents, but may have zero or more + children. + - **Child:** this refers to a lower level Entity relative to their parent. Any child + will have exactly one parent. Children can have children of their own. + - **Sibling:** a child that shares the same parent as another different entity. + - **Roots:** These are entities without parents. May or may not have children. + - **Leaves:** These are entities without children. May or may not have a parent. + +Currently, there are two components under `bevy_transform` that describe Bevy's +current hierarchy: `Parent` and `Children`. Adding or altering the `Parent` +component will not immediately manifest in an update to `Children`. + +A dedicated transform hierarchy maintenance system is run every game tick to +update `Children` to match the actual Entities that point to said Entity as a +Parent. separate `PreviousParent` component is used to keep track of which +entity was the entity's parent before it's current one. This is created, managed, +and removed by the maintenance system. This introduces state where both +components on different entities are out of sync, and relies on the maintenance +system to make it eventually consistent. + +In the meantime, querying for either results in a inconsistent state of the +world, and is the subject of much frustration from those not intimately familiar +with the inner workings of the system. + +## User-facing explanation +Both of the current components' public interface will be made *read-only*. +Structural mutations to the hierarchy: de-parenting, re-parenting, and moving +children from one parent to another, etc. can only be done via commands. This +defers all changes to the hierarchy until the next command buffer flush. +This enforces a globally consistent view at any given time. Delaying any +modifications to a global synchronization point, much like component removal. +Both components will not publicly constructible and must be made via the +dedicated hierarchy management commands. + +`PreviousParent` will also be removed and replaced with `ChildAdded`, +`ChildRemoved` and `ChildMoved` events instead. These will signal changes in the +hierachy instead of relying on change detection on `Parent` and `PreviousParent`. + +## Implementation strategy +This design attempts to minimize the changes made to the current design while +addressing the issues enumerated above. The core changes to be made are as +follows: + + - Make `Parent` and `Children` (mostly) publicly immutable. + - Make mutations to the hierarchy rely solely on commands. + - Update hierarchy commands to automatically update both `Parent` and + `Children` to remain consistent with each other. + - Remove the `parent_update_system`. + - Remove `PreviousParent`. + - Register several hierarchy events that fires whenever time a new child is + added or removed from a parent. + - Create a `HiearchyQuery` custom SystemParam for more easily iterating over + components in the hierarchy. + +This change will repurpose the following existing `EntityCommand` extension +functions for this purpose: + + - `add_child` + - `push_children` + - `insert_children` + - `remove_child` (this will be added as a wrapper around `remove_children`) + - `remove_children` + +An additional `set_parent(Option)` command should also be added as a +child-side way of altering the hierarchy, defaulting to adding the entity as the +last child to the provided parent if provided, and detaching the child from it's +parent if `None`. + +### HierarchyQuery +An extension to this design is to create dedicated `HierarchyQuery`, which +is a custom `SystemParam` that wraps multiple `Query` objects to simplify +querying for items in the hierarchy: + + - `roots(_mut)` - returns an iterator over entities that match the provided + query parameters that do not have a parent. + - `parent(_mut)` - returns single result from the parent of an entity. + - `iter_children(_mut)` - returns an iterator over `(parent, child)` results + from a query parameter. This does not return Children of Children. + - `iter_descendants(_mut)` - returns an iterator over `(parent, child)` results + from a query parameter. This does a depth-first traversal over the hierarchy + starting at some entity. + - `iter_ancestors(_mut)` - returns an iterator over `(parent, child)` results + from a query parameter. This bubbles up queries from the bottom to the top + starting at some entity. + +As suggested by the generic parameters, the query results and filters are +identical to the ones used by a normal Query or QueryState. + +The main motivation for this is to make creating systems like +`transform_propagate_system` easier. These queries can be generically used for +any system that needs some form of hierachy traversal. These queries can be quite +expensive to run, so the documentation for them should reflect this and +discourage misuing it. + +### HierarchyEvent +One new event should be added here, with enum variants corresponding to a +specific change in the hierarchy: + + - `ChildAdded` - Fired when a previous root entity has been added as a child to + a entity in the hierarchy. + - `ChildRemoved` - Fired when an entity is removed from the hierarchy and is now + a root entity. + - `ChildMoved` - Fired when an child is moved from one parent in the hierarchy + to another. + +These can be used to detect changes where `PreviousParent` is currently used. +Each event is exclusive, meaning that any change only generates one event. A move +will not fire additional separate `ChildAdded` and `ChildRemoved` events. + +Only one event type is used here to ensure that hierarchy alterations are +properly sequenced. + +As hierarchy edits typically happen in bursts (i.e. scene loads). There may be +large number of these events generated at once. To reclaim this memory, it may be +necessary to add support for automatic garbage collection to events. + +```rust +#[derive(Clone)] +pub enum HierarchyEvent { + ChildAdded { + parent: Entity, + child: Entity, + }, + ChildRemoved { + parent: Entity, + child: Entity, + }, + ChildMoved { + parent: Entity, + previous_parent: Entity, + new_parent: Entity, + } +} +``` + +### `bevy_hierarchy`-`bevy_transform` split. +These hierarchy types should be moved to a new `bevy_hierarchy` crate to avoid +requiring compiling the hierachy alongside the default "f32-based affine +transform system in Euclidean space". This should allow users to create their own +transform systems (i.e. f64-based transforms, fixed-point transforms, +non-Euclidean spaces) while also being able to leverage the same +parent-sibling-child hierarchies currently found in `bevy_transform`. This should +also allow us to internally leverage the same decoupling too for specialized +transforms for 2D and UI. + +## Benefits + + - This rocks the boat the least compared to the following alternatives and + largely provides the same benefits. + - This design has (nearly) guarenteed global consistency. The hierarchy cannot + be arbitrarily mutated in inconsistent between synchronization points. + Changes are deferred to a until the end of the current stage instead of a + single global maintainence system. + - The cache locality when iterating over the immediate children of an entity is + retained. This saves an extra cache miss per entity when traversing the + hierarchy. + - The removal of `PreviousParent` should reduce the archetype fragmentation of + the current solution. + - Any update to the hierarchy is "atomic" and is `O(n)` in the number of + children an entity has. For any practical hierarchy, this will typically be + a very small n. + - Existing query filters still work as is. For example, `Without` can + still be used to find root entities, and `Changed` can be used to + find any entity with either new children, removed children, or reordered + children. + - The hierarchy is decoupled from the base transform system, allowing both Bevy + developer and third-party crates to make their own independent transform + system without compiling the default transform system. + +## Drawbacks + + - The fact that the hierarchy is not optimally stored in memory is still an + issue, and all hiearchy traversals require heavy random access into memory. + - Updates are still not immediately visible within a stage and deferred until + the next command buffer flush. + - Hierarchy updates are now single threaded. Commands can be generated from + multiple systems, [or from multiple threads in the same + system](https://github.com/bevyengine/bevy/pull/4749) at the same time, but + they now require exclusive access to apply. + - Each of the changed commands do a lot more work before and are signifgantly + more branch heavy. This could negatively impact the performance of + command-heavy workloads (i.e. scene loading) + - The commands applied for controlling the hierarchy are computationally more + expensive, requiring more branching access and more `World::get` calls. + - Some potential implementations of this hierarchy requires normal ECS commands + to be aware of the hierarchy. Using `EntityCommands::remove` to remove + `Children` or `Parent` will break the invariants of the system without some + hooks to enforce the invariants on removal. This is partially mitigatable via + documentation until hooks or relations lands. + +## Rationale and alternatives +The primary goal of this proposal is to get rid of the hierarchy maintenance +system and the consistency issues that come with it's design, which it delivers +on. + +### Alternative: Hierarchy as a Resource +This would store the resource of the entire world as a dedicated Resource +instead of as components in normal ECS storage. A prototypical design for this +resource would be: + +```rust +pub struct Hierarchy { + relations: Vec<(Option, Vec)>, + entities: Vec, + indices: SparseArray, +} +``` + +This would be strikingly similar to the `SparseSet` used in `bevy_ecs`. The main +difference here is that the parallel `Vec`s are roughly kept in topological order. +Inserting should be `O(1)` as it just updates the new parent. Removing should be +`O(n)` with the number of children of that `Entity`. At least once per frame, a +system should run a topological sort over the resource if it's dirty. + +An alternative version of this hierarchy resource only stores parents. This may +speed up iteration as individual relations are smaller during swaps, but finding +children in the hierarchy is O(n) over the entire world. If the depth of each +entity within the hierarchy, and the hierarchy is kept sorted in depth order this +can be kept to O(n) in the next depth. + +#### Benefits + + - Iteration in hierarchy order (parents first) is entirely linear. This may make + transform and other parent-reliant propagation signfigantly faster. + - Finding roots in the sorted hierarchy is both linear and compacted tightly in + memory. + - Removes `Parent` and `Children` ECS components entirely. Could further reduce + archetype fragmentation. The hierarchy is stored densely in a single location + and is not spread between multiple archetypes/tables. + - Can still be used with the command-driven interface proposed in this design + instead of using ECS storage. + +#### Drawbacks + + - Merging and splitting hierarchies loaded from scenes can be quite difficult to + maintain. + - The non-hierarchy components are still stored in an unordered way in ECS data. + Iterating over the hierarchy will still incur random access costs for anything + non-trivial. + - Requires a dedicated topological sort system. Mitigated by only sorting when + the hierarchy is dirty, or simply tracking which entities in the hierarchy are + dirty. + - Editing the hierarchy requires exclusive access to the resource, which can + will serialize all systems editing the hierarchy. This also prevents internal + parallelism (i.e. via `Query::par_for_each`) as `ResMut` cannot be cloned. + This can be partially mitigated by deferring edits via commands, as proposed + in this RFC. + +### Alternative: Linked List Hierarchy +Instead of using two separate components to manage the hierarchy, this +alternative aims to have only one: `Hierarchy`. This hierarchy component provides +*read-only* access into an entity's immediate hierarchial relationships: its +parent, previous sibling, next sibling, and it's first child. All of these may +return `Option`, signalling that such relationships do not exist. This +creates a single component that creates a top-down forest of entity trees, where +each entity forms a doubly-linked list of siblings. It may also cache infromation +about the component's position within the hierarchy like it's depth. + +```rust +#[derive(Component)] +pub struct Hierarchy { + parent: Option, + prev: Option, + next: Option + first_child: Option, +} +``` + +Like the main proposal here, there are no public constructors or mutation APIs, +relying only on commands to mutate the internals. + +#### Benefits + + - Hierarchy updates are all `O(1)`, and can be maintained within a system. Does + not require commands to ensure consistency. + - The Hierarchy component is smaller in ECS storage than the current + Parent/Children combination. With Entity-niching, a `Hierarchy` component is + only 32 bytes in memory, compared to the 64 + 8 used by the current two + components. + - Requires less archetype fragmentation due to only requiring one component + instaed of two. + - Getting siblings does not require a parent Query::get call. + +#### Drawbacks + + - Iterating over children requires `O(n)` Query::get calls. + - Despawning a child without updating it's sibilings causes all subsequent + children to be in a broken state. This exacerbates the lack of hooks, and + would require query access in hooks to fix up. + - Change detection and With/Without filters no longer work as is. Will need + dedicated ZST marker components to get the same filtering capacity. + +### Alternative: HashSet `Children` +The internals of `Children` could be replaced with a `HashSet`. +operations close to `O(1)`, versus `O(n)` operations against a `SmallVec`. + +#### Benefits +This makes all operations against `Children` O(1). Could speed up a number of +hierarchy related commands. + +#### Drawbacks + + - We lose any ability to order children. This is critically important for + certain use cases like UI. + - We lose cache locality for entities with few children. Adding a single child + forces an allocation. + +### Alternative: Hierarchical ECS Storage +This involves directly adding a third ECS storage option. In this solution, a +data-oriented SoA structure of split BlobVec columns is used, virtually identical +to the way Tables are currently stored. However, entities are stored in +depth-first order, and an implicit `Option` parent reference is stored +alongisde every Entity. Hierarchy roots are stored at the beginning of the +slices, and leaves near the end. When a entity's components are mutated, it's +marked "dirty" by moving it to the end of the slice. A separate "first dirty" +index is kept to track a list of all dirty hierarchy entities as a form of change +detection. Finally, when iterating over the storage, if a child is found before +it's parent, it's swapped with it's parent to enforce the depth-first invariant. + +This is very similar to the "Hierarchy as a Resource" alternative, but + +#### Benefits + + - Iteration in hierarchy order (parents first) is entirely linear. This may make + transform and other parent-reliant propagation signfigantly faster. + - Unlike the hierarchy as a resource, this guarentees linearity of all iterated + components, not just the hierarchy relations itself. + +#### Drawbacks +This method is very write heavy. Components are moved whenever there is a +mutation, and shuffled around to ensure that the topologically ordered iteration +invariant is maintained. This may be impractical for hierarchy dependent +components that are very big in size. + +The need to mutate and shuffle the storage during iteration also means parallel +iteration cannot be supported on it, so going wide might not be possible. Even +read-only queries against this storage requires exclusive access to the table. + +### Alternative: ECS Relations +ECS relations can be used to represent the exact same links in the hierarchy, and +might be usable as a different backing for the hierarchy, provided the same +consistency guarentees. + +#### Benefits +Relations are generic, expressing more than just hierarchical relations, but any +link between multiple entities. The exact benefits and drawbacks relative to this +design is reliant on the exact implementation. + +## Prior art +The original design that this proposal is based on is this 2014 article from +[bitsquid +developers](https://bitsquid.blogspot.com/2014/10/building-data-oriented-entity-system.html). +The memory layout sugggested is awfully similar to bevy's ECS tables. However, +components are regularly swapped based on their dirty status. Placing dirty +entities near the end of each contiguous slice. The hierarchical ECS storage +alterantive aims to implement this. + +A similar [blog +post](https://blog.molecular-matters.com/2013/02/22/adventures-in-data-oriented-design-part-2-hierarchical-data/) +from 2013 suggests a similar approach. + +Unity offers similar APIs to the `HierarchyQuery`, with similar performance +penalties for heavy use. + + - [GameObject.GetComponentsInChildren](https://docs.unity3d.com/ScriptReference/GameObject.GetComponentsInChildren.html) + - [GameObject.GetComponentsInParent](https://docs.unity3d.com/ScriptReference/GameObject.GetComponentsInParent.html) + +## Future Work +One of the immediate pieces of future work here is to patch up the potential +footgun of using `EntityCommands::remove` on either `Parent` or `Children` via +add/remove/changed hooks. This is currently mitigatable via documentation, but a +complete solution prevents users from breaking the invariant in the first place. + +### Dedicated RectTransform and Transform2D systems +Transform/GlobalTransform is not particularly well suited for encoding the +hierarchy of UIs and 2D-only scenes. By splitting out the hierarchy from the +exsisting transform system, dedicated transform types and systems for these two +different types of hierarchies. This could also enable further parallelism by +allowing each transform system to independently handle propagation. + +### Multiple Hierarchies via Type Markers +It may be possible to include type markers as a part of the hierarchy components +to differentiate multiple hierachies from each other. This would allow an entity +to be part of multiple hierarchies at the same time. + +```rust +#[derive(Component)] +pub struct Parent { + parent: Entity, + _marker: PhantomData, +} + +#[derive(Component)] +pub struct Children { + children: SmallVec<[Entity, 8]>, + _marker: PhantomData, +} +``` + +This allows us to directly tie hierarchies to the components they're targetting. +Instead of a general `Parent`, a `Parent` declares it's meant +for `bevy_transform`'s transform components. + +It also allows multiple overlapping hierarchies to be made that may differ from +each other depending on the use case. + +However, this may be quite confusing to show in an editor and is reasonably +complex to understand from a user perspective. A compelling use case might be +needed to justify this kind of change. + +## Unresolved questions + + - How do we resolve ordering problems for the same entity? This ties into the + question of Command determinism. From 18c93af71786f19c76a96e3bf97b431cd612bdc2 Mon Sep 17 00:00:00 2001 From: Christian Legnitto Date: Sun, 3 Jul 2022 18:55:56 -0700 Subject: [PATCH 14/15] Review feedback * remove redundant paragraph * Fix bare urls * Break on sentence boundaries. * Update with `rand` changes * Format --- rfcs/55-deterministic_rng.md | 70 +++++++++++++++++++++++------------- 1 file changed, 46 insertions(+), 24 deletions(-) diff --git a/rfcs/55-deterministic_rng.md b/rfcs/55-deterministic_rng.md index fcd82000..31b9e9ca 100644 --- a/rfcs/55-deterministic_rng.md +++ b/rfcs/55-deterministic_rng.md @@ -38,31 +38,52 @@ crates/bevy_ecs/examples/change_detection.rs ### Overview -Games often use randomness as a core mechanic. For example, card games generate a random deck for each game and killing monsters in an RPG often rewards players with a random item. While randomness makes games more interesting and increases replayability, it also makes games harder to test and prevents advanced techniques such as [deterministic lockstep](https://gafferongames.com/post/deterministic_lockstep/). +Games often use randomness as a core mechanic. +For example, card games generate a random deck for each game and killing monsters in an RPG often rewards players with a random item. +While randomness makes games more interesting and increases replayability, it also makes games harder to test and prevents advanced techniques such as [deterministic lockstep](https://gafferongames.com/post/deterministic_lockstep/). -Let's pretend you are creating a poker game where a human player can play against the computer. The computer's poker logic is very simple--when the computer has a good hand, it bets all of its money. To make sure the behavior works, you write a test to first check the computer's hand and if it is good confirm that all its money is bet. If the test passes does it ensure the computer behaves as intended? Sadly, no. +Let's pretend you are creating a poker game where a human player can play against the computer. +The computer's poker logic is very simple--when the computer has a good hand, it bets all of its money. +To make sure the behavior works, you write a test to first check the computer's hand and if it is good confirm that all its money is bet. +If the test passes does it ensure the computer behaves as intended? Sadly, no. -Because the deck is randomly shuffled for each game--without doing so the player would already know the card order from the previous game--it is not guaranteed that the computer player gets a good hand and thus the betting logic goes unchecked. While there are ways around this--a fake deck that is not shuffled, running the test many times to increase confidence, breaking the logic into units and testing those--it would be very helpful to have randomness as well as a way to make it _less_ random. +Because the deck is randomly shuffled for each game--without doing so the player would already know the card order from the previous game--it is not guaranteed that the computer player gets a good hand and thus the betting logic goes unchecked. +While there are ways around this--a fake deck that is not shuffled, running the test many times to increase confidence, breaking the logic into units and testing those--it would be very helpful to have randomness as well as a way to make it _less_ random. -Luckily, when a computer needs a random number it doesn't use real randomness and instead uses a [pseudorandom number generator](https://en.wikipedia.org/wiki/Pseudorandom_number_generator). Popular Rust libraries containing pseudorandom number generators are [`rand`](https://crates.io/crates/rand) and [`fastrand`](https://crates.io/crates/fastrand). +Luckily, when a computer needs a random number it doesn't use real randomness and instead uses a [pseudorandom number generator](https://en.wikipedia.org/wiki/Pseudorandom_number_generator). +Popular Rust libraries containing pseudorandom number generators are [`rand`](https://crates.io/crates/rand) and [`fastrand`](https://crates.io/crates/fastrand). -Pseudorandom number generators require a source of [entropy](https://en.wikipedia.org/wiki/Entropy) called a [random seed](https://en.wikipedia.org/wiki/Random_seed). The random seed is used as input to generate numbers that _appear_ random but are instead in a specific and deterministic order. For the same random seed, a pseudorandom number generator always returns the same numbers in the same order. +Pseudorandom number generators require a source of [entropy](https://en.wikipedia.org/wiki/Entropy) called a [random seed](https://en.wikipedia.org/wiki/Random_seed). +The random seed is used as input to generate numbers that _appear_ random but are instead in a specific and deterministic order. +For the same random seed, a pseudorandom number generator always returns the same numbers in the same order. -For example, let's say you seed a pseudorandom number generator with `1234`. You then ask for a random number between `10` and `99` and the pseudorandom number generator returns `12`. If you run the program again with the same seed (`1234`) and ask for another random number between `1` and `99`, you will again get `12`. If you then change the seed to `4567` and run the program, more than likely the result will not be `12` and will instead be a different number. If you run the program again with the `4567` seed, you should see the same number from the previous `4567`-seeded run. +For example, let's say you seed a pseudorandom number generator with `1234`. +You then ask for a random number between `10` and `99` and the pseudorandom number generator returns `12`. +If you run the program again with the same seed (`1234`) and ask for another random number between `1` and `99`, you will again get `12`. +If you then change the seed to `4567` and run the program, more than likely the result will not be `12` and will instead be a different number. +If you run the program again with the `4567` seed, you should see the same number from the previous `4567`-seeded run. -There are many types of pseudorandom number generators each with their own strengths and weaknesses. Because of this, Bevy does not include a pseudorandom number generator. Instead, the `bevy_entropy` plugin includes a source of entropy to use as a random seed for your chosen pseudorandom number generator. The plugin can be completely disabled if no source of entropy is required, the default entropy from the OS can be used if randomness is needed but deterministic execution is not, or a **world seed** can be specified for deterministic random number generation. +There are many types of pseudorandom number generators each with their own strengths and weaknesses. +Because of this, Bevy does not include a pseudorandom number generator. +Instead, the `bevy_entropy` plugin includes a source of entropy to use as a random seed for your chosen pseudorandom number generator. Note that Bevy currently has [other sources of non-determinism](https://github.com/bevyengine/bevy/discussions/2480) unrelated to pseudorandom number generators. ### Usage -The `bevy_entropy` plugin ships with Bevy and is enabled by default. If you do not need randomness, you may [disable the plugin](https://docs.rs/bevy/latest/bevy/app/struct.PluginGroupBuilder.html#method.disable) or use Bevy's [minimal set of plugins](https://docs.rs/bevy/latest/bevy/struct.MinimalPlugins.html). +The `bevy_entropy` plugin ships with Bevy and is enabled by default. +If you do not need randomness, you may [disable the plugin](https://docs.rs/bevy/latest/bevy/app/struct.PluginGroupBuilder.html#method.disable) or use Bevy's [minimal set of plugins](https://docs.rs/bevy/latest/bevy/struct.MinimalPlugins.html). -When enabled, `bevy_entropy` provides one world resource: `Entropy`. `Entropy` is then used as the seed for the pseudorandom number generator of your choosing. A complete example leveraging the [`StdRng`](https://docs.rs/rand/latest/rand/rngs/struct.StdRng.html) pseudorandom number generator from the [`rand`](https://crates.io/crates/rand) crate can be found in https://github.com/bevyengine/bevy/tree/main/examples. +When enabled, `bevy_entropy` provides one world resource: `Entropy`. +`Entropy` is then used as the seed for the pseudorandom number generator of your choosing. +A complete example leveraging the [`StdRng`](https://docs.rs/rand/latest/rand/rngs/struct.StdRng.html) pseudorandom number generator from the [`rand`](https://crates.io/crates/rand) crate can be found in the examples. -The default source of world entropy [`Entropy::default()`] is non-deterministic and seeded from the operating system. It is guarenteed to be suitable for [cryptographic applications](https://en.wikipedia.org/wiki/Pseudorandom_number_generator#Cryptographic_PRNGs) but the actual seeding mechanism is an implementation detail that will change over time. +The default source of world entropy [`Entropy::default()`] is non-deterministic and seeded from the operating system. +It is guarenteed to be suitable for [cryptographic applications](https://en.wikipedia.org/wiki/Pseudorandom_number_generator#Cryptographic_PRNGs) but the actual algorithm is an implementation detail that will change over time. -You may choose to determinstically seed your own world entropy via [`Entropy::from`]. The seed you choose may have security implications or influence the distribution of the resulting random numbers. See https://rust-random.github.io/book/guide-seeding.html for more details about how to pick a "good" random seed for your needs. +You may choose to determinstically seed your own world entropy via [`Entropy::from`]. +The seed you choose may have security implications or influence the distribution of the resulting random numbers. +See [this resource](https://rust-random.github.io/book/guide-seeding.html) for more details about how to pick a "good" random seed for your needs. Depending on your game and the type of randomness you require, when specifying a seed you will normally do one of the following: @@ -71,25 +92,23 @@ Depending on your game and the type of randomness you require, when specifying a 3. Dynamically call to the OS and share the seed with a server so the client and server deterministically execute together. 4. Load the seed from a server so the client and server deterministically execute together. - ## Implementation strategy https://github.com/bevyengine/bevy/pull/2504 - The PR includes `rand` as a dependency but it is not in public API. We might be able to slim it down to [`getrandom`](https://crates.io/crates/getrandom). - ## Drawbacks - This may not be general enough to include in Bevy. - This may not be general enough to be on by default. -- Includes `rand` as a dependency +- Includes `rand_core` and `rand_chacha` as a dependency. - But not in public API. ## Rationale and alternatives - Why is this design the best in the space of possible designs? - - It gives flexibility for PRNG crates while also enforcing standards on the ecosystem. + - It gives flexibility for PRNG crates while also enforcing standards on the ecosystem. - It defaults to what someone would resonably expect if they wrote it themselves. - It defaults to something safe (suitable for cryptographic functions), removing a possible footgun. - What other designs have been considered and what is the rationale for not choosing them? @@ -97,7 +116,7 @@ https://github.com/bevyengine/bevy/pull/2504 - We could go lower and merely expose a static `WorldSeed`. I'm worried about what the default would be and forcing a seed at world creation feels heavyweight. - We could default to a faster PRNG rather than a safer one. I wanted folks to fall into the pit of success. - What objections immediately spring to mind? How have you addressed them? - - `rand` is too heavy of a dependency. + - `rand_core` and `rand_chacha`are too heavy of a dependency. - This should not be in core. - What is the impact of not doing this? - There is no chance of the Bevy ecosystem supporting deterministic execution. @@ -106,22 +125,24 @@ https://github.com/bevyengine/bevy/pull/2504 ## Prior art -* https://github.com/bevyengine/bevy/discussions/2480 -* https://github.com/bevyengine/bevy/discussions/1678 +- https://github.com/bevyengine/bevy/discussions/2480 +- https://github.com/bevyengine/bevy/discussions/1678 Other engines such as [Godot](https://docs.godotengine.org/en/stable/tutorials/math/random_number_generation.html), [Unity](https://docs.unity3d.com/ScriptReference/Random.html), and [Unreal](https://docs.unrealengine.com/4.27/en-US/BlueprintAPI/Math/Random/) include engine support for randomness (with a higher-level API). -* https://towardsdatascience.com/how-to-use-random-seeds-effectively-54a4cd855a79 -* https://www.whatgamesare.com/determinism.html -* https://gafferongames.com/post/deterministic_lockstep/ -* https://ruoyusun.com/2019/03/29/game-networking-2.html -* https://arxiv.org/abs/2104.06262 +- https://towardsdatascience.com/how-to-use-random-seeds-effectively-54a4cd855a79 +- https://www.whatgamesare.com/determinism.html +- https://gafferongames.com/post/deterministic_lockstep/ +- https://ruoyusun.com/2019/03/29/game-networking-2.html +- https://arxiv.org/abs/2104.06262 ## Unresolved questions - What parts of the design do you expect to resolve through the RFC process before this gets merged? - Do we want this? - - What about `rand`? + - ~~What about `rand`?~~ + - `rand` was [switched to](https://github.com/bevyengine/bevy/pull/2504/commits/d255fc40b65ea358c40c71a17852cd865387b869) `rand_core` and `rand_chacha`, thanks @bjorn3! + - Are `rand_core` and `rand_chacha` small enough? - Is `Entropy` naming too obscure? - How do we deal with security vs speed tradeoff here? - Is this the right level of abstraction? Lower? Higher? @@ -133,3 +154,4 @@ Other engines such as [Godot](https://docs.godotengine.org/en/stable/tutorials/m - Higher-level randomness APIs - Entropy per-system rather than per-world +- Seed from arbitrary-length [`&[u8]`](https://github.com/bevyengine/rfcs/pull/55#issuecomment-1138159824) From 38f06105cfb928d23c1e3a9c54d5ba4320cbbf8e Mon Sep 17 00:00:00 2001 From: Christian Legnitto Date: Sun, 3 Jul 2022 19:14:02 -0700 Subject: [PATCH 15/15] Remove sentence about `rand` --- rfcs/55-deterministic_rng.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/rfcs/55-deterministic_rng.md b/rfcs/55-deterministic_rng.md index 31b9e9ca..d68496e8 100644 --- a/rfcs/55-deterministic_rng.md +++ b/rfcs/55-deterministic_rng.md @@ -96,8 +96,6 @@ Depending on your game and the type of randomness you require, when specifying a https://github.com/bevyengine/bevy/pull/2504 -- The PR includes `rand` as a dependency but it is not in public API. We might be able to slim it down to [`getrandom`](https://crates.io/crates/getrandom). - ## Drawbacks - This may not be general enough to include in Bevy.