Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - Add a SystemParam primitive for deferred mutations; allow #[derive]ing more types of SystemParam #6817

Closed
wants to merge 37 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
c0a66aa
add the `Buf<>` systemparam
joseph-gio Nov 29, 2022
e819787
use `Buf` to derive `Commands`
joseph-gio Nov 29, 2022
3bff701
derive `ParallelCommands`
joseph-gio Nov 29, 2022
7f324d6
improve diff
joseph-gio Nov 29, 2022
bf9840e
rename trait `Buffer` -> `SystemBuffer`
joseph-gio Nov 29, 2022
1daccbf
rename struct `Buf` -> `Buffer`
joseph-gio Nov 29, 2022
cdf6bbd
Merge remote-tracking branch 'upstream/main' into buffer-param
joseph-gio Dec 17, 2022
b4b1461
propagate `SystemMeta`
joseph-gio Dec 17, 2022
3a64d16
Merge remote-tracking branch 'upstream/main' into buffer-param
joseph-gio Dec 28, 2022
78ea2cc
Merge remote-tracking branch 'upstream/main' into buffer-param
joseph-gio Jan 5, 2023
b0d4e2e
make `ParallelCommandQueue` private
joseph-gio Jan 5, 2023
d585826
Merge remote-tracking branch 'upstream/main' into buffer-param
joseph-gio Jan 7, 2023
e568345
fix a doc link
joseph-gio Jan 8, 2023
ff97bf1
make those docs good
joseph-gio Jan 12, 2023
bcba009
add an example to `Buffer<T>`
joseph-gio Jan 12, 2023
4a46300
add more docs to `SystemBuffer`
joseph-gio Jan 12, 2023
6020791
rename `Buffer<>` -> `Deferred<>`
joseph-gio Jan 15, 2023
1a24fa7
add `Deferred<>` to the prelude
joseph-gio Jan 15, 2023
4a0d59b
Merge branch 'main' into buffer-param
joseph-gio Jan 23, 2023
a1b8f45
fix a redundant panic in the example
joseph-gio Jan 23, 2023
9637850
no coziness for these methods :(
joseph-gio Jan 23, 2023
35c79fe
cargo fmt
joseph-gio Jan 23, 2023
2817097
improve a note about up-front computations
joseph-gio Jan 30, 2023
0ff8aee
reorganize docs for `Deferred`
joseph-gio Feb 3, 2023
6a1aa3d
polish the example more
joseph-gio Feb 3, 2023
a86ce13
document when `Deferred` should and should not be used
joseph-gio Feb 3, 2023
ef12641
make the example a better use-case
joseph-gio Feb 3, 2023
e403150
fix CI
joseph-gio Feb 3, 2023
5ded352
Merge branch 'main' into buffer-param
joseph-gio Feb 5, 2023
094141d
add a note to the example about `SystemBuffer::apply`
joseph-gio Feb 5, 2023
27b6795
cargo fmt
joseph-gio Feb 5, 2023
e85e34e
Merge remote-tracking branch 'upstream/main' into buffer-param
joseph-gio Feb 6, 2023
9d6212a
fix a merge artefact
joseph-gio Feb 6, 2023
be1b372
migrate example to stageless
joseph-gio Feb 6, 2023
542e42a
Revert "fix a merge artefact"
joseph-gio Feb 6, 2023
1a746a6
Merge remote-tracking branch 'upstream/main' into buffer-param
joseph-gio Feb 6, 2023
8ac61af
merge error
joseph-gio Feb 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions crates/bevy_ecs/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,8 @@ pub mod prelude {
system::{
adapter as system_adapter,
adapter::{dbg, error, ignore, info, unwrap, warn},
Commands, In, IntoPipeSystem, IntoSystem, Local, NonSend, NonSendMut, ParallelCommands,
ParamSet, Query, Res, ResMut, Resource, System, SystemParamFunction,
Commands, Deferred, In, IntoPipeSystem, IntoSystem, Local, NonSend, NonSendMut,
ParallelCommands, ParamSet, Query, Res, ResMut, Resource, System, SystemParamFunction,
},
world::{FromWorld, World},
};
Expand Down
28 changes: 21 additions & 7 deletions crates/bevy_ecs/src/system/commands/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,18 @@ mod command_queue;
mod parallel_scope;

use crate::{
self as bevy_ecs,
bundle::Bundle,
entity::{Entities, Entity},
world::{FromWorld, World},
};
use bevy_ecs_macros::SystemParam;
use bevy_utils::tracing::{error, info};
pub use command_queue::CommandQueue;
pub use parallel_scope::*;
use std::marker::PhantomData;

use super::Resource;
use super::{Deferred, Resource, SystemBuffer, SystemMeta};

/// A [`World`] mutation.
///
Expand Down Expand Up @@ -97,22 +99,31 @@ pub trait Command: Send + 'static {
/// [`System::apply_buffers`]: crate::system::System::apply_buffers
/// [`apply_system_buffers`]: crate::schedule::apply_system_buffers
/// [`Schedule::apply_system_buffers`]: crate::schedule::Schedule::apply_system_buffers
#[derive(SystemParam)]
pub struct Commands<'w, 's> {
queue: &'s mut CommandQueue,
queue: Deferred<'s, CommandQueue>,
entities: &'w Entities,
}

impl SystemBuffer for CommandQueue {
#[inline]
fn apply(&mut self, _system_meta: &SystemMeta, world: &mut World) {
#[cfg(feature = "trace")]
let _system_span =
bevy_utils::tracing::info_span!("system_commands", name = _system_meta.name())
.entered();
self.apply(world);
}
}

impl<'w, 's> Commands<'w, 's> {
/// Returns a new `Commands` instance from a [`CommandQueue`] and a [`World`].
///
/// It is not required to call this constructor when using `Commands` as a [system parameter].
///
/// [system parameter]: crate::system::SystemParam
pub fn new(queue: &'s mut CommandQueue, world: &'w World) -> Self {
Self {
queue,
entities: world.entities(),
}
Self::new_from_entities(queue, world.entities())
}

/// Returns a new `Commands` instance from a [`CommandQueue`] and an [`Entities`] reference.
Expand All @@ -121,7 +132,10 @@ impl<'w, 's> Commands<'w, 's> {
///
/// [system parameter]: crate::system::SystemParam
pub fn new_from_entities(queue: &'s mut CommandQueue, entities: &'w Entities) -> Self {
Self { queue, entities }
Self {
queue: Deferred(queue),
entities,
}
}

/// Pushes a [`Command`] to the queue for creating a new empty [`Entity`],
Expand Down
37 changes: 9 additions & 28 deletions crates/bevy_ecs/src/system/commands/parallel_scope.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,16 @@ use std::cell::Cell;
use thread_local::ThreadLocal;

use crate::{
self as bevy_ecs,
entity::Entities,
prelude::World,
system::{SystemMeta, SystemParam},
system::{Deferred, SystemBuffer, SystemMeta, SystemParam},
};

use super::{CommandQueue, Commands};

/// The internal [`SystemParam`] state of the [`ParallelCommands`] type
#[doc(hidden)]
#[derive(Default)]
pub struct ParallelCommandsState {
struct ParallelCommandQueue {
thread_local_storage: ThreadLocal<Cell<CommandQueue>>,
}

Expand Down Expand Up @@ -43,41 +42,23 @@ pub struct ParallelCommandsState {
/// }
/// # bevy_ecs::system::assert_is_system(parallel_command_system);
///```
#[derive(SystemParam)]
pub struct ParallelCommands<'w, 's> {
state: &'s mut ParallelCommandsState,
state: Deferred<'s, ParallelCommandQueue>,
entities: &'w Entities,
}

// SAFETY: no component or resource access to report
unsafe impl SystemParam for ParallelCommands<'_, '_> {
type State = ParallelCommandsState;
type Item<'w, 's> = ParallelCommands<'w, 's>;

fn init_state(_: &mut World, _: &mut crate::system::SystemMeta) -> Self::State {
ParallelCommandsState::default()
}

fn apply(state: &mut Self::State, _system_meta: &SystemMeta, world: &mut World) {
impl SystemBuffer for ParallelCommandQueue {
#[inline]
fn apply(&mut self, _system_meta: &SystemMeta, world: &mut World) {
#[cfg(feature = "trace")]
let _system_span =
bevy_utils::tracing::info_span!("system_commands", name = _system_meta.name())
.entered();
for cq in &mut state.thread_local_storage {
for cq in &mut self.thread_local_storage {
cq.get_mut().apply(world);
}
}

unsafe fn get_param<'w, 's>(
state: &'s mut Self::State,
_: &crate::system::SystemMeta,
world: &'w World,
_: u32,
) -> Self::Item<'w, 's> {
ParallelCommands {
state,
entities: world.entities(),
}
}
}

impl<'w, 's> ParallelCommands<'w, 's> {
Expand Down
210 changes: 178 additions & 32 deletions crates/bevy_ecs/src/system/system_param.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ use crate::{
query::{
Access, FilteredAccess, FilteredAccessSet, QueryState, ReadOnlyWorldQuery, WorldQuery,
},
system::{CommandQueue, Commands, Query, SystemMeta},
system::{Query, SystemMeta},
world::{FromWorld, World},
};
pub use bevy_ecs_macros::Resource;
Expand Down Expand Up @@ -153,6 +153,8 @@ pub unsafe trait SystemParam: Sized {

/// Applies any deferred mutations stored in this [`SystemParam`]'s state.
/// This is used to apply [`Commands`] during [`apply_system_buffers`](crate::prelude::apply_system_buffers).
///
/// [`Commands`]: crate::prelude::Commands
#[inline]
#[allow(unused_variables)]
fn apply(state: &mut Self::State, system_meta: &SystemMeta, world: &mut World) {}
Expand Down Expand Up @@ -593,37 +595,6 @@ unsafe impl<'a, T: Resource> SystemParam for Option<ResMut<'a, T>> {
}
}

// SAFETY: Commands only accesses internal state
unsafe impl<'w, 's> ReadOnlySystemParam for Commands<'w, 's> {}

// SAFETY: `Commands::get_param` does not access the world.
unsafe impl SystemParam for Commands<'_, '_> {
type State = CommandQueue;
type Item<'w, 's> = Commands<'w, 's>;

fn init_state(_world: &mut World, _system_meta: &mut SystemMeta) -> Self::State {
Default::default()
}

fn apply(state: &mut Self::State, _system_meta: &SystemMeta, world: &mut World) {
#[cfg(feature = "trace")]
let _system_span =
bevy_utils::tracing::info_span!("system_commands", name = _system_meta.name())
.entered();
state.apply(world);
}

#[inline]
unsafe fn get_param<'w, 's>(
state: &'s mut Self::State,
_system_meta: &SystemMeta,
world: &'w World,
_change_tick: u32,
) -> Self::Item<'w, 's> {
Commands::new(state, world)
}
}

/// SAFETY: only reads world
unsafe impl<'w> ReadOnlySystemParam for &'w World {}

Expand Down Expand Up @@ -787,6 +758,181 @@ unsafe impl<'a, T: FromWorld + Send + 'static> SystemParam for Local<'a, T> {
}
}

/// Types that can be used with [`Deferred<T>`] in systems.
/// This allows storing system-local data which is used to defer [`World`] mutations.
///
/// Types that implement `SystemBuffer` should take care to perform as many
/// computations up-front as possible. Buffers cannot be applied in parallel,
/// so you should try to minimize the time spent in [`SystemBuffer::apply`].
pub trait SystemBuffer: FromWorld + Send + 'static {
/// Applies any deferred mutations to the [`World`].
fn apply(&mut self, system_meta: &SystemMeta, world: &mut World);
}

/// A [`SystemParam`] that stores a buffer which gets applied to the [`World`] at the end of a stage.
/// This is used internally by [`Commands`] to defer `World` mutations.
///
/// [`Commands`]: crate::system::Commands
///
/// # Examples
///
/// By using this type to defer mutations, you can avoid mutable `World` access within
/// a system, which allows it to run in parallel with more systems.
///
/// Note that deferring mutations is *not* free, and should only be used if
/// the gains in parallelization outweigh the time it takes to apply deferred mutations.
/// In general, [`Deferred`] should only be used for mutations that are infrequent,
/// or which otherwise take up a small portion of a system's run-time.
///
/// ```
/// # use bevy_ecs::prelude::*;
/// // Tracks whether or not there is a threat the player should be aware of.
/// #[derive(Resource, Default)]
/// pub struct Alarm(bool);
///
/// #[derive(Component)]
/// pub struct Settlement {
/// // ...
/// }
///
/// // A threat from inside the settlement.
/// #[derive(Component)]
/// pub struct Criminal;
///
/// // A threat from outside the settlement.
/// #[derive(Component)]
/// pub struct Monster;
///
/// # impl Criminal { pub fn is_threat(&self, _: &Settlement) -> bool { true } }
///
/// use bevy_ecs::system::{Deferred, SystemBuffer, SystemMeta};
///
/// // Uses deferred mutations to allow signalling the alarm from multiple systems in parallel.
/// #[derive(Resource, Default)]
/// struct AlarmFlag(bool);
///
/// impl AlarmFlag {
/// /// Sounds the alarm at the end of the current stage.
/// pub fn flag(&mut self) {
/// self.0 = true;
/// }
/// }
///
/// impl SystemBuffer for AlarmFlag {
/// // When `AlarmFlag` is used in a system, this function will get
/// // called at the end of the system's stage.
/// fn apply(&mut self, system_meta: &SystemMeta, world: &mut World) {
/// if self.0 {
/// world.resource_mut::<Alarm>().0 = true;
/// self.0 = false;
/// }
/// }
/// }
///
/// // Sound the alarm if there are any criminals who pose a threat.
/// fn alert_criminal(
/// settlements: Query<&Settlement>,
/// criminals: Query<&Criminal>,
/// mut alarm: Deferred<AlarmFlag>
/// ) {
/// let settlement = settlements.single();
/// for criminal in &criminals {
/// // Only sound the alarm if the criminal is a threat.
/// // For this example, assume that this check is expensive to run.
/// // Since the majority of this system's run-time is dominated
/// // by calling `is_threat()`, we defer sounding the alarm to
/// // allow this system to run in parallel with other alarm systems.
/// if criminal.is_threat(settlement) {
/// alarm.flag();
/// }
/// }
/// }
///
/// // Sound the alarm if there is a monster.
/// fn alert_monster(
/// monsters: Query<&Monster>,
/// mut alarm: ResMut<Alarm>
/// ) {
/// if monsters.iter().next().is_some() {
/// // Since this system does nothing except for sounding the alarm,
/// // it would be pointless to defer it, so we sound the alarm directly.
/// alarm.0 = true;
/// }
/// }
///
/// let mut world = World::new();
/// world.init_resource::<Alarm>();
/// world.spawn(Settlement {
/// // ...
/// });
///
/// let mut schedule = Schedule::new();
/// schedule
/// // These two systems have no conflicts and will run in parallel.
/// .add_system(alert_criminal)
/// .add_system(alert_monster);
///
/// // There are no criminals or monsters, so the alarm is not sounded.
/// schedule.run(&mut world);
/// assert_eq!(world.resource::<Alarm>().0, false);
///
/// // Spawn a monster, which will cause the alarm to be sounded.
/// let m_id = world.spawn(Monster).id();
/// schedule.run(&mut world);
/// assert_eq!(world.resource::<Alarm>().0, true);
///
/// // Remove the monster and reset the alarm.
/// world.entity_mut(m_id).despawn();
/// world.resource_mut::<Alarm>().0 = false;
///
/// // Spawn a criminal, which will cause the alarm to be sounded.
/// world.spawn(Criminal);
/// schedule.run(&mut world);
/// assert_eq!(world.resource::<Alarm>().0, true);
/// ```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This example does a great job of illustrating the API, but I am worried it might teach "bad practice". It might lead people to think that using such a deferred pattern is expected to be faster / perform better than simply mutating the resource normally from the system. That is not true: deferring actions via system buffers adds a lot of overhead (that typically outweighs gains from parallelism for such small systems), and in this example the operation (mutating a resource value) could have been done from parallel systems just fine. Deferred does not win anything.

I think the example could be made better if it actually uses an operation that cannot be done from parallel systems without using Deferred, such as adding/removing components or spawning entities. "Sounding the alarm" could be done by spawning an entity or adding a component, instead of mutating a value. That way, we actually have a reason to use Deferred.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deferring actions via system buffers adds a lot of overhead

Why do you say that this adds a lot of overhead? apply_buffers gets called on every system via dynamic dispatch, so there is already an overhead for each system regardless of whether it has any buffers to apply. Any additional overhead associated with Deferred<> should be very small, and entirely dependent on how the user implements SystemBuffer.

That said, I agree with your other points. I believe I have addressed them all.

pub struct Deferred<'a, T: SystemBuffer>(pub(crate) &'a mut T);

impl<'a, T: SystemBuffer> Deref for Deferred<'a, T> {
type Target = T;
#[inline]
fn deref(&self) -> &Self::Target {
self.0
}
}

impl<'a, T: SystemBuffer> DerefMut for Deferred<'a, T> {
#[inline]
fn deref_mut(&mut self) -> &mut Self::Target {
self.0
}
}

// SAFETY: Only local state is accessed.
unsafe impl<T: SystemBuffer> ReadOnlySystemParam for Deferred<'_, T> {}

// SAFETY: Only local state is accessed.
unsafe impl<T: SystemBuffer> SystemParam for Deferred<'_, T> {
type State = SyncCell<T>;
type Item<'w, 's> = Deferred<'s, T>;

fn init_state(world: &mut World, _system_meta: &mut SystemMeta) -> Self::State {
SyncCell::new(T::from_world(world))
}

fn apply(state: &mut Self::State, system_meta: &SystemMeta, world: &mut World) {
state.get().apply(system_meta, world);
}

unsafe fn get_param<'w, 's>(
state: &'s mut Self::State,
_system_meta: &SystemMeta,
_world: &'w World,
_change_tick: u32,
) -> Self::Item<'w, 's> {
Deferred(state.get())
}
}

/// Shared borrow of a non-[`Send`] resource.
///
/// Only `Send` resources may be accessed with the [`Res`] [`SystemParam`]. In case that the
Expand Down