Reactive components

[ffreyer]

This is partially just me thinking out loud, but maybe it's good to discuss this and start some "best practices" style documentation as well. Some components naturally have a reactive relationship. For example you only want to run conversion pipeline Vector{Symbol} -> Vector{RGBA} -> Texture when the input is updated. There are a couple of ways one could achieve this:

Observables

I think Observables are pretty clean in this case, and maybe also the fasted way to do things. It seems like EnTT also supports something along those lines. Perhaps it makes sense to implement a light observable type for this, which restricts what you are allowed to do. E.g. only one synchronized callback?

@component struct InputColor
    data::Observable{Vector{Any}}
end

@component struct ConvertedColor
    data::Observable{Vector{RGBAf0}}
end

@component struct ColorTexture
    data::Observable{Texture1D}
end

input = Node(:red)
converted = map(convert_color, input)
texture = to_texture(converted)

ledger[entity] = InputColor(input)
ledger[entity] = ConvertedColor(converted)
ledger[entity] = ColorTexture(texture)

Per component has_changed or needs_update

If we want to avoid Observables using mutable components that keep track of whether they have changed/need updates could work. I guess this would be faster when updates are frequent and slower if they are rare.

@component mutable struct InputColor
    data::Vector{Any}
    has_changed::Bool
end

@component mutable struct ConvertedColor
    data::Vector{RGBAf0}
    has_changed::Bool
end

@component struct ColorTexture
    data::Texture1D
end

function update(::ColorUpdater, m::AbstractLedger)
    for e in @entities_in(m[InputColor] && m[ConvertedColor])
        if m[InputColor][e].has_changed
            m[ConvertedColor].data = convert_color(m[InputColor].data)
            m[ConvertedColor].has_changed = true
            m[InputColor].has_changed = false
        end
    end
    for e in @entities_in(m[ConvertedColor] && m[ColorTexture])
        if m[ConvertedColor][e].has_changed
            update!(m[ColorTexture][e].data, m[ConvertedColor][e].data)
            m[ConvertedColor].has_changed = false
        end
    end
end

Update buffer

Another option would be to keep track of entity-component pairs that need updates in a buffer. I assume this ends up being worse in general as you're frequently searching through the buffer.

@component struct UpdateBuffer
    Vector{Tuple{Entity, DataType}}
end

...

function update(::ColorUpdater, m::AbstractLedger)
    has_changed = m[UpdateBuffer]
    isempty(has_changed) && return
    for e in @entities_in(m[InputColor] && m[ConvertedColor])
        idx = findfirst(isequal((e, InputColor)), has_changed)
        if idx !== nothing
            m[ConvertedColor].data = convert_color(m[InputColor].data)
            update!(m[ColorTexture][e].data, m[ConvertedColor][e].data)
            has_changed[idx] = (e, ConvertedColor)
        end
    end
    for e in @entities_in(m[ConvertedColor] && m[ColorTexture])
        idx = findfirst(isequal((e, ConvertedColor)), has_changed)
        if idx !== nothing
            update!(m[ColorTexture][e].data, m[ConvertedColor][e].data)
            deleteat!(has_changed, idx)
        end
    end
end

There are couple of alternatives that follow the same idea. For example, the buffer could be moved to the system. That would make it smaller and the components could be implied depending on how things are set up. You still have frequent de/allocations though.

Another possibility would be a more trait-like update component. It also avoids needing to check component types, but we're still essentially filling and clearing arrays frequently.

@component struct InputColorHasChanged end
@component struct ConvertedColorHasChanged end

...

function update(::ColorUpdater, m::AbstractLedger)
    for e in @entities_in(m[InputColor] && m[ConvertedColor] && m[InputColorHasChanged])
        m[ConvertedColor].data = convert_color(m[InputColor].data)
        pop!(m[InputColorHasChanged], e)
        m[e] = ConvertedColorHasChanged()
    end
    for e in @entities_in(m[ConvertedColor] && m[ColorTexture] && m[ConvertedColorHasChanged])
        update!(m[ColorTexture][e].data, m[ConvertedColor][e].data)
        pop!(m[ConvertedColorHasChanged], e)
    end
end

My impression is:

• option 1 / observables are best when updates are rare
• option 2 / mutable components with has_changed fields are best when updates are frequent and many of the same component type need updates
• a system that runs once per tick is best when updates are very frequent

[louisponet]

I have ran into the same situation and thought about this relatively deeply before.

Some of my thoughts:

• As you know, I have a general aversion towards using Observables and anything Reactive in the context where an ECS really makes sense. I.e. while it might make sense to trigger whole systems to start updating, or do something related to singleton components like camera and whatnot, I could see the use. For things like positions and whatnot it really has no place in an ECS.
• Related to the previous point is that generally I don't really like the "undeterministic" runtime nature of observables in general, it leads to hiccups, and it is much smarter to use free time after running through everything to update those things, or just update them deterministically always, you get my point.
• How often does 1 component of only 1 Entity change? For me it didn't happen too often in the past.
• That being said, I really think some kind of functionality like this should really exist.
• Some kind of automatic has_changed I think is quite elegant, and very flexible on first thought.

A little along the lines of your last idea could be also to have a Component wide buffer that stores the id's that have been changed. @entities_changed could then be used quite elegantly with a component struct like that.

To summarize:

• Observables: for me only nice for singletons and really enormous componentdata's, non-deterministic hiccups etc
• UpdateBuffer: Not sure
• has_changed: sounds pretty performant and flexible
• ChangingComponent: I think I may like this the most

I will look again at how EnTT did it, usually they are a very good source for inspiration. I think from what I remember what they have are callbacks, in the sense that if any setindex! runs on a particular component, all callback functions run. I wasn't 100% on board with this idea in the past, since I thought Systems should suffice to do the same thing and it breaks the idea of not attaching logic to data.

[ffreyer]

Related to the previous point is that generally I don't really like the "undeterministic" runtime nature of observables in general, it leads to hiccups, and it is much smarter to use free time after running through everything to update those things, or just update them deterministically always, you get my point.

If you enforce synchronized execution inside the Observable it would be deterministic, wouldn't it? input[] = :blue would basically expand to:

input.val = :blue
converted.val = convert_color(input.val)
texture.val = to_texture(converted.val)

How often does 1 component of only 1 Entity change? For me it didn't happen too often in the past.

In the context of a game engine maybe not. I'd say it's fairly common with plotting though. Most plot entities will be static and when they change it's often just one or two components. E.g. I have made arrows plots where I only wanted to adjust a rotation vector, or a mesh plot where I'm only adjusting colors. Things controlled by sliders also often end up controlling just one component for me.

I think from what I remember what they have are callbacks, in the sense that if any setindex! runs on a particular component, all callback functions run.

That's pretty much just an Observable then. 😆

[louisponet]

That's pretty much just an Observable then. 😆

Sorry with component I meant Component in the sense of the whole datastructure. We should try to distinguish Component and ComponentData

[dumblob]

Links in the following discussion might be of interest to you: traffaillac/traffaillac.github.io#1 .