Smooth following via generalized interpolation

[mweatherley]

Objective

Partially addresses #13408.

Unify interpolation between various math types under a shared trait, allowing for common operations utilizing interpolation to be shared between them. In particular, implement a version of "smoothed following" wherein the distance between an entity and its target shrinks exponentially over time, framerate-independently.

Solution

A new trait, Interpolate, has been added to bevy::math::common_traits. Its essential part looks like this:

/// A type that can be intermediately interpolated between two given values
/// using an auxiliary linear parameter.
///
/// The expectations for the implementing type are as follows:
/// - `interpolate(&first, &second, t)` produces `first.clone()` when `t = 0.0`
///   and `second.clone()` when `t = 1.0`.
/// - `interpolate` is self-similar in the sense that, for any values `t0`, `t1`,
///   `interpolate(interpolate(&first, &second, t0), interpolate(&first, &second, t1), t)`
///   is equivalent to `interpolate(&first, &second, interpolate(&t0, &t1, t))`.
pub trait Interpolate: Clone {
    /// Interpolate between this value and the `other` given value using the parameter `t`.
    /// Note that the parameter `t` is not necessarily clamped to lie between `0` and `1`.
    /// However, when `t = 0.0`, `self` is recovered, while `other` is recovered at `t = 1.0`,
    /// with intermediate values lying between the two in some appropriate sense.
    fn interpolate(&self, other: &Self, t: f32) -> Self;
}

This has a blanket implementation over V: VectorSpace using lerp, and it is also implemented for Quat and the direction types using slerp.

Furthermore, the interpolation functionality has been extracted from bevy_animation's Animatable trait, which instead has an Animatable: Interpolate bound. The remaining implementations required to make this work are the f64 vectorial types — f64 and the DVec types — together with bool and Transform, all of which now have Interpolate implementations in bevy_math, except for Transform, which lives in bevy_transform. There is no functional difference between the old Animatable implementations and the new ones.

Now for the "following" functionality that this facilitates:

/// Smoothly nudge this value towards the `target` at a given decay rate. The `decay_rate`
/// parameter controls how fast the distance between `self` and `target` decays relative to
/// the units of `delta`; the intended usage is for `decay_rate` to generally remain fixed,
/// while `delta` is something like `delta_time` from an updating system. This produces a
/// smooth following of the target that is independent of framerate.
///
/// More specifically, when this is called repeatedly, the result is that the distance between
/// `self` and a fixed `target` attenuates exponentially, with the rate of this exponential
/// decay given by `decay_rate`.
///
/// For example, at `decay_rate = 0.0`, this has no effect.
/// At `decay_rate = f32::INFINITY`, `self` immediately snaps to `target`.
/// In general, higher rates mean that `self` moves more quickly towards `target`.
///
/// # Example
/// ```
/// # use bevy_math::{Vec3, Interpolate};
/// # let delta_time: f32 = 1.0 / 60.0;
/// let mut object_position: Vec3 = Vec3::ZERO;
/// let target_position: Vec3 = Vec3::new(2.0, 3.0, 5.0);
/// // Decay rate of ln(10) => after 1 second, remaining distance is 1/10th
/// let decay_rate = f32::ln(10.0);
/// // Calling this repeatedly will move `object_position` towards `target_position`:
/// object_position.smooth_nudge(&target_position, decay_rate, delta_time);
/// ```
fn smooth_nudge(&mut self, target: &Self, decay_rate: f32, delta: f32) {
    self.interpolate_assign(target, 1.0 - f32::exp(-decay_rate * delta));
}

For any type implementing Interpolate, this allows a value to be smoothly nudged in the direction of another, with the "speed" of this nudging determined by the decay_rate parameter. In particular, this can be used positionally to make an entity follow another in space, as demonstrated by the smooth_follow example:

Screen.Recording.2024-05-31.at.3.58.07.PM.mov

However, since this is abstracted over Interpolate, this can similarly be applied to, e.g., make directions, rotations, and Transforms also smoothly follow a target (which is perhaps moving). In a system, this looks something like this (taken from the example):

let delta_time = time.delta_seconds();

// Calling `smooth_nudge` is what moves the follower smoothly toward the target.
following
    .translation
    .smooth_nudge(&target.translation, decay_rate, delta_time);

Testing

Recommend testing by running/tweaking the included example.

---

Changelog

• Interpolate created in bevy_math::common_traits.
• interpolate moves from bevy_animation's Animatable to Interpolate.
• New example smooth_follow demonstrates usage of smooth_nudge.

Migration Guide

Anyone creating their own implementations of Animatable will have to move the interpolate part of such implementations into an implementation of bevy_math's Interpolate.

---

Discussion

Design decisions

Originally smooth_nudge was more like interpolate in that it just returned the value and users had to assign it themselves or use a smooth_nudge_assign method for that. I figured that the would-be smooth_nudge_assign would be what users would actually want the vast majority of the time, so I made that the default behavior instead. We could probably bikeshed about this a little.

Notably, there are also other ways of specifying the exponential decay rate: for example, the half-life related to the units of delta is another natural choice, as is the fraction of distance that remains after a single one of the delta units. The present version was chosen because f32::exp is more efficient than alternatives, but there may be reason to include the others in the future for ease of use (see future directions).

Unrelatedly, a slightly unsatisfying (to me) thing about the present suite of implementation is that it has to contain bool in order to make bool: Animatable continue to be possible, and for types with stepped interpolation, smooth_nudge is basically useless. I have mixed feeling about this kind of thing being in bevy_math in general, but I don't really see any alternative that doesn't alter bool: Animatable.

Finally, I wanted to say something about consistency with notions of distance. Naïvely, of course, there is no real notion of distance between two interpolable values; however, the second assumed constraint on Interpolate implies that the "interpolation distance" is kept consistent between successive calls. For example, if a decay_rate of ln(2) is used, then every unit of "time", the interpolation parameter will progress by half: first from 0 to 0.5, then to 0.75, 0.875, and so on. In normed vector spaces, this can be interpreted in terms of actual distance, but for things like directions, this coincides with the intuition about a shrinking angle between the source and the target.

(Note that, in general, this interpolation constraint is automatically satisfied by any form of interpolation which is shadowed by linear interpolation in the associated group's Lie algebra.)

Future directions

bevy_color's Mix should probably also be absorbed into Interpolate, since it is exactly the same thing, but this was intentionally excluded from this PR to limit the scope of its impact.

There are probably other ergonomic ways of specifying similar behavior to smooth_nudge, but I decided to leave it at just the one for the sake of simplicity. Another important thing in this direction is that in the special case of V: NormedVectorSpace, it makes sense to allow for limitations on instantaneous velocity. This could be done by exposing a non-self-modifying version of smooth_nudge in Interpolate and looking at the norm of the difference relative to delta.

I think it would also be good to demonstrate non-positional smooth following behavior somehow in an example (perhaps just expanding this one); e.g. following a rotation or a full Transform.

[alice-i-cecile]

Great: my first question was going to be "how does this relate to Animatable".

[NthTensor]

Nice work! I support this direction and I quite like how following is implemented.

how does this relate to Animatable?

In my view, this generalizes part of Animatable that was too specific but otherwise leaves it intact. The problem is that we have slowly added several traits in different crates that define similar (but not always identically specified) interpolate methods. This lets us combine all of those into a single standardized interpolation operation on each type.

Animatable is still useful on it's own. According to the Animation-Graph RFC, Animatable seems to have been intended as a way to generalize VariableCurve. But that hasn't happened yet (or was tried and dropped). Perhaps smooth_nudge and Animatable could be combined, but I think we might need @james7132 to weigh in on that.

Actually, I'd like to hear from any of the people who participated in the (very extensive) Animatable discussion, so I've requested reviews from them (ping @atlv24, since I can't request review from you).

Regarding smooth_nudge being useless for bool. We only need interpolate to support Animatable right? Perhaps smooth_nudge could live in it's own Smoothable: Interpolate trait which bool won't implement.

[djeedai]

I'm perplex about this change. See the comment about the trait assumptions and guarantees. The mention about using for colors is the future is also concerning, because color mixing is fairly complex when color spaces are involved, and there's more than one way to do it so wouldn't fit a trait which doesn't really guarantee anything.

I'd be fine if we provided stronger guarantees, like saying the interpolation is always linear (including for rotations and directions, so the rotation angle is linear, so slerp). But as it stands I don't think it's useable outside the animation library without assuming a particular implementation, which defeats the abstraction.

[djeedai]

Your comment below shows an example with a non linear t. In general I don't see any reason why we should restrict this to be linear. Unless you mean "linear interpolation", but again, no reason for that.

[mweatherley]

The parameter itself is "linear" in the sense that it is one-dimensional, and the second trait requirement treats the parameter itself linearly, since it involves interpolating f32s with lerp via interpolate, but this doesn't mean at all that the interpolation has to be linear.

That assumption is more like a constant-velocity requirement; I'm not sure if I actually like it, but its raison d'être is that it guarantees that smooth_nudge's decay parameter actually means something relatively precise.

[djeedai]

The parameter itself is "linear" in the sense that it is one-dimensional,

That's "scalar" not "linear".

and the second trait requirement treats the parameter itself linearly, since it involves interpolating f32s with lerp via interpolate, but this doesn't mean at all that the interpolation has to be linear.

I don't understand this sentence, you sem to contradict yourself. If the implementation is lerp then the only source of non linearity is t so it's not linear.

[mweatherley]

I am not contradicting myself. The condition stated in the trait documentation involves linear reparametrization: if $C(p,q): \lbrack 0, 1 \rbrack \rightarrow X$ is the curve connecting $p$ and $q$ specified by interpolate, then for $a,b \in \lbrack 0, 1 \rbrack$, $a < b$, $C(C(p,q)(a), C(p,q)(b))$ is demanded to be the obvious linear reparametrization of $C(p,q)|_{\lbrack a, b \rbrack}$.

(Another natural condition that is more lax is that the former merely be some reparametrization of the latter.)

[djeedai]

I'm slightly concerned about the fact we're adding to a foundation library a trait which is very vague, and whose implementation is relied upon by another library (animation) so is making implicit assumptions in its implementation for math types (directions and rotations use slerp, other types use lerp). I understand the drive to uniformize and remove duplicates code, but what happens when someone wants to use that trait in another library? There's almost zero guarantee on the produced value other than some vague "interpolation" mention, so if I were a crate author I'd avoid that trait because I can't reason about it. While it's located in the animation library at least there's an assumption it's used for animation, and therefore that the interpolation is linear. But here, nothing prevents me from implementing this trait as some weird random value provided I satisfy the constraints at t=0 and t=1.

[mweatherley]

To me, using lerp/slerp for linear and rotational types has anything to do with animation per se — those are the modes of interpolation that are mathematically natural; my expectation is that if bevy_animation wanted something else specific to their use-cases, they would newtype them to customize the interpolation. In my opinion, the only concession here is for bool, but perhaps the better way forward would be to just force bevy_animation to newtype it.

As to the matter of vagueness: this is at least partially by design; for some additional context, this trait's real reason for existence is to allow for quite varied notions of interpolation, especially in light of the future Curve RFC, which presents a general framework wherein this trait allows curves to be consistently defined by discrete data. That was designed primarily with the needs of future work in bevy_math in mind (i.e. curve geometry, RMFs, reparametrizing motion, and so on), but the idea is that bevy_animation would also use it for curves in animation.

I'm not sure exactly what the concern about API consumers is; I would expect that calling interpolate on vectors or directions should produce an intuitive notion of interpolation between them, and it does. If third-party libraries want to implement weird implementations of interpolation, there is nothing anyone can do to stop them regardless, but that's true of all public trait interfaces.

On some level, I do think this is almost lower-level than bevy_math (i.e. bevy_math is kind of more of an API consumer than a producer), but I don't think there's really any better place for it right now.

[djeedai]

if bevy_animation wanted something else specific to their use-cases, they would newtype them to customize the interpolation

Probably impossible, Bevy Animation works on the types it's asked to work on by the user, not the types it wants to.

has anything to do with animation per se

Maybe, but animation relies on linear interpolation. If we changed it, everything would break. And this trait doesn't provide any assurance about linearity.

I'm not sure exactly what the concern about API consumers is; I would expect that calling interpolate on vectors or directions should produce an intuitive notion of interpolation between them, and it does.

This explanation highlights very precisely what I'm worried about, the only definition here is "intuitive" and that's subjective. There's millions of values between 0. and 1., all of them can be defined as an interpretation of 0. and 1. via some function. If we don't specify the function, and just say "intuitive", then the trait is unusable because what's intuitive depends on the usage context, and it's not intrinsic such that we can have a single implementation.

this trait's real reason for existence is to allow for quite varied notions of interpolation

I don't understand how a trait with a single important per type can allow varied notions. The traditional way of doing non linear interpolation, like Bevy Animation, is to likely interpolate values using a non linear parameter t. But then your other comment states that it's linear, which prevents this pattern.

[mweatherley]

Probably impossible, Bevy Animation works on the types it's asked to work on by the user, not the types it wants to.

Actually quite easy, considering that Curves are mappable: the underlying data that is being interpolated does not need to have the same type as its output.

Maybe, but animation relies on linear interpolation. If we changed it, everything would break. And this trait doesn't provide any assurance about linearity.

No, but it makes very strong demands on interpolation nonetheless.

This explanation highlights very precisely what I'm worried about, the only definition here is "intuitive" and that's subjective. There's millions of values between 0. and 1., all of them can be defined as an interpretation of 0. and 1. via some function. If we don't specify the function, and just say "intuitive", then the trait is unusable because what's intuitive depends on the usage context, and it's not intrinsic such that we can have a single implementation.

"Some function" is not likely to satisfy the requirements of the trait implementation, which are quite strict.

[mweatherley]

The mention about using for colors is the future is also concerning, because color mixing is fairly complex when color spaces are involved, and there's more than one way to do it so wouldn't fit a trait which doesn't really guarantee anything.

I am referring specifically to the Mix trait, which already exists and looks exactly like Interpolate:

/// Linear interpolation of two colors within a given color space.
pub trait Mix: Sized {
    /// Linearly interpolate between this and another color, by factor.
    /// Factor should be between 0.0 and 1.0.
    fn mix(&self, other: &Self, factor: f32) -> Self;

    /// Linearly interpolate between this and another color, by factor, storing the result
    /// in this color. Factor should be between 0.0 and 1.0.
    fn mix_assign(&mut self, other: Self, factor: f32) {
        *self = self.mix(&other, factor);
    }
}

[NthTensor]

I think we should set colorspaces and mix aside, they are out of scope of the current PR and deserve a discussion all to themselves.

[mweatherley]

Let me clarify something about what's going on with the trait requirements here (the documentation pretty brief and not all that clear). The main idea is that interpolation should satisfy some version of "self-similarity". To make this precise, I'm going to be a little mathy.

Let $p,q \in X$ be two values being interpolated, and $C(p,q): \lbrack 0, 1 \rbrack \rightarrow X$ the curve defined by interpolation (for all choices of $p$ and $q$). Then the idea is to impose a relationship between $C(p,q)$ and $C(C(p,q)(a), C(p,q)(b))$ for $a,b \in \lbrack 0,1 \rbrack$. In particular, $C(p,q)|_{\lbrack a, b \rbrack}$ are both curves connecting the same points $C(p,q)(a)$ and $C(p,q)(b)$, so one can demand that...

• (weak) The two are related by reparametrization; i.e., they both traverse the same values in the same order.
• (strong) The two are related by the obvious linear reparametrization $\lbrack a, b \rbrack \rightarrow \lbrack 0, 1 \rbrack$.

The latter is the condition that is currently stated on the trait. Note that:

• The weak condition allows things like linear interpolation of rotations (e.g. Rotation2d::nlerp) while the strong condition precludes it.
• The strong condition is still satisfied by a lot of things; in particular:
  • If $C$ is a constant-speed minimal geodesic, the strong condition is automatic (hence lerp, slerp etc all satisfy it).
  • If $G$ is a Lie group and $U \subset G$ is an open set on which a logarithm is defined, then for $p, q \in U$, $C(p,q)(t) := $exp(lerp(log(p), log(q), t)$ satisfies the strong condition.
  • Step "interpolation" transitioning at $t = 1$ satisfies the strong condition. Step interpolation at intermediate values does not.

One reason to care about this is that the strong condition guarantees a kind of subdivision stability, while the weak one does not: if you split the interval into two parts and interpolate each separately and then join them back together in the obvious way, then the strong condition implies that you get the same thing as just interpolating the whole thing at once, while the weak condition does not.

[djeedai]

This is not the clarification which is need for reviewers. The one piece missing is that this change is based on the Curve RFC proposal, and most notably on a fundamental change to how Bevy Animation works, which is only proposed in a Draft PR #13105 which has only been reviewed by @nzhao95, and is missing approval from the folks involved in the current design of Bevy Animation, notably @pcwalton, as well as probably @james7132, @rparrett, @NthTensor, and @mockersf.

The proposed change as I understand it flips the interpolation scheme on its head by making all interpolations go through the curve trait, moving the non-linear sampling from the parameter t: f32 (which is relatively easy to handle) to the interpolation function itself (which handles all the types we want to interpolate, some of which like Quat-based rotations become very hairy to work with in non-linear contexts).

From:

let t: f32 = sample_curve(t: f32);
let out: T = lerp(a: T, b: T, t: f32);

into:

let out: T = sample_curve(a: T, b: T, t: f32);

This change is I think flexible for e.g. an Editor scenario where you want to have maximum control over curves and editing capabilities, including allowing things like constant velocity (since the parameter varies linearly), but has a good chance of being a bottleneck performance-wise. There's also no indication that this scheme can allow batching / SIMD implementations, in the same way that the current Bevy Animation was designed for (there were extensive discussions about current and future plans for performance-oriented improvements during the initial PR #4482).

So, reiterating, as currently designed, this change breaks Bevy Animation by failing to guarantee that Interpolate does a linear interpolation, which is required by the current Bevy Animation design. If that design was to be changed, then we could re-discuss the current change. As it stands though, I don't think we should merge this.

[NthTensor]

The one piece missing is that this change is based on the Curve RFC proposal, and most notably on a fundamental change to how Bevy Animation works.

To be fair, I originally suggested this line of inquiry when reviewing the curves RFC. I'm going to upgrade this to controversial, though I'm hopeful we will still find a path forward on this through coordination between the math and animation domains.

I suspect there may be some "right hand doesn't know what the left is doing" communication issues here as well. For example, bevy_math has a trait called VectorSpace which supports all linear operations, including linear interpolation. So if you want to do something like the following

let t: f32 = sample_curve(t: f32);
let out: T = lerp(a: T, b: T, t: f32);

then sample_curve would be a Curve<f32> and T would be a VectorSpace.

VectorSpace exists to make splines work. But the issue with VectorSpace (and part of what this PR exists to solve) is that we can't implement it for Quat because, well, normalized quaternions are not a vector space. So we wanted a trait that would provide linear interpolation on vector spaces, and spherical interpolation on quaternions. The thought was that other traits that already worked that way (like Animatable) could be based on that.

Some of this motivation has gotten a little lost in the formal specification, I fear. It might be easier to chat about some of this stuff in a less structured setting (the discord).

[nzhao95]

Hello ! I did review the draft a little and came up with, I think, similar conclusions than @djeedai . I've been using bevy for a very short while so I don't want to overstep but I think there's definitely a way for math and animation to work together.
Curve editing is a great feature that is awesome for animation edition but not so much in terms of runtime, so I would suggest we start having separate runtime and raw data.
It's fairly common to have. This would allow us to edit using curves and when going live have the transformed and optimized animation be played. We can also always have a debug option to play the raw animation on screen (keep the option on at your own risk of performance drop).
(Sorry if I don't see the big picture and if I'm missing some important objectives)

[mweatherley]

VectorSpace exists to make splines work. But the issue with VectorSpace (and part of what this PR exists to solve) is that we can't implement it for Quat because, well, normalized quaternions are not a vector space. So we wanted a trait that would provide linear interpolation on vector spaces, and spherical interpolation on quaternions. The thought was that other traits that already worked that way (like Animatable) could be based on that.

I sat with this a while and I think that even this might be misguided (having a direct relationship with Animatable). Rather, I think the idea that interpolation should be uniformly reified at the type level is not a decision that should be made at the level of bevy_math. However, I do think that abstracting over the common (and especially nice) interpolations that occur in bevy_math is still quite a useful thing to do (for instance, by allowing the kind of smooth following behavior that is developed in this PR).

Here is a concrete proposal in the direction of ameliorating this:

1. Maintain something that looks like the version of Interpolate developed here in this PR, with an appropriate name and docs that clearly delineate the exact requirements that are depended upon by consumers. This will bear no direct relationship to bevy_animation.
2. bevy_animation can use the traits in its own implementation of interpolation, but there will be no direct relationship between Animatable::interpolate and bevy_math's strict version of interpolation. If a more flexible interpolation trait is desired, it can be created in bevy_animation and not bevy_math. Such a weaker trait would not be consumed directly by the Curve API.
3. In the Curve API, the resampling and sample interpolation apparatus that currently requires an interpolation trait will instead accept an explicit closure for interpolation. This allows consumers not to be burdened by juggling newtypes and so on just to specify a mode of interpolation. (We were already heading in this direction by restricting T: Interpolable to the resampling methods in the actual implementation.)
4. Also in the Curve API, we could have convenience methods that interpolate based on the more strict math-owned trait for types that satisfy it. This is good for ergonomics, in the sense that you don't have to explicitly specify interpolation in cases where it is "obvious". This will be the only place in the Curve API that consumes an interpolation trait.

This also allows us to get rid of the awkward implementation for bool, worrying about generalizing a notion of interpolation for all of humanity, and so on. I think the biggest improvement is the acknowledgement that type-level specification of interpolation is not the right thing to enforce in full generality.

[mweatherley]

I'm confident enough that the redesign is a better idea that I'm just going to close this and file a new PR with the new design later.