How can I stack multiple animations correctly, such as hands and feet?

[3dportable]

I have several hands animations and several leg animations in glb file.
How do I properly stack several different animations?

[emackey]

Are you writing a glTF exporter yourself, or using an existing software package to author glTF?

[3dportable]

I wrote my little GLB file parser for my program (thanks for the document "gltf-What the daсk"). I can get animation keys and create mesh transformation matrices. I pass them to the shader and the animation displays fine if I only use one animation at a time.
And I wanted to try using two or more animations and layering them but didn't seem to get it. This is needed to overlay the animation of the arms and the whole body, this will reduce unnecessary duplication of animations. How to do it correctly?

[javagl]

If you have several animations that affect the local transform matrices of nodes, then it should be possible to combine them, but I think that this has some caveats: Imagine one animation that rotates object X clockwise and the other one rotating object X counterclockwise - you cannot say what the result should be. More broadly speaking: It might be possible to combine multiple animations if they are "independent" of each other. But I'm pretty sure that this is not covered by the specification.

But more importantly:

create mesh transformation matrices

That sounds like you're doing vertex skinning. If this is the case: Combining "multiple skinnings" (with animation) is really difficult (it might even be impossible to do this sensibly, but I couldn't give any advice here...)

[3dportable]

well. Maybe I chose a dead end path and there is another way to do it.
For example, I need to turn my head depending on the direction of the gaze during the playback of the walking animation.
How to do it?

[javagl]

It's a bit too early to talk about a "dead end" - at lest for me, it's not entirely clear what the goal is. The last statement sounded like you wanted to make the decision about the actual animations that are played at runtime. That might be difficult, but here is a short pointer to some related discussion: KhronosGroup/glTF#1052:

This eventually led to the implementation note in the spec:

Implementation Note: glTF 2.0 does not specifically define how an animation will be used when imported but, as a best practice, it is recommended that each animation is self contained as an action. For example, "Walk" and "Run" animations might each contain multiple channels targeting a model's various bones. The client implementation may choose when to play any of the available animations.

This sounds pretty similar to your application case. I think that it should be possble do do play several animations simultaneously, with some "caveats":

• Other glTF viewers may not be able to play multiple animations at the same time
• Your viewer/client would need dedicated controls and an infrastructure so that it is possible to tell it something like "Play the 'walk' animation, for 2.3 seconds, and after 1.8 seconds, play a certain part of the 'turn head' animation".
• You'd have to be really careful to not play animations that affect the same bones - otherwise, the results would be unpredictable

It could be a challenge to support this sensibly, but should be certainly be worth a try.

[emackey]

This isn't a core feature of glTF itself. But for what it's worth, it's possible to implement this on top of glTF, with custom code.

For example, here's the glTF sample fox, playing the "Run" animation and watching the International Space Station fly by at the same time:

In our case this is done with a vendor extension, AGI_articulations. An "articulation" is a joint that is permitted to move or rotate about particular axis/axes by amounts defined in the extension, but without a pre-scripted animation. Such transformations are applied separately and subsequently to the transformations applied by the glTF itself and its own built-in animations.

So what you're seeing here is that a bone in the Fox skeleton's head has been given an AGI_articulations extension and allowed to swivel by some number of degrees, along with a defined "pointing vector". The host software STK aligns this pointing vector with the user's target of choice, in this case the space station passing overhead. The Fox's "Run" animation is playing, causing the head to bob up and down, and the pointing articulation is applied subsequently to that, causing the head to track the station during the pass.

But again, this is vendor-specific, not part of the glTF standard itself. If you're not using STK, it's a feature that would need some development effort for your platform of choice.

[3dportable]

Yes, this wonderful video inspired me to find new solutions, thank you. I was thinking about a method to change the position of the bone programmatically and recalculate all matrices in real time.
And my idea was that the rotation of the fox's head was recorded in a separate animation and, specifying the number of the current frame of the head, merge it with the current animation of the body.

[3dportable]

thank you