Motion In-Betweening with Phase Manifolds
In 22nd ACM SIGGRAPH/EUROGRAPHICS Symposium on Computer Animation (SCA 2023)
Paul Starke,
Sebastian Starke,
Taku Komura,
Frank Steinicke
Proc. ACM Comput. Graph. Interact. Tech. 6, 3, Article 37 (August 2023), 17 pages. https://doi.org/10.1145/3606921
This work introduces a novel data-driven motion in-betweening system to reach target poses of characters by making use of phases variables learned by a Periodic Autoencoder. The approach utilizes a mixture-of-experts neural network model, in which the phases cluster movements in both space and time with different expert weights. Each generated set of weights then produces a sequence of poses in an autoregressive manner between the current and target state of the character. In addition, to satisfy poses which are manually modified by the animators or where certain end effectors serve as constraints to be reached by the animation, a learned bi-directional control scheme is implemented to satisfy such constraints. Using phases for motion in-betweening tasks sharpen the interpolated movements, and furthermore stabilizes the learning process. Moreover, more challenging movements beyond locomotion behaviors can be synthesized. Additionally, style control is enabled between given target keyframes. The framework can compete with state-of-the-art methods for motion in-betweening in terms of motion quality and generalization, especially in the existence of long transition durations. This framework contributes to faster prototyping workflows for creating animated character sequences, which is of enormous interest for the game and film industry.
- Video - Paper - Code, Demo & Tool - ReadMe
This project is only for research or education purposes, and not freely available for commercial use or redistribution. The motion capture data and 3D character model from ubisoft-laforge is available only under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Public License.