Access to bones and changing their transformation

[fvito]

Hi, in this sceneform fork I'm trying to integrate the ARCore augmented faces and port the ARCore augmented faces sample, but I'm having some issues getting the bone translation to work. I know filament support bone based animations, but I was not able to find anything about modifying them at runtime without an animation.

At the moment, I'm only able to get the entity ID that gets assigned to the bone. Is there any other way to get the bone information in filament?. I saw that the parser library does support parsing bones, but I was not able to find what the filament asset does with that information.

I was trying to apply transformation of 1 entity to another (via TransformManager) but it did not work, I assume I made an error in terms of transform space. In which space is the entity transformation?

If I modify the transform of a bone entity, will that also affect the mesh that's connected to the bone, or do I need to use something else (like setBonesAsMatrices)?

Thanks, any help is much appreciated

[romainguy]

Bones are not represented by entities. setBonesAsMatrices and setBonesAsQuaternions are what you are looking for.

[61315]

@fvito If you have to use TransformManager, you modify the transform node not the bone itself, then call updateBoneMatrices yourself.

filament/libs/gltfio/src/Animator.cpp

Line 293 in 24b8821

void Animator::updateBoneMatrices() {

Take RiggedSimple.glb for instance, you can modify the transformation data of a node named Bone.001.

Like so:

inner class FrameCallback : Choreographer.FrameCallback {
    private val startTime = System.nanoTime()

    override fun doFrame(frameTimeNanos: Long) {
        choreographer.postFrameCallback(this)

        val elapsedTimeSeconds = (frameTimeNanos - startTime).toDouble() / 1_000_000_000
        val tcm = modelViewer.engine.transformManager

        modelViewer.asset?.getFirstEntityByName("Bone.001")?.let { entity ->
            val boneTransformComponent = tcm.getInstance(entity)
            val transform = rotation(Float3(0.0f,0.0f,1.0f), sin(elapsedTimeSeconds.toFloat()) * 20)
            tcm.setTransform(boneTransformComponent, transpose(transform).toFloatArray())
        }

        modelViewer.animator?.updateBoneMatrices()

        modelViewer.render(frameTimeNanos)
    }
}

and you get this:

This works because updateBoneMatrices invokes setBonesAsMatrices or setBonesAsQuaternions eventually.

filament/libs/gltfio/src/Animator.cpp

Line 320 in 24b8821

renderableManager->setBones(renderable, boneVector.data(), boneVector.size());

[fvito]

Thank you so much @61315 , this was exactly what I was looking for.
And also thank you @romainguy for clarifying the part about entities and bones

[ThomasGorisse]

Link to our PR : SceneView/sceneform-android#93 (comment)