EXT_mesh_lod extension proposal

[takahirox]

Working repository: https://github.com/takahirox/EXT_mesh_lod

I would like to propose EXT_mesh_lod extension that allows LODs for mesh in glTF.

I know already MSFT_lod extension exists for LOD. The new extension is less flexible but simpler than MSFT_lod so it should be easier for the handlers (implementations) to handle.

JSON structure and examples

EXT_mesh_lod can be defined at the node level.

"nodes": [{
    "mesh": 0,
    "extensions": {
        "EXT_mesh_lod": {
            "mesh_lod": [
                {"mesh": 1, "coverage": 0.5},
                {"mesh": 2, "coverage": 0.2}
            ]
        }
    }
}]

"nodes": [{
    "mesh": 0,
    "skin": 0,
    "weights": [0.0, 1.0],
    "extensions": {
        "EXT_mesh_lod": {
            "mesh_lod": [
                {"mesh": 1, "skin": 1, "weights": [0.0, 1.0], "coverage": 0.5},
                {"mesh": 2, "skin": 2, "weights": [0.0, 1.0], "coverage": 0.2}
            ]
        }
    }
}]

Difficulty of handling MSFT_lod

While I was writing a handler for MSFT_lod I realized that MSFT_lod has some difficulties to handle.

• Both node and material can define LOD. The behavior is unstated in the specification if material containing LOD is referred by node containing LOD.
• A node can specify other nodes so LOD node can form nest or even infinity loop. And the behavior for them is unstated in the specification.
• A node can specify other nodes. node can have multiple type objects; mesh, camera, light (with the extension). The behavior is unstated in the specification if LODs have different type objects ex: High LOD has only mesh and Low LOD has only camera or even doesn't have any objects.
• Threshold for switching LOD is defined in extra as optional. Unclear how to switch LOD for applications/viewers/engines if it is undefined because they don't know the qualities of LODs.

Even the behaviors are stated in the specification, the implementations can be complex because they need to check a lot of definitions.

Simplification ideas

I would like to suggest to simplify the specification. It will lose some flexibilities but be simpler for the handlers to handle.

• Allow to define LOD only at node for mesh. This change can avoid the case where both node and material have LOD. Material LOD capability is still covered by LOD meshes refer to the same geometry attributes and refer to different materials. And the change can clarify what LOD can be defined for.
• The extension in a node specifies mesh indices as LOD meshes. This can avoid the nest and different type node object LOD.
• Require threshold values. This can avoid unknown threasholds for applications/viewers/engines.

Additional notes

• The reason why allowing the extension at node, not mesh, is to avoid nest across meshes caused by refering a mesh from another (or self) mesh. And also to allow skin and weights properties for LOD. If LOD is defined at mesh, lower LOD meshes must have the same bones/morphs with the highest LOD mesh, it may be a bit too limited.
• With the simplification suggestions, the extension will be easier to be handled so I think it would be good to be a multi-vendor extension (EXT_ prefix) rather than a vendor extension.

Discussions

I'm still thinking how threshold values should be, screen coverage, distance from camera, or others. In the current PR, I chose screen coverage for now. Probably compatibility is important so we may need to choose the one that can be handled consistently across applications/viewers/engines.

takahirox/EXT_mesh_lod#1

Related threads

• Adding LODs to glTF  #1045
• Level of Detail (LoD) or Quality of glTF asset #1743
• MSFT_lod: Clarify the behavior of the case where node level and material level LODs are specified #1952
• MSFT_lod: Clarify whether recursive LOD is allowed or not #1953

[takahirox]

From: https://twitter.com/onirenaud/status/1589890750267162624

We may need to think of the combination with EXT_mesh_gpu_instancing.

An easy solution idea for this EXT_mesh_lod extension may be to allow EXT_mesh_gpu_instancing in the extensions of EXT_mesh_lod.mesh_lod array entry. I'm not sure if the implementations can simply handle it tho.

"nodes": [{
    "mesh": 0,
    "extensions": {
        "extensions": {
            "EXT_mesh_gpu_instancing": {
                ....
            }
        },
        "EXT_mesh_lod": {
            "mesh_lod": [
                {
                    "mesh": 1,
                    "coverage": 0.5,
                    "extensions": {
                        "EXT_mesh_gpu_instancing": {
                            ...
                        }
                    }
                }
            ]
        }
    }
}]

Honestly, MSFT_lod that allows to specify node, not mesh, as LOD may have advantage for EXT_mesh_gpu_instancing (and skin for skinning and weights for morph) because probably the implementations can easily handle.

[takahirox]

I started to feel that allowing to specify node, rather than mesh, as LOD would be better. And probably the extension name should be replaced to EXT_node_lod or something like that. I will close this PR for now and I will make a new PR because it should be less confusing than updating this PR.

[takahirox]

Made a new PR #2228