Idea: Fidelity Tests like model-viewer has

[hybridherbst]

This is an awesome resource.

I'm not sure if you're aware of that already, but model-viewer has a fidelity test result page and a format for how to integrate other renderers / engines:
https://modelviewer.dev/fidelity/

They have a pretty simple JSON format for how IBL, camera etc. are specified to ensure matching conditions between renderers.

For automating along with what model-viewer already does, renderers would need to be added here:
https://github.com/google/model-viewer/tree/master/packages/render-fidelity-tools/src/components/renderers

[cx20]

Sorry for the delay in replying. Thanks for the great ideas.
I too wish I could do a validation like model-viewer.
However, I lack the skills and time to do so.

BTW, I am making the following efforts to improve testing with gltf-test.
#68

[hybridherbst]

Thanks for responding!
I think the interesting thing would be that the validation / image generation could run as part of what model-viewer already does. Maybe @elalish has some insights as well into whether wider comparisons would be desired inside model-viewer.

Regarding #68, that's a good start and something definitely lacking from glTF currently (defining which IBL, exposure, tonemapping a renderer should use)! Seems like model-viewer actually had similar issues, here's what looks like a shell script to prepare an IBL for filament rendering: https://github.com/google/model-viewer/blob/master/packages/render-fidelity-tools/test/renderers/filament/prepare-ibl.sh

[elalish]

We'd be thrilled to have more renderers added to our fidelity test suite. It's not generally a heavy lift, especially if your package is on npm. If you're interested I'd be happy to share some pointers (like old PRs that added other renderers).

[cx20]

@ybridherbst Sorry for my late reply.

defining which IBL, exposure, tonemapping a renderer should use

Thank you for sharing your insights.
I have had a lot of advice from the Filament and Khronos peoples that if the exposures and tonemapping are not set up correctly, the results displayed will be different. I need to understand more about exposure and tonemapping besides IBL.
google/filament#3366 (comment)
KhronosGroup/glTF-Sample-Models#340 (comment)

@elalish I would like to understand how the fidelity test works.
I would be glad to know if you have any other renderer samples or pointers that would be helpful.
BTW, My skill set is old and I may need to start by getting familiar with TypeScript.

[cx20]

Thanks again to everyone who has been advising me on this repository.
@hybridherbst @elalish @emackey @donmccurdy @willeastcott @slimbuck @emadurandal @Reon90 @06wj @sonygod @DRx3D

I'm aware that this repository mechanism is a legacy mechanism.
I don't want to break links such as URL references from outside, so I'm thinking of preparing a new repository.
I have set up a discussion room. If you have any other ideas, I would appreciate your contribution.

https://github.com/cx20/gltf-test/discussions

Thanks!

[Reon90]

@cx20 What is the status of current repo? Are you planning to continue contributing or move your effort to new one?

[cx20]

@Reon90 I'm planning a new (modern) repository, but I think it will take some time.
Until then, I think we will continue to update this repository.

See below for our plans.
#560 (reply in thread)