First-class support for Basis textures in glTF

[zeux]

I'm looking into adding support for encoding textures using BasisU to gltfpack and wondering what the plan is for Basis support in glTF.

Right now three.js can load Basis encoded textures from glTF if:

1. The texture is specified as a URI with .basis extension
2. BasisTextureLoader is added as a handler for .basis extension

However, there does not seem to be any way to do this if the image is embedded - mimeType is effectively ignored as far as I can tell when loading embedded images, so you can't specify a handler.

There is an upcoming KHR_image_ktx2 extension that will allow including Basis files wrapped in a KTX2 header, however if the image is embedded then it will have a MIME type of image/ktx2 and, again, some special logic will be necessary to load this.

Just wondering what the plan for supporting all of this is, and if I can help in some way.

[Mugen87]

It seems this issue is better placed here, no?

https://github.com/KhronosGroup/glTF

[zeux]

This issue is specific to three.js handling; glTF already has a plan for this but as mentioned above mime type differentiation is crucial for it to work in three.js.

[donmccurdy]

The plan for glTF is KhronosGroup/glTF#1612. That was recently updated, see KHR_texture_basisu. The extension specification is stable enough for prototyping, although I don't have any example KTX2 assets yet.

I expect the changes required for three.js will be relatively minor, similar to the current MSFT_texture_dds extension – which I think we should remove support for, actually, since it is really for Microsoft tools. The API might be:

var loader = new THREE.GLTFLoader();
loader.setBasisTextureLoader( new THREE.BasisTextureLoader() );
loader.load( 'model.glb', function ( gltf ) { ... } );

For now this will require a bit of Basis-aware logic in GLTFLoader, like:

three.js/examples/js/loaders/GLTFLoader.js

Lines 1972 to 1980 in d380853

	if ( textureExtensions[ EXTENSIONS.MSFT_TEXTURE_DDS ] ) {
	source = json.images[ textureExtensions[ EXTENSIONS.MSFT_TEXTURE_DDS ].source ];
	} else {
	source = json.images[ textureDef.source ];
	}

three.js/examples/js/loaders/GLTFLoader.js

Lines 2004 to 2012 in d380853

	var loader = THREE.Loader.Handlers.get( sourceURI );
	if ( ! loader ) {
	loader = textureExtensions[ EXTENSIONS.MSFT_TEXTURE_DDS ]
	? parser.extensions[ EXTENSIONS.MSFT_TEXTURE_DDS ].ddsLoader
	: textureLoader;
	}

With the proposal in #11682, this could be done through a plugin system, but that's probably a longer-term direction and not necessary for this.

[zeux]

Thanks, that makes sense - so there's going to be some special case code in the loader. I'm assuming that this also means that there are no plans to support Basis files outside of the KTX container through the glTF flow fully (as mentioned in the original issue, this doesn't work atm due to mime type).

Closing this since I originally thought mime type based loading might be important but this can be solved without it I suppose.

[donmccurdy]

I'm assuming that this also means that there are no plans to support Basis files outside of the KTX container through the glTF flow fully.

Correct, I don't believe a glTF extension will reference .basis files or mime types directly. Only within a KTX2 container. Feedback on KhronosGroup/glTF#1612 is welcome if you have any concerns about that!