KTX2 ecosystem requirements

[donmccurdy]

This is a meta issue, tracking tasks required for KTX2 (and KHR_texture_basisu) to be considered production-ready. Related issues are grouped in the compressed textures milestone.

KTX2 material authoring:

• Complete pipeline for producing glTF files with KTX2 embedded, given glTF files with PNG/JPG embedded.
  • gltfpack converts textures to ETC1S or UASTC
  • libktx will be a useful part of this pipeline, but doesn't do the full glTF->glTF conversion itself.
  • glTF-Transform converts textures to ETC1S or UASTC
  • glTF-Pipeline can roundtrip files using KTX, converting textures in progress.
• Test pipeline against many glTF models, compare results in viewers (see below), and write up best practices if necessary.
• (Optional) Direct export to glTF+KTX2 from DCC tools (Maya, Blender, 3DS Max, ...)

KTX2 material clients:

• 2+ viewers/engines for testing glTF assets containing KTX2 material textures.
  • three.js#18490
  • babylonjs
  • glTF-Sample-Viewer?
• 1+ production-ready, lightweight transcoder.
  • KTX-Software provides a transcoder. Some remaining tasks surrounding where the final transcoder should be maintained, and minimizing the size of the transcoder.
  • AssemblyScript/WASM transcoder

IBL:

To be determined.

[donmccurdy]

I think next steps here might be:

1. Come up with a (tentative) recommendation on KHR_materials_basisu and independent data channels #1682.
2. Implement that recommendation in gltfpack and 2+ viewers/engines.
3. Test extensively to confirm or improve solution to KHR_materials_basisu and independent data channels #1682.

[abwood]

For KTX2 Material Authoring, it would be useful to update VSCode glTF tools to provide a mechanism to convert png/jpg to KTX2. We would need encoding/decoding in web assembly for this to happen, but I'm not certain of the state of wasm ktx2 encoding/decoding.

[zeux]

Yeah I've been thinking about wasm ktx2 myself. gltfpack is now (optionally) distributed as an npm package: https://www.npmjs.com/package/gltfpack - but that version can't compress textures because the compressor is external. I'm not sure what the state of threading is in wasm right now though - the compressor is really slow on a single thread :(

[donmccurdy]

I've heard WASM threading is working in Chrome, per https://twitter.com/prestomation/status/1218269561616683009. Don't know status beyond that. I'd love to be able to wrap gltfpack in a little web app — like https://squoosh.app/ but for glTF files. But certainly that's more of a nice-to-have for getting the KTX2 ecosystem ready for wider use. :)

[donmccurdy]

[Just added some updates to the OP on this thread.]

[zeux]

FWIW gltfpack now has full support for this, including ETC1S and UASTC and is able to use either basisu or toktx executable to do the actual conversion (only toktx path supports zstd super compression for UASTC).

[lexaknyazev]

Optimized low-level transcoders are public now. The collection currently includes ASTC, BC7, and RGBA32 targets for UASTC source data.

https://github.com/KhronosGroup/Universal-Texture-Transcoders

[SaschaWillems]

Not sure if this is the correct spot to ask, but are there any KTX2 image viewers available?

[donmccurdy]

@SaschaWillems we could use more in this category, but https://sandbox.babylonjs.com/ will preview KTX2 images well.

[abbaswasim]

@SaschaWillems Kram claim to support ktx and ktx2 with the following limitations:

KTX - only uncompressed, mip levels are unaligned to block size from 4 byte length at chunk 0 
  metadata/props aren't standardized or prevalent
  libkram supports only text props for display in kramv

KTX2 - works in kram and viewer, has aligned levels of mips when uncompressed, 
  libkram supports None/Zlib/Zstd supercompression for read/write
  libkram does not support UASTC or BasisLZ yet

[andreasplesch]

Is it time to update the check list on the top ?
What is the most light weight solution to support KHR_texture_basisu, perhaps including sacrificing least common format combinations ?
ktx-parse seems light weight for parsing. Transcoding still seems heavy with large wasm files required. https://github.com/KhronosGroup/Universal-Texture-Transcoders seems incomplete.
Would support for ktx2 containers with DDS style/GPU supported compression and no transcoding be considered useful, as a first step towards replacing dds which still seems popular ?
Is direct GPU/driver support for basisu compression on the horizon ? Or already considered outdated or out of scope for GPU support ?
Apologies for all the questions, but the ktx2/basis ecosystem is quite complex.

[donmccurdy]

Originally this was written around the KTX2 launch. Perhaps this issue shouldn't be the permanent home for the list. Is KTX2-specific software wanted on https://github.khronos.org/glTF-Project-Explorer/, or on a KTX2 equivalent?

What is the most light weight solution to support KHR_texture_basisu ?

I think both the Basis Universal and KTX Software "MSC" transcoders are of similar weight, a couple hundred kilobytes gzipped. I do wish that KhronosGroup/Universal-Texture-Transcoders were more complete, for that reason. But I suspect it will require volunteers or resources to complete that.

Would support for ktx2 containers with DDS style/GPU supported compression and no transcoding be considered useful...

Personally I would consider it useful, yes. At least as a standalone capability – whether it belongs in an official glTF extension is probably a longer discussion. I believe the https://github.com/KhronosGroup/KTX-Software maintainers have indicated interest in providing compression tools but don't have the time/resources to do it. It may (?) already be possible to pack the files if you have the compressed data already, or ktx-parse could do that (packing into KTX2, not GPU compression...) with a bit of scripting.

Is direct GPU/driver support for basisu compression on the horizon ?

I suspect that's not something the GPU itself can provide. Implementing it in drivers or browsers would be very interesting, though I haven't heard discussion of either at this point.

[alecazam]

kram can load and generate BC/ASTC/ETC2 textures stored in KTX, KTX2, and DDS files. There is a library that is all C++, and utils for quick load from mmap-ed archives. KTX would need compressed in the archive, and KTX2 can compress the mips and simply be archived.

I haven't seen any adoption of KTX2 by any of the IHVs or consoles. Lack of viewer support, Photoshop/Figma support, and encoders. I tried to address all of these, but the Photoshop viewer was too much work, and the API unchanged for 30 years.

kram has been in production use for 4 years now. PNG and DDS seems to be good enough for most tools. And consoles have to convert to platform-specific textures in the end that are pre-tiled. We never use basis, and the time to encode are too high. I was considering adding load of basis data for the kramv viewer. My work uses KTX, and we haven't moved to KTX2 yet.

Also unclear with sparse textures, how that fits into the KTX2 roadmap. Material indirection alleviates some of the need for sparse textures.