BufferAttribute type / precision conversion API

[FrostKiwi]

BufferAttribute.convert()

e0ae5b1 introduces a .convert() function in BufferAttribute to allow for precision conversion and buffer quantization of attributes like position, uv etc.
This allows one to drop any BufferAttribute to a lower precision like HalfFloat or Int8 to save on VRAM (and RAM).

Thanks to already existing data conversion like

three.js/src/extras/DataUtils.js

Line 150 in 69a97bf

function toHalfFloat( val ) {

and its already existing use in setter and getters of the various BufferAttribute types like

three.js/src/core/BufferAttribute.js

Line 876 in 69a97bf

setX( index, x ) {

ThreeJS already has the ability to convert to and from various types for values, but not for buffers as a whole yet. This commit creates a conversion function which makes use of these preexisting data conversions to do exactly that.

The idea is to allow all the various Geometry generation functions like SphereGeometry.js to output with lower precision, without introducing a bunch of per-geometry generator custom code.

I see some potentially strong VRAM savings for this like dynamically imported 3D Satellite maps.

SphereGeometry( ... , precision = { ... } )

To showcase this, in dd01e63 I extended SphereGeometry.js to (optionally) allow to set precisions for each BufferAttribute or entirely remove said attribute. It also clarifies which precisions were used previously by default, which was Float32. Default case remains as is, nothing changes here, no API break.

E.g. this allows the creation of a sphere with no normals, Half-Float Positions and 8-bit UVs.:

new THREE.SphereGeometry(radius, ... ,
    {
        position: THREE.Float16BufferAttribute,
        uv: THREE.Uint8BufferAttribute,
        normal: null
    }
);

Motivation and VRAM savings in GroundedSkybox

Commit 0ecfbb9 uses these new parameters to get a 50% VRAM and RAM size reduction of GroundedSkybox geometry on default settings with no visual impact.

In #27422 & #27448 @elalish changed the previous shader based GroundProjectedSkybox to the new geometry GroundedSkybox. To get the newly introduced rounded curve from floor to dome, GroundedSkybox generates 65,024 triangles on default resolution and deforms it, with full precision BufferAttributes and unused normals.

GroundedSkybox with default resolution	Wireframe of GroundedSkybox with default resolution

Due to how SphereGeometry.js works, it produces very dense mesh on the flat floor. The 65,024 triangles default chosen by @elalish is the only way to get enough geometry and avoid distortions of the spherical texture on the curved transition.

The VRAM and RAM usage of the default GroundedSkybox is as follows:

IndexBuffer: `Uint16`, itemSize: `1`, Length: `195,072` = 390 KB
Position Attribute: `Float32`, itemSize: `3`, Length: `99,459` = 398 KB
Normal Attribute: `Float32`, itemSize: `3`, Length: `99,459` = 398 KB
UV Attribute: `Float32`, itemSize: `2`, Length: `66,306` = 265 KB
---------------------------------------------------------------
Total: `1.45 MB`

0ecfbb9 now requests a Sphere with the following precision properties:

{
    position: Float16BufferAttribute,
    uv: Float16BufferAttribute,
    normal: null
}

This results in no visual change and the following RAM and VRAM usage:

IndexBuffer: `Uint16`, itemSize: `1`, Length: `195,072` = 390 KB
Position Attribute: `Uint16`, isFloat16BufferAttribute: `true`, itemSize: `3`, Length: `99,459` = 199 KB
UV Attribute: `Uint16`, isFloat16BufferAttribute: `true`, itemSize: `2`, Length: `66,306` = 132 KB
---------------------------------------------------------------
Total: `0.72 MB`

The 50% GroundedSkybox VRAM savings aren't that important in the grand scheme of things, but this shows how I think a Three.JS user might want use such a attribute precision api to get VRAM savings on dynamically generated geometry and avoid hard crashes due to strict VRAM limits that some iPhones impose in WebGL.

0ecfbb9 also adds a material name in GroundedSkybox, so it doesn't show up as blank in Debuggers like https://github.com/utsuboco/r3f-perf

Stuff to decide

If we wanna merge this, I'll properly write the docs and stuff, but some things need to be clarified first:

• I would add the optional precision = { ... } to all the geometry constructors in src/geometries or this this out of scope? Rename it to attributes = { ... }?
• As I understand, there is no way to fromArray create all the types yet, so this leads to the bit awkward expression of this.setAttribute( 'position', new Float32BufferAttribute( vertices, 3 ).convert( precision.position ) ); which creates a Float32 Buffer to immediately convert it. Is this fine or do we need to do something about this?
• I guess using the classes directly as parameters would break the toJSON() stuff, something I ignored so far in this PR. So we would need to switch out THREE.Float16BufferAttribute for constant like float16. But the already existing constants like HalfFloatType are texture related and there is no 8bit vs 16bit int constant in the first place, so what to do? Create a new set of Geometry constants and correct BufferAttribute.gpuType which has been using Texture constants for Geometry Attributes?

[github-actions]

📦 Bundle size

Full ESM build, minified and gzipped.

	Before	After	Diff
WebGL	355.34 84.47	355.34 84.47	+0 B +0 B
WebGPU	618.89 171.93	618.89 171.93	+0 B +0 B
WebGPU Nodes	617.5 171.68	617.5 171.68	+0 B +0 B

🌳 Bundle size after tree-shaking

Minimal build including a renderer, camera, empty scene, and dependencies.

	Before	After	Diff
WebGL	487.5 119.33	487.96 119.45	+462 B +118 B
WebGPU	691.15 187.57	691.76 187.78	+606 B +211 B
WebGPU Nodes	641 174.81	641.6 175.01	+606 B +202 B

[FrostKiwi]

In e272553 Fixed GitHub Actions Linter complaint https://github.com/mrdoob/three.js/actions/runs/20463774152/job/58802394771?pr=32614

Also here is a playground for the different Precisions. Just saw the GitHack demo on #32613 and first time hearing about it, thought it was neat way to showcase the precision api. In a different branch, modified the existing grounded Skybox example to have precision select boxes.
Precision demo: https://rawcdn.githack.com/FrostKiwi/three.js/ce212c1cf09fc2d2aa6cf65eace74e0a8f74e885/examples/webgl_materials_envmaps_groundprojected.html

Kinda funny how dropping down to 8bit ints for position shows up as squares and how making the positions any unsigned int makes the skybox all mush into one Quadrant, as there are no more negative coordinates possible.