simplify: Add color support to simplifyPoints #619
Conversation
Following the new reservoir code, we can now easily accumulate colors into the reservoirs, and add the color error to the position error with a user-supplied weight. The color error is squared distance; as such, we scale it by the squared weight. In addition, we now use color alpha as a point weight; the weighted sum will thus bias towards more opaque points. Note that the alpha is not accounted for during point selection.
|
A few options for handling alpha:
... one issue with all of these is that no point clouds in my test set have alpha, to the point where I'm not sure we even need to support alpha to begin with. Need to see if I can find better test assets here... |
|
I found a grand total of one Sketchfab asset with alpha :D but for that asset whether we support alpha or not seems completely immaterial as the alpha is relatively uniform with few deviations across the entire cloud, whereas the current weighting and potential improvements to candidate selection only matter when alpha is discontinuous. There's also a downside wrt current implementation in that it requires 4-float colors; since alpha is rarely useful, this unnecessarily restricts the operation for glTF 3-float colors by requiring to expand the buffer to 4-floats. That said, without alpha the solution feels a little incomplete :-/ |
There are a few different options for how we should support alpha during point simplification, and they are all difficult to evaluate properly because point clouds with meaningfully diverse alpha that would affect the simplification noticeably are vanishingly rare. Additionally, it might be useful to support float3 inputs for densely packed arrays, as that may make glTF integration easier -- gltfpack pads all RGB colors with 1.0, but not all other processors might. Since meshopt_simplifyPoints is experimental we can wait until we get feedback on this and potentially change it, or maybe introduce meshopt_simplifyPointsWithWeight in the future where we could supply weights as a separate channel (and the callers could decide whether to use alpha or not?).
|
Since we're lacking good test cases, and |
|
Motivating example (scene from https://sketchfab.com/3d-models/little-cabin-3b9fbbd468e54227ad7ef7e74f8f479c, simplified with ratio 0.2): Before: After: The colors on the simplified result are less noisy and it is more representative of the source scene. |
Following the new reservoir code, we can now easily accumulate colors into the reservoirs, and add the color error to the position error with a user-supplied weight. The color error is squared distance; as such, we scale it by the squared weight.
In addition, we now use color alpha as a point weight; the weighted sum will thus bias towards more opaque points.
Note that the alpha is not accounted for during point selection.