Sanitize tangents when creating mesh surfaces to avoid triggering the compressed mesh path in the shader #83179
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes: #83175
This was a fun one to track down.
In the shader we check if (tangent.x > 0 || tangent.y < 1.0), to see if we should run the uncompressed path. I wrongly assumed that we would never send (0, 1) to the shader because (0, 1) maps to a tangent of (0, 0, -1, 1), but a tangent of (0, 0, -1, 1) maps to (1, 1). However, (-1e8, 0, -1, 1) maps to (0, 1) and sometimes we create tangents that are (-1e8, 0, -1, 1). We can't represent (-1e8, 0, -1, 1) in octahedral tangents anyway, so its fine to just force all cases of (0, 1) to (1, 1)