[3.x] Fix navigation merge errors on too small triangles

[Scony]

Fixes #56786 and part of #57254

See #56786 (comment) for detailed explanation.

Please note this change may potentially have a severe impact on navigation, so it would be nice if someone else can test the lack of errors as well as Agents traversing navmesh.

Once this is accepted I'll open similar for master.

@Duroxxigar @AndreaCatania

[Scony]

I've created a proper demo to visualize paths going through "small" triangles which I've skipped and it turns out the triangles are not bypassed. They're removed. See the demo with drawing random paths on generated navigation with 2 merge errors:

And hence I need to improve it further so they're taken into account.

[Scony]

While analyzing further I've found yet another problem - when calculating cell key (id) we're dividing Y coordinate by cell_size while cell_height should be used.

With additional precision improvement, everything works as expected, but it's still not perfect.

I'll sync regarding it with @AndreaCatania offline.

[Scony]

I had to change detail/sample_max_error default to 5 so that vertices computed by recast are aligned with the voxels grid much better. This way we're avoiding problems even on very complex meshes while sacrificing some navmesh details.

In practice (flat meshes or meshes with some gentle hills) this makes no difference while this way we're able to avoid potential problems.

The grid alignment algorithm will have to be re-done anyway, but IMO this commit at least polishes what's already there.

[AndreaCatania]

It looks nice! Great job, just a small comment on the function get_point_key.

[AndreaCatania]

Looks good!! Thanks 🍉

[akien-mga]

Thanks!