LocoRoco2 Tropuca 1 rendering issue with Direct3D near the end

[nagisa]

What happens?

Direct3D 9/11:

What should happen?

OpenGL:

What hardware, operating system, and PPSSPP version? On desktop, GPU matters for graphical issues.

Radeon R5 230, Windows 10, PPSSPP v1.8.0.

[hrydgard]

So missing ground, missing loco, and strange colors? Or are the colors normal?

[nagisa]

From what I can tell this is a z-index issue or something similar. Somewhere further along the level after the "underwater" part ends (and thus the underwater background is cut off), you can see the missing layers (containing ground and the loco) get drawn from under the same cut-off. With that in mind I would bet that colours are fine – there will probably be some sort of colour blending going on.

This is the only segment and the only level so far where I could observe an issue like this.

Hope the explanation makes sense.

[nagisa]

I can try to make a screenshot of the cut-off later.

[nagisa]

Cutoff pictured. This cutoff, its inclination, etc move around smoothly. I thin I’ve seen in a few cases a corner of this "layer" as well.

[hrydgard]

That is really quite odd :) Thanks for reporting this. Which world and level is this?

[nagisa]

Not sure what you mean by "world", level is Tropuca 1.

[hrydgard]

Oh, right, was thinking LocoRoco 1.

[unknownbrackets]

Could you try exporting a GE frame dump? These help a lot.

See here for instructions - it's not hard and works on Android too:
https://github.com/hrydgard/ppsspp/wiki/How-to-create-a-frame-dump

You can zip that and then drag and drop it into a reply here.

-[Unknown]

[nagisa]

UCES01059_0001.ppdmp.gz

[unknownbrackets]

The missing draw is at 234/244, depth settings:

• Depth test src >= dst
• Depth write enabled
• Depth clamp enabled
• Depth buffer is filled with 1.0 / 65535
• Incoming depth is flat 49150 (fails depth test)

The depth of 1.0 is written at 111/244:

• Depth test src >= dst
• Depth write enabled
• Depth clamp enabled
• Depth buffer is filled with mostly 0, some others < 0.5
• Incoming depth is flat 93387 (clamped)
• Standard viewport (z center at 32767 and scale -32768) and offset
• Alpha test: a != 0, but there's no color so it's using material which passes
• Incoming verts are within 4096x4096 drawing area

It seems like this particular draw should not happen / not be clamped? If it's skipped, the red area shows as expected.

This is the correct PSP output:

-[Unknown]

[Sanaki]

This is easier to verify at the start of Tropuca 2, seemingly emitting from the hermit crab thing circling in the path after the two corals you push past. Unfortunately, I just tested this with both standalone and the libretro core, with both vulkan and OpenGL, and am seeing the issue across the board now. Tested on x86_64 Linux with f89d5b7. Tested the Windows libretro core via wine with OpenGL as well, same result.

Having bisected the issue, OpenGL started exhibiting the same buggy behavior in df6abe8. Vulkan seemingly always did. As of that commit, no renderer is capable of working correctly in these areas.

[unknownbrackets]

I was looking at this again and found an interesting typo in some of the tests I used for depth clamp - although I remember checking some games to validate this behavior as well. It was a simple, but stupid mistake: one of the coord's z was not being set correctly (rather, I set another coord twice.)

Will validate more, but this might explain some of the z clamp behavior from a few issues.

-[Unknown]

[unknownbrackets]

Okay, so the behavior that would fix this (and I suspect others) is that some additional triangles are culled, when:

• The depth clamp flag is enabled.
• All three vertices of the triangle have Z outside (but see below.)
• The vertices are all outside the range in the same direction.
• Irrespective of viewport parameters.
• Transform is enabled, of course.
• Applies to sprites too, but again BOTH coords must be outside or the full rectangle is clamped (if splitting to two triangles, this would be easy to get wrong.)

My previous tests on this were meant to be flat, but were incorrectly not flat.

The interesting thing is how "Z outside" is defined, because it's irrespective of viewport Z scale or center. Instead, after applying view, model, and projection - it's actually down to the bits.

Any Z value with greater magnitude than 0x3F8000FF (i.e. where its 24-bit truncation would be greater than 1.0) is considered outside. Same for negative. It could even check (value & 0x7FFFFF) >= 0x3F8000, since this only happens for that exponent value.

The next obvious question is: can the PSP tell post-transform Z apart from 0x3F800000 and 0x3F8000FF? That's a difference of 0.000030517578125. To answer that, I crafted large viewport parameters and used the following Z values with identity matrices:

• 0x3F7FFE00 (0.999969) -> 0x138A
• 0x3F7FFEFF (0.999985) -> 0x138A
• 0x3F7FFF00 (0.999985) -> 0x1388
• 0x3F7FFFFF (0.999999) -> 0x1388
• 0x3F800000 (1.000000) -> 0x1386
• 0x3F8000FF (1.000030) -> 0x1386

That's a pretty clear result to me; this might even explain some of the depth rounding trickiness we've seen. I assume once the matrices are applied, the result is normalized to something that maintains only 15 mantissa bits (excluding the implied.) Obviously, this makes some sense given we know most GE registers are 24-bit floats.

But this doesn't seem to control the actual clamping. Using the same test as above, if I pass a triangle with 2 vertices of 0x3F7FFE00 (within Z) and the third 0x3F8FF100 (1.124542, well outside), then it writes a depth of 0x01F4. This means it didn't clamp that third Z before viewport, but did it afterward.

If depth clamp is disabled, primitives get discarded if even one vertex is outside Z, and importantly this seems to follow the same rules as above for "outside Z." Again, viewport doesn't matter. This may explain some of the times we incorrectly rangecull when we shouldn't?

-[Unknown]

[unknownbrackets]

Additional testing: it does seem like negative Z (only) is clipped, but notably this is different from the above culling behavior.

That is, only when the depth clamp flag is on, the portion of a triangle with pre-viewport Z < -1 (after w, but before viewport) will be clipped. This doesn't happen on the positive side (i.e. Z = 12 is still displayed), and seems to happen regardless of whether the vertex is clamped.

This also only occurs for triangles. A rectangle will not be clipped on negative Z at all (another easy-to-get-wrong bit.)

For example, let's say we changed viewport Z scale from the typical 32767.5f to 3276.75f. Then we used triangles to draw Z=10 to Z=-10 from top to bottom. This would produce 65535 at the top, but stop where Z=-1 would be, even though that would only be 29490 or so.

This happens without regard to viewport or minz/maxz register values.

If depth clamp is off, the entire triangle would be drawn (no clipping) in the above example.

-[Unknown]

[hrydgard]

Really interesting with all this pre-viewport behavior!

Just realized that there's another way to cull triangles, which might be perfect: shaderCullDistance. https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/gl_CullDistance.xhtml

This unfortunately does not seem to be supported by Mali, but other than that it's widely supported in Vulkan at least.

[unknownbrackets]

Ah, I'd looked at this and clip before for guardband, but it's true this might be a good way to implement the cull without a geometry shader, given "Primitives whose vertices all have a negative clip (sic) distance for plane i will be discarded." It doesn't help as much for guardband / clamp off for the same reason, though.

-[Unknown]

[unknownbrackets]

Update on above: lines are also clipped based on Z (again, rectangles are not.)

For guardband handling, I had done some of my tests (and observations from games) based on the mistaken assumption that post-viewport Z was what mattered and clearly that was wrong. Speciifcally, I'd just played with viewport parameters to validate negative/positive, which in hindsight was pretty stupid of me.

Now that I'm testing properly, I'm seeing:

• Triangles/lines only avoid guardband culling based on negative pre-viewport Z.
• Rectangles cannot avoid guardband culling, and are always culled if any coord is outside the drawing area (X/Y) regardless of Z.
• Guardband behavior definitely doesn't depend on clamping to valid depth, i.e. even with a large viewport scale, the triangle will still be culled unless pre-viewport Z is less than -1.0.

So in other words, it's just as you'd expect: triangles avoid guardband culling entirely based on being clipped, which only occurs on the negative Z side (pre-viewport.) Based on my review of the dumps in the linked issues, I'm pretty sure this would fix most of the issues with guardband culling, and some of the issues with depth clamp.

Considering implementing this with cull distances (2, pos and neg) and clip distances (1), and I think we're pretty much always going to have 3 of those combined, outside Mali. But for Mali, it's trickier to come up with a good solution. Doing clipping using depth range might be problematic for correct depth values (probably could get it working, though), but I guess cull would just leave Mali out for now?

For the non-clamp case (no Z clip, and even one vert culls), could continue using the NAN strategy.

But haven't had time yet to implement the guardband rules in softgpu yet, which I intend as the first step to validate it.

-[Unknown]

[hrydgard]

Very nice to confirm all this! And of course softgpu will be an even better confirmation.

Not stupid at all to think that post-viewport was what mattered, that's how all modern hardware works and what I always assumed as well. Kinda makes me think someone accidentally inserted the clipper at the wrong position in the pipeline...

Rectangles are treated as constant-Z I believe, so they cannot cross the near Z plane and get clipped, so yeah, no avoiding the guardband...

At least modern Mali does support geometry shaders (even though their optimization manual discourages them) and geometry overload is rarely the problem for PPSSPP, so that might actually be a workable solution for Mali.. But absolutely fine to leave out for now, I can take care of that path later, I have plenty of Mali devices to test on.

[unknownbrackets]

Rectangles are treated as constant-Z I believe, so they cannot cross the near Z plane and get clipped, so yeah, no avoiding the guardband...

Yes, but interesting note: the culling behavior inside the guardband pays attention to both Zs, i.e. if the TL Z is inside [-1, 1] and the BR Z is outside [-1, 1] then it won't cull the rectangle, but if both TL and BR are outside in the same direction, it will. So clearly, that happens before it treats the rectangle as flat Z. Just trying to avoid assumptions...

-[Unknown]

[hrydgard]

Huh, that is somewhat surprising! Maybe they're just going through the line hardware path right up until the clipper, given that they have two points...