Add support for binding the depth buffer as a color target. Fixes Kurohyo depth sorting

[hrydgard]

Fixes #9576

Kurohyo uses 2d backdrops with 3d characters rendered on top of them, and to fill in the Z buffer to make the characters sort against the background correctly, it writes a depth image directly to the Z buffer.

The PSP doesn't have any special mode for this, instead this game uses 565 color mode, binds the Z buffer as a color buffer, and writes the image as a color image.

I am not introducing a game specific compat flag, hoping for the best, heh (that it won't break something else).

Since such rebinding is entirely impossible on PC, but we can write explicitly to depth, we find the corresponding framebuffer that last touched that depth buffer, bind it normally for rendering, and we use a new rendering mode where:

• Color writes are masked away (don't happen)
• Depth writes are enabled in ALWAYS mode (no testing, just writing)
• The fragment color value is interpreted as a 565 value overlapping a 16-bit depth value, and the computed depth value is written to the depth buffer

Result - the characters should clip correctly!

WIP, only works in Vulkan in straight-through depth mode so far, but the other backends will be fixed too.

[hrydgard]

Hm, still has problems on Vulkan on Adreno, works on OpenGL though.

[dante3732]

It seems to work fine, I will write if there are problems.

[dante3732]

The character is partially visible through the car.

[dante3732]

Same

[hrydgard]

Ah interesting, think we might be mistakenly filtering the depth texture. I'll get a fix in soon.

[hrydgard]

I believe that change should take care of it.

[dante3732]

I believe that change should take care of it.

[hrydgard]

Okay, I guess not, but it kinda has to be that, or something very related... Can I have a GE dump or savestate file from that location?

[hrydgard]

Also, does it look like that in all backends (Vulkan, D3D etc?)

[dante3732]

Okay, I guess not, but it kinda has to be that, or something very related... Can I have a GE dump or savestate file from that location?

How make GE dump?

[hrydgard]

https://github.com/hrydgard/ppsspp/wiki/How-to-create-a-frame-dump

[dante3732]

Also, does it look like that in all backends (Vulkan, D3D etc?)

Only on D3D9 all fine.

[dante3732]

Okay, I guess not, but it kinda has to be that, or something very related... Can I have a GE dump or savestate file from that location?

Savestate https://drive.google.com/file/d/1-o47utQjASHd0EDx3EOaSkEfmXU_y5IU/view?usp=sharing

[dante3732]

GE Dump https://drive.google.com/file/d/10aHhuhEuLsi8i_8nlzidwocI7zlO9Dlh/view?usp=sharing

[hrydgard]

thanks!

[hrydgard]

Yeah there's something weird going on:

source image:

resulting z buffer:

Then it gets fuzzy because of some blur effect later in the frame, but the depth buffer is wrong right from the beginning. hmm

[hrydgard]

Alright, I think I got it sorted now. Thanks so much for the test case.