Ratchet & Clank: Tools of Destruction/Resistance: Fall of Man instanced mesh system broken

[JimScript]

Quick summary

In some early Insomniac games, only one foliage instance can be visible at a time

Details

In Ratchet & Clank: Tools of Destruction, New minor foliage will replace rather than supplement other older foliage.
Foliage fades in as normal:

And then when a new foliage loads in it completely replaces the old one:

Now this would be a minor nitpick, if Resistance: Fall of Man (another insomniac game) didn't use the system more extensively and for a gameplay purpose like hiding mines.

Here is an RSX Capture:
BCUS98127_20210823151612_capture.zip

And here is the system:

• SYS: RPCS3 v0.0.17-12659-d0451932 Alpha | HEAD
• SYS: AMD Ryzen 5 3600 6-Core Processor | 12 Threads | 15.95 GiB RAM | TSC: 3.600GHz | AVX+ | FMA3
• SYS: Operating system: Windows, Major: 10, Minor: 0, Build: 19043, Service Pack: none, Compatibility mode: 0
• RSX: Found vulkan-compatible GPU: 'GeForce RTX 2070 SUPER' running on driver 456.38.0.0
• SYS: Firmware version: 4.85

[JimScript]

Well I might as well show an example of Resistance: Fall of Man:

RSX Capture:
BCUS98107_20211120040735_capture.zip

[JimScript]

Sorry if this is confusing, here is a video of it in action:

RFOMFoliageProblem.mp4

[JimScript]

Here is a visual aid on the differences between ps3 and rpcs3 foliage sorting logic in Ratchet and Clank Future: Tools of Destruction.
PS3:

RPCS3:

Sorry I couldn't get a photo or video of a real ps3, I don't own a capture card or something like that. But there is a surprising amount of foliage missing in these games because of this error, more so than I originally thought.

[JimScript]

Ok, so I figured my phone picture was good enough to compare so...
PS3:

RPCS3:

And with this I see that some of the foliage is in clusters that are never seen even in the culling order that RPCS3 produces for some reason. I wonder what causes it to only show one mesh at a time, but 2-3 meshes in Resistance: Fall of Man?

[JimScript]

Along with the previously stated sorting issue, it appears that some foliage is rendering only as an outline. This example is in Quest for Booty, but it has much the same rendering as ToD does, so it is applicable to both games:

RSX Capture:
NPUA80145_20220215141422_capture.zip

[JimScript]

OK, so I have been playing around with renderdoc to see what the problem is and this is what I found. The missing foliage is well and truly not there, no draw calls except for one solitary plant in this foliage system, which leads me to believe this is a cpu problem. I would guess this is an spu problem as a task like this seems too minor for the ppu. What I think what is suppose to happen is this spu function is to go around and update the foliage to different levels of opacity as the camera gets closer, but in RPCS3 the function stays at the first foliage it can find until it has to make a unique mesh fully opaque wherein it goes to the new foliage and dumps the old one.

As for the outline shader, I may not know much about these things, but I don't think a single minor mesh is suppose to draw over a thousand times in a single frame. A third of the frame render time is dedicated to that one plant, and to add to that the first several hundred draws just draw the same few pixels on the screen. Although that could be just how the game does it, but that seems like a horrible optimization.

Anyway that is just some speculation on my part.

[kd-11]

Needs testing on AMD/INTEL. NVIDIA is notorious for having rounding issues that can cause strange visuals like this.

[Shadowsplace]

I tested it on two different machines in Tools of Destruction and got the same result.

• SYS: RPCS3 v0.0.20-13288-4be9e0d3 Alpha | HEAD
• SYS: Intel(R) Core(TM) i7-4700MQ CPU @ 2.40GHz | 8 Threads | 15.92 GiB RAM | TSC: 2.394GHz | AVX+ | FMA3
• SYS: Operating system: Windows, Major: 10, Minor: 0, Build: 19043, Service Pack: none, Compatibility mode: 0
• RSX: Found vulkan-compatible GPU: 'GeForce GTX 770M' running on driver 425.31.0.0
• SYS: Firmware version: 4.88

Nvidia.Capture.mp4

• SYS: RPCS3 v0.0.20-13288-4be9e0d3 Alpha | HEAD
• SYS: AMD Ryzen 3 2200U with Radeon Vega Mobile Gfx | 4 Threads | 7.66 GiB RAM | TSC: 2.495GHz | AVX+ | FMA3
• SYS: Operating system: Windows, Major: 10, Minor: 0, Build: 19043, Service Pack: none, Compatibility mode: 0
• RSX: Found vulkan-compatible GPU: 'AMD Radeon(TM) Vega 3 Graphics' running on driver 2.0.173
• SYS: Firmware version: 4.88

AMD.Capture.mp4

Both using Vulkan but OpenGL doesn't seem to change anything (on Nvidia at least).

[JimScript]

I think there are two separate issues in this issue, the foliage sorting and the outline shader. I think what kd-11 is talking about is the shader issue. What you have tested is the sorting issue. If you want an example of the shader problem in action, go to hoolifar island in QFB. If you don't have QFB, Planet Mukow's foliage has the same problem, so you can check it there.

[JimScript]

Hey @Shadowsplace, can you check the foliage in the imperial fight festival with your AMD machine? It is different than the ones on cobalia and I was wondering if @kd-11 is right about this particular part of the issue is vendor related. And sorry I couldn't do it, I don't own any AMD gpus. Here is an example of the problem on Nvidia hardware:

If your AMD machine still makes the foliage here an outline, then it isn't vendor related.

[Shadowsplace]

Sorry about the delay. Same specs (RPCS3 v0.0.21-13333-aafd74f9 Alpha | HEAD build)

• Nvidia
  

• AMD
  

It appears the issue isn't vendor related.

[JimScript]

Thank you for testing this, Shadowsplace, with this we now know both issues are not vendor related. I still think that the sorting part is a spu(or ppu) problem, but I don't know what could cause this particular shader to render as an outline.

[JimScript]

I was looking at an old resistance video when I noticed that the amount of foliage missing is worse than Tools of Destruction's. Here is a screenshot of RPCS3 in roughly the same location as 0:28 in the Somerset(40p) map:

[homembarata]

Wish I had seen this thread before...

It's not just grass, but small stuff like small rocks are also missing. Here's a later level in Resistance:

In the original hardware there are a bunch of small debris on the floor (not exactly the same screen but the same area):

My guess is that this is an instanced mesh system, which is common for this kind of small objects and foliage. And the emulator is only rendering one instance at a time. Just throwing some ideas here.

[JimScript]

It probably is that since I made a Renderdoc of this issue and didn't find any of the missing meshes anywhere in it. But it also seems to be different than the fade in/out as that works correctly with the one mesh that does render. Also certain multiplayer maps can run more than one at a time for some reason.

[isJuhn]

I believe ratchet games culls lots of stuff in case of low fps before the rendering stage at all. I've checked a lot of ToD renderdocs and if it runs too slow a lot of stuff is not even in the renderdoc. But at the same time the foliage is never there no matter the fps so it might be some stage before the render stage that removes them, and not some rsx culling.

[JimScript]

I have thought a bit about the possibility of this being related to a performance cull, and now I think that this culling system might not be receiving a proper(or any) performance report. That would explain why there is generally only one mesh rendered at a time, because to the instanced mesh system the PS3 is getting absolutely slammed, so it renders the bare minimum amount of meshes it can(only one). That would also explain why Grimsby(40p) in CTF can render more meshes at once, because it is getting a lighter report. Now this is all guesswork, but it is the only thing that I can think of that would cause this.

[Darkhost1999]

I have to host the video somewhere else because GitHub will say the video is corrupt or too big.
Oblivion (looks better upscaled and if I load all of the environment first.) Trees lose leaves as you approach them.
https://youtu.be/rBs_uqE2V1k
RPCS3.log

OpenGL Comparison
https://youtu.be/8nowDtWDMJY
RPCS3.log

I know this is a different game. Spooky. But the issues look the same.
I mean let's compare Oblivion only loses leaves not entire plants.
All 3 games lose foliage at similar distances to each other.
The other games plants are on the ground. Oblivion's leaves are off the ground.
If you want a separate issue ticket just lmk.

I can test on my Intel uhd 620 igpu laptop later for a comparison there? Would that be helpful or because igpus are not officially supported would it be meaningless?

[kd-11]

This one never got triaged. I've added it to backlog.

[JimScript]

I have to host the video somewhere else because GitHub will say the video is corrupt or too big. Oblivion (looks better upscaled and if I load all of the environment first.) Trees lose leaves as you approach them. https://youtu.be/rBs_uqE2V1k RPCS3.log

OpenGL Comparison https://youtu.be/8nowDtWDMJY RPCS3.log

I know this is a different game. Spooky. But the issues look the same. I mean let's compare Oblivion only loses leaves not entire plants. All 3 games lose foliage at similar distances to each other. The other games plants are on the ground. Oblivion's leaves are off the ground. If you want a separate issue ticket just lmk.

I can test on my Intel uhd 620 igpu laptop later for a comparison there? Would that be helpful or because igpus are not officially supported would it be meaningless?

I'll admit, I don't see the similarity. In TOD & R:FOM they both can only show one or few pieces of foliage at a time with them popping around as you get closer or farther, this seems more gradual(except for the trees suddenly popping up in the background in the Vulkan example). I wouldn't even say it was the outlined plants part of this issue, as that remains consistent at any distance with any renderer. But maybe the underlining renderer could be used differently?

[Darkhost1999]

I'll admit, I don't see the similarity

You never see similarity to ANYTHING I say so IDK
OGL did same thing as vulkan tree pop in I just needed to load the water so there actually is no difference between renderers

[Darkhost1999]

I definitely see as you described this
RPCS3:

[JimScript]

I'll admit, I don't see the similarity

You never see similarity to ANYTHING I say so IDK OGL did same thing as vulkan tree pop in I just needed to load the water so there actually is no difference between renderers

I'm sorry about that. But issues with Oblivion were fixed with the PC uptime problem I posted, so maybe it is similar and would be fixed with this.

[JimScript]

I think the reason why some of the ground plants look hollowed out and spindly is that that the alpha is limited to around 127 and anything past that is all zeroed out for that shader, making the bulk of it not render. This is the alpha of the QFB plant edited to demonstrate:

And the original to compare:

[JimScript]

I've found something interesting about the hollowed out shader:

It doesn't block shadows, even though it does on a real PS3. It makes sense since this is a transparent material, but what if it can block shadows on PS3 because it's filled with an opaque material, like it fades from transparent into opaqueness. And what if it's hollowed doing around a thousand draw calls because it's caught in a loop trying to get the opaque version but it's just getting the transparent version over and over again until the emulator gives up? I guess I'm making a lot of assumptions here.

[JimScript]

Turns out it does draw the opaque part first in another color pass, but it only does so on a very slim threshold. That must have been the few pixels I saw being drawn first before, I could only really see it now when the LOD bias offset is set to 10 since the threshold is so tight.
Opaque Shader Drawn:

Translucent Shader and Shadow Drawn:

Still don't know why both the opaque and translucent parts draw hundreds of times each frame though.

[kd-11]

Test #15471. It's not directly aimed at this issue (I haven't deep-dived this one yet) but the cause could be related.

[rogeriodomingos]

I can still replicate the issue in Resistance: Fall of Man as of version 0.0.31-16351-ea891829.

• SYS: RPCS3 v0.0.31-16351-ea891829 Alpha | master
• SYS: AMD Ryzen 5 3600 6-Core Processor | 12 Threads | 15.93 GiB RAM | TSC: 3.600GHz | AVX+ | FMA3
• SYS: Operating system: Windows, Major: 10, Minor: 0, Build: 22631, Service Pack: none, Compatibility mode: 0
• RSX: Found Vulkan-compatible GPU: 'AMD Radeon RX 6600 XT' running on driver 2.0.299
• SYS: Firmware version: 4.90

Untitled.video.mp4

[JimScript]

You know, with Resistance: Fall of Man showing multiple clusters rather than a single plant, could that in fact be a different issue? I mean this issue turned out to be different from the one that affected Resistance 2 and ACIT on NVIDIA, who's to say this isn't a different issue either? Still, this is only speculation, I won't make another issue until we know for sure.

[rogeriodomingos]

It might be the same issue, but it has improved. Here is a screenshot with version 0.0.30-15956:

And version 0.0.30-16096 with the same behavior as the previous video I posted:

#15471 doesn't seem to have made any difference in this case. I'll try to narrow down the commit that made the improvement.

[rogeriodomingos]

It was actually this pull request that did it:
#15065

I see that you also reported an improvement in Tools of Destruction so it might be related.

Summing things up: #15065 improved the issue but #15471 had no effect.

[kd-11]

This one is weird. I'll have to investigate assuming it's multiple issues. Hopefully it's not related to depth precision since that can only be correctly resolved with software rendering.

[kd-11]

Ok, nothing too special here. It's what I expected. PS3 doesn't have dedicated instancing hardware, but it can do instancing through like a million different methods by maniupulating the FIFO puller. We just didn't have a pattern match for the registers this game is using to implement it.
Fix incoming.

[kd-11]

#15483 implements the feature, though in a very suboptimal way right now. A lot more work is needed to get this ready for merge.