[Regression] Perfomance drop after LLVM 16.0.0 update (#13534)

[Ordinary205]

Quick summary

After updating RPCS3, the performance gets decreased.

Details

Before: 14845

After: 14854

Build with regression

v0.0.27-14854-6fbca1ac

Attach two log files

14845:
RPCS3.log.gz

14854:
RPCS3.log.gz

Attach capture files for visual issues

No response

System configuration

AMD Ryzen 5900X 12-Core Processor | 24 Threads | 15.89 GiB RAM | RTX 3080 driver 531.41.0.0 | Windows 10 Pro 22H2

Other details

No response

[AniLeo]

Not valid as range has more than one build, might be related to the LLVM update but we need the exact build

Also, needs retest with #13629

[solarmystic]

Definitely seems in line with the minor performance regressions i noted in the LLVM16 PR, but please test with 14846 to confirm that the LLVM16 PR is the responsible one @Ordinary205 https://github.com/RPCS3/rpcs3-binaries-win/releases/download/build-fb88e1c1c933a792f96600206ed03a07cd7c3661/rpcs3-v0.0.27-14846-fb88e1c1_win64.7z

See my comment here for prior testing done in the LLVM16 commit #13534 (comment)

[Ordinary205]

Definitely seems in line with the minor performance regressions i noted in the LLVM16 PR, but please test with 14846 to confirm that the LLVM16 PR is the responsible one @Ordinary205 https://github.com/RPCS3/rpcs3-binaries-win/releases/download/build-fb88e1c1c933a792f96600206ed03a07cd7c3661/rpcs3-v0.0.27-14846-fb88e1c1_win64.7z

I didn't notice that the 14846 build was causing the regression. I thought it was the recent update but nevermind, I'll test the 14846.

[Ordinary205]

Heres the correct results:

Before - 14845:

After - 14846:

14845: RPCS3.log.gz
14846: RPCS3.log.gz

[Ordinary205]

I can confirm that the 14846 is a regression.

[solarmystic]

Thanks for the verification that the LLVM16 PR is the responsible one.

[Darkhost1999]

Is the performance restored in #13629

[Ordinary205]

@Darkhost1999 I did some performance tests using 14845 and 14855.

InFamous 2:
14845:

14855:

Demon's Souls:
14845:

14855:

God of War III:
14845:

14855:

Sadly, some games like InFamous 2 and God of War III still has minor performance regression, but unlike Demon's Souls it gained a minor performance boost.

InFamous 2 Log:
RPCS3.log.gz
RPCS3.log.gz

Demon's Souls log:
RPCS3.log.gz
RPCS3.log.gz

God of War III log:
RPCS3.log.gz
RPCS3.log.gz

[solarmystic]

@Darkhost1999 I did some performance tests using 14845 and 14855.

@Ordinary205 Looking at your Infamous 2 screenshot results, performance is improved now and back to normal?

14845 - 33.88 FPS
14846 (LLVM16 build) - 29.24 FPS
14855 (LLVM16.01) - 35.99 FPS

So I'd say the Infamous 2 performance regression is fixed right? In fact, it's faster than even the older pre LLVM 16 master now from your own screenshots.

As for GOW3, 114 FPS and 113.94 FPS is within margin of error (less than 1% difference) so technically not even a notable regression per se.

[Ordinary205]

I'd say the performance on Infamous 2 is almost fixed.

[Ordinary205]

It just needs a little improvement, and then it gets fixed.

[Asinin3]

DeS isn't a good test, the framerate counter doesn't work properly in that game due to how it works.
Your GoW 3 and Infamous test are showing basically the same avg fps despite major fluctuations and the usual run-to-run variance. The regression to performance looks like it was fixed by 14855 due to neko properly reverting a upstream LLVM commit that we identified as lowering performance (he reverted it in the 16.0.1 PR)

Can't see any reason to leave this open.

[Nekotekina]

I think some issue still remains. I didn't revert upstream commit (it's not technically possible), I only did a workaround

[Asinin3]

Ok re-opened. But infamous fps difference on avg is 0.1 and GoW3 was 1.5 (but only after a random drop). Both games fluctuate a lot, i think RDR would be a better test or TLoU.. something that doesn't fluctuate a lot.

[Ordinary205]

Should I test Red dead redemption?

[Asinin3]

Yes

[Ordinary205]

So I tested Red Dead Redemption on 14845 and 14855.

14845:

14855:

Log:
14845: RPCS3.log.gz
14855: RPCS3.log.gz

[Asinin3]

Performance will need to be tested again in 14876. Expect performance to go down, because neko had to revert something to fix a regression

[Ordinary205]

I might have to test 14876 on The Last of Us aswell.

[Ordinary205]

Heres the results for Red Dead Redemption and The Last of Us.

Red Dead Redemption:
14845:

14876:

The Last of Us:
14845:

14876:

Red Dead Redemption:
14845: RPCS3.log.gz
14876: RPCS3.log.gz

The Last of Us:
14845: RPCS3.log.gz
14876: RPCS3.log.gz

[solarmystic]

Borrowing this Issue report to note that God of War 3 and God of War Ascension have also regressed in performance since #13670 got merged to master with 0.0.27-14876.

GOW3

14875 -129/95/70 FPS

14876 - 122/97/64 FPS

GOWA

14875 - 65/60/55 FPS

14876 - 61/55/51 FPS

[Ordinary205]

Good thing that I'm not the only one who is experiencing a performance loss.

[Asinin3]

Test with #13672 Windows build just finished a few seconds ago

[Ordinary205]

@Asinin3 Sorry if this is a dumb question, but how do I test this PR build?

[Blackbird88]

@Asinin3 Sorry if this is a dumb question, but how do I test this PR build?

Download the artifact here

[Ordinary205]

@Asinin3 Sorry if this is a dumb question, but how do I test this PR build?

Download the artifact here

Thanks

[Asinin3]

Don't link people to artifacts. That commit is already old, and we want people testing the latest commits on PRs.
To download a PR build just go to the PR: #13672 > go to checks tab at top or bottom of page, then go to Cirrus Windows > View More Details on Cirrus > Artifact > download the artifact for the emulator, not the SHA file.

[solarmystic]

Another update to this issue report - after #13672 got merged to master in v0.0.27-14881, performance drops have been fixed for GOW3 and GOW Ascension, they are back to within margin of error compared to 14845.

The performance regression in Persona 5 that exists since LLVM16 got merged to master still remains.

14845 (last pre-LLVM16 master) - 97/72/63 FPS

14881 (current master with #13672) - 86/63/56 FPS

[Ordinary205]

Sadly it still has performance regression on Persona 5, even after 14898 was released.

[cipherxof]

MGS4 is also suffering from performance loss after the LLVM 16 upgrade.

Pre LLVM 16

LLVM 16

The master build is about the same.

RPCS3-13534.log.gz
RPCS3-13613.log.gz

[cipherxof]

It turns out that at least for Linux performance is restored when using the rocketlake LLVM profile instead of icelake-client. I'll have to retest this on Windows when I get a chance.

RPCS3.log.gz

[cipherxof]

This is still an issue on Windows for my machine. Initially I thought it was related to AVX-512, so I decided to test without it and alas performance is still better on LLVM 15.

It's worth noting that I had someone with a 5800x test the same scene/settings and there was no regression.

[cipherxof]

It turns out the performance loss isn't due to LLVM 16 at all, but one specific change that was introduced in that same PR. The following change restores performance back to normal (+10% gains).

RPCS3-tweaked.log.gz

My local build used in the log also reverted another PR which resulted in further performance loss (#14386). With both of these changes performance is now better than it was 14845.