[Bug] [macOS] [Arm] LLVM not working

[shinra-electric]

Quick summary

LLVM is not working on macOS Arm builds.

Details

Yesterday PR #15992 was merged to master, and should enable LLVM for Arm builds on macOS.

I'm aware that not all games will work, but I'm unable to get any games to work.

There are no obvious errors in the log, it will just stall at the end of compiling the SPU cache.

The app then can't be quit, and trying to stop emulation will produce error messages saying the main thread is not responsive ("too sleepy"). The app must be force-quit to close at this point.

Edit: Forgot to mention I'm on the latest macOS 15 beta.

Attach a log file

This is a sample log from Arkedo Jump.

- SYS: RPCS3 v0.0.32-16873-e5b03d9c Alpha | master | local_build
- SYS: Apple M1 Pro (6P+2E) | 8 Threads | 16.00 GiB RAM | TSC: 0.024GHz | FMA3
- SYS: Operating system: macOS, Version: 15.0.0
- SYS: Current Time: 2024-08-26T08:55:14
- RSX: Found Vulkan-compatible GPU: 'Apple M1 Pro' running on driver 0.2.994
- SYS: Firmware version: 4.91
S SPU: Found 85 SPU function(s)
S PPU: Enabled: Non-Java Mode Fixup
- RSX: Found Vulkan-compatible GPU: 'Apple M1 Pro' running on driver 0.2.994
E RSX: Your GPU driver does not support some required MSAA features. MSAA will be disabled.
E RSX: Your GPU does not support double precision floats in shaders. Graphics may not render correctly.
E RSX: Your GPU does not support depth bounds testing. Graphics may not render correctly.
E RSX: Your GPU does not support wide lines. Graphics may not render correctly.
E RSX: Your GPU does not support framebuffer logical operations. Graphics may not render correctly.
E RSX: Apple GPUs are incompatible with the current implementation of asynchronous texture decoding.
S Input: The pad mode was changed to Single-threaded
S GUI: Boot successful.
S PPU: LLVM: Loaded module v6-kusa-XEKk2WaT9LSoTSsoaSihLr-00001G-apple-m1.obj
S PPU: LLVM: Loaded module v6-kusa-cn4reYFFUFr7WVVZAxiSwx-00001G-apple-m1.obj
S PPU: LLVM: Loaded module v6-kusa-fKHez1R3Ew3mPerMLzmeCk-00001G-apple-m1.obj
S PPU: LLVM: Loaded module v6-kusa-cq8VqWmTkrnNQ6juK1FeSr-0004Xj-apple-m1.obj
S PPU: LLVM: Loaded module v6-kusa-UAxtfWCozPHiv30yoG2Ucd-0004Xj-apple-m1.obj
S SPU: SPU Runtime: Built the interpreter.
S SPU: STQR/LQR Atomic Loop Hack: abs_pos=0x18e0, abs=0x2b10, i16*4=0x1848, ls_bound=0x17f
U SPU TODO: [__spu-0x0d950-PU182Mn8wZrev6ZNu8ca823smTgy:0x0d9d0] Unmatched spu_re(b) found in FMA
U SPU TODO: [__spu-0x0a0b8-f1SdpoPyhLo24rCmqYC36ukzpuLH:0x0a264] Unmatched spu_re(b) found in FMA
U SPU TODO: [__spu-0x09a38-m56P7wJX2Yp3qMKybocsyba5QKQ1:0x09c5c] Unmatched spu_re(b) found in FMA
U SPU TODO: [__spu-0x05880-0m63s00WnqvrKv8mQz60i74jReC1:0x05a60] Unmatched spu_re(b) found in FMA
U SPU TODO: [__spu-0x05880-0m63s00WnqvrKv8mQz60i74jReC1:0x05a64] Unmatched spu_re(b) found in FMA
U SPU TODO: [__spu-0x05880-0m63s00WnqvrKv8mQz60i74jReC1:0x05a68] Unmatched spu_re(b) found in FMA
U SPU TODO: [__spu-0x05880-0m63s00WnqvrKv8mQz60i74jReC1:0x05a6c] Unmatched spu_re(b) found in FMA
U SPU TODO: [__spu-0x05880-0m63s00WnqvrKv8mQz60i74jReC1:0x05a70] Unmatched spu_re(b) found in FMA
U SPU TODO: [__spu-0x05880-0m63s00WnqvrKv8mQz60i74jReC1:0x05a74] Unmatched spu_re(b) found in FMA
U SPU TODO: [__spu-0x05880-0m63s00WnqvrKv8mQz60i74jReC1:0x05a78] Unmatched spu_re(b) found in FMA
U SPU TODO: [__spu-0x05880-0m63s00WnqvrKv8mQz60i74jReC1:0x05a80] Unmatched spu_re(b) found in FMA
U SPU TODO: [__spu-0x09ca0-9mjT7cXqCSvhE0SL3zXQnwN5pGWd:0x09d04] Unmatched spu_re(b) found in FMA
U SPU TODO: [__spu-0x04938-hZ0jqR40gNwtYYLymm0K2geFuopv:0x049dc] Unmatched spu_re(b) found in FMA
U SPU TODO: [__spu-0x05e58-Fp6cgy1krUdpK7pWEMWZAZ7XeUyF:0x05ec8] Unmatched spu_re(b) found in FMA
U SPU TODO: [__spu-0x043a0-USjTxdvuK8ZpWpGA8dbUpN11YSsL:0x043dc] Unmatched spu_re(b) found in FMA
U SPU TODO: [__spu-0x043a0-USjTxdvuK8ZpWpGA8dbUpN11YSsL:0x043f0] Unmatched spu_re(b) found in FMA
U SPU TODO: [__spu-0x04368-az8sN0g8RVpFRhFpJaZMRv5Kiuk0:0x04390] Unmatched spu_re(b) found in FMA
U SPU TODO: [__spu-0x042e8-nE2dTX9n0AaToTzLvu8GWvVTrzKQ:0x04340] Unmatched spu_re(b) found in FMA
S SPU: SPU Runtime: Built 285 functions.
U sys_prx TODO: _sys_prx_register_module(name=“cellProcessElf”, opt=*0xd0040c90)
S PPU: LLVM: Loaded module v6-kusa-7LHmPKQURuTbHwAhSpWFH6-0004Xj-apple-m1.obj
S sys_prx: Loaded module: “/dev_flash/sys/external/libsysmodule.sprx” (id=0x23000b00)
E SIG: Thread [cellAudio Thread] is too sleepy. Waiting for it 21412.542µs already!
E SIG: Thread [PPU[0x1000000] main_thread] is too sleepy. Waiting for it 21289.083µs already!
E SIG: Thread [PPU[0x1000000] main_thread] is too sleepy. Waiting for it 41242.792µs already!
E SIG: Thread [PPU[0x1000000] main_thread] is too sleepy. Waiting for it 80582.500µs already!
E SIG: Thread [PPU[0x1000000] main_thread] is too sleepy. Waiting for it 161039.792µs already!
E SIG: Thread [PPU[0x1000000] main_thread] is too sleepy. Waiting for it 321316.458µs already!
E SIG: Thread [PPU[0x1000000] main_thread] is too sleepy. Waiting for it 642892.125µs already!
E SIG: Thread [PPU[0x1000000] main_thread] is too sleepy. Waiting for it 1287043.333µs already!
E SIG: Thread [PPU[0x1000000] main_thread] is too sleepy. Waiting for it 2574110.375µs already!
E SIG: Thread [PPU[0x1000000] main_thread] is too sleepy. Waiting for it 5147888.458µs already!
E SIG: Thread [PPU[0x1000000] main_thread] is too sleepy. Waiting for it 10295964.292µs already!
E SIG: Thread [PPU[0x1000000] main_thread] is too sleepy. Waiting for it 20593209.667µs already!
E SIG: Thread [PPU[0x1000000] main_thread] is too sleepy. Waiting for it 41185059.250µs already!

Attach capture files for visual issues

No response

System configuration

No response

Other details

No response

[shinra-electric]

Where did those spambots come from?

I'm just thinking I might be using a version of LLVM that's too new (18). I will test using prior versions.

Edit: Using llvm@16 does not make any difference, so that's probably not the issue.

[kd-11]

@shinra-electric Please share your build config.
Also start with simple games and binaries after nuking your cache completely. Remove ~/Library/caches/rpcs3 completely just to be safe.

[shinra-electric]

Remove ~/Library/caches/rpcs3 completely just to be safe.

This seems to have fixed the issue.
Deleting all caches using the context menu in the UI didn't help before, but deleting the whole folder did the trick.

[shinra-electric]

Just to add to this, I have LLVM working on one Mac, but can't get it working on another with similar specs.

I think it may be a threading/deadlock issue when compiling the SPU cache (PPU seems to be fine), but this is just speculation.

Edit: I'll often get this error message:

F {SPU Worker 6} SIG: Thread terminated due to fatal error: Verification failed (object: 0x0)
(in file /Applications/Games/Emulators/Playstation 3/rpcs3/rpcs3/util/vm_native.cpp:323[:3], in function void utils::memory_commit(void *, usz, protection)) (errno=13=Permission denied)

[kd-11]

Just to add to this, I have LLVM working on one Mac, but can't get it working on another with similar specs.

I think it may be a threading/deadlock issue when compiling the SPU cache (PPU seems to be fine), but this is just speculation.

Edit: I'll often get this error message:

F {SPU Worker 6} SIG: Thread terminated due to fatal error: Verification failed (object: 0x0)
(in file /Applications/Games/Emulators/Playstation 3/rpcs3/rpcs3/util/vm_native.cpp:323[:3], in function void utils::memory_commit(void *, usz, protection)) (errno=13=Permission denied)

We're aware of the error and the root cause is understood. It will be fixed soon. It happens sometimes when a large number of blocks is being compiled in parallel. As a workaround, set your "LLVM compiler threads" to something low once the cache has been built the first time. 1 thread will always work, but may be too slow, 2-4 should be ok.

[meh7an]

I have tried clearing the whole cache and then trying with only 1 max LLVM thread, but still an almost same error occurs:

F {PPU[0x1000000] main_thread} SIG: Thread terminated due to fatal error: Verification failed (object: 0x0)
(in file /Users/mehran/Documents/Projects/rpcs3/RPCS3-Arm-Build-Script/rpcs3/rpcs3/util/types.hpp:942[:102], in function decltype(auto) ensure(T &&, const_str, std::source_location) [T = bool]) (errno=2=No such file or directory)

I did the compilation using @shinra-electric's script.

[kd-11]

Please open a ticket for that. It is still not clear why only some people are affected by this while we have a workaround in the code already.

[meh7an]

Opened the ticket at #16002