My findings trying to run simple triangle homebrew app on MacOS using MoltenVK..

[oscarbg]

Hi @kd-11,
I see you opened two bug reports for current Vulkan MacOS limitations.. i.e. to SPIRV-Cross and MoltenVK GitHub sites and seems at least SPIRV-Cross issue has been resolved (KhronosGroup/SPIRV-Cross@4979d10) have you tested updated MoltenVK with the submodule to be up to this commit already? seems MoltenVK updated SPIRV-Cross a day ago upto KhronosGroup/SPIRV-Cross@0f9cb0d which is not enough..

Anyway I want to report my findings..
I build on my own from master (5f04703) and with LLVM disabled to start with simple build..
I use QT 5.10.x from official Qt installer and point it similar to:
export Qt5_DIR=/Volumes/s/Qt/5.10.1/clang_64
which seems OK..
my flags are:
cmake ../rpcs3/ -DUSE_VULKAN=ON -DUSE_NATIVE_INSTRUCTIONS=ON -DWITHOUT_LLVM=ON -DCMAKE_BUILD_TYPE=Release
strangely to build OK from latest master needed:
1)comment:

//LOG_UNIMPLEMENTED(RSX, "Swap control not implemented for this platform. Vsync options not available.");

in OpenGL.cpp if not get it complained about missing..

error: use of undeclared identifier 'RSX'
LOG_UNIMPLEMENTED(RSX, "Swap control not implemented for this platform. Vsync options not available.");

had to add also "/opt/X11/lib/libX11.dylib" to rpcs3/Cmakelists.txt to finally have:

if(APPLE)
target_link_libraries(rpcs3 hidapi-mac "-framework CoreFoundation" "-framework IOKit" "/opt/X11/lib/libX11.dylib")

if not I get:

Undefined symbols for architecture x86_64:
"_XCloseDisplay", referenced from:
VKGSRender::~VKGSRender() in VKGSRender.cpp.o
"_XCreateImage", referenced from:
vk::swapchain_X11::init() in VKGSRender.cpp.o
"_XFlush", referenced from:
VKGSRender::VKGSRender() in VKGSRender.cpp.o
vk::swapchain_X11::present(unsigned int) in VKGSRender.cpp.o
"_XGetGeometry", referenced from:
vk::swapchain_X11::init() in VKGSRender.cpp.o
"_XMatchVisualInfo", referenced from:
vk::swapchain_X11::init() in VKGSRender.cpp.o
"_XOpenDisplay", referenced from:
gs_frame::handle() const in gs_frame.cpp.o
non-virtual thunk to gs_frame::handle() const in gs_frame.cpp.o
"_XPutImage", referenced from:
vk::swapchain_X11::present(unsigned int) in VKGSRender.cpp.o
ld: symbol(s) not found for architecture x86_64

I had already vulkan on /Library/Frameworks/vulkan.framework/ so seems build worked ok:
otool -L rpcs3 shows correctly:
@rpath/vulkan.framework/Versions/A/vulkan (compatibility version 1.0.0, current version 1.0.69)
then program launches OK I can see Vulkan option and selects Vega 56 so seems using MoltenVK correctly..
Also installing firmware is OK..
anyway even using simple test elf like from TEST12345/gs_gcm_tetris.elf or gs_gcm_basic_triangle.elf crashes with:

X Error of failed request: BadDrawable (invalid Pixmap or Window parameter)
Major opcode of failed request: 14 (X_GetGeometry)
Resource id in failed request: 0x16c49270
Serial number of failed request: 7
Current serial number in output stream: 7

this error is not Vulkan related and LLVM should not be needed for running it and asmjit should be enough for that sample, right?

seeing this are you using MoltenVK Vulkan Qt integration correctly on MacOS X? I mean please see this thread:
"Metal backend layer for Qt Window"
KhronosGroup/MoltenVK#78
you need something like:

static void makeViewMetalCompatible(void* handle)
{
    NSView* view = (NSView*)handle;
    assert([view isKindOfClass:[NSView class]]);

    if (![view.layer isKindOfClass:[CAMetalLayer class]])
    {
        [view setLayer:[CAMetalLayer layer]];
        [view setWantsLayer:YES];
    }
}

finally also are the swizzling MoltenVK limitation and pack2half SPIRV-Cross limitation needed for running such simple homebrew demos?

finally assuming someone does this swizzling work on MoltenVK do you need some more Vulkan things not supported by MoltenVK, I mean extensions or optional caps?

thanks..

[hcorion]

Yes, issues are known. You can update SPIRV-Cross in moltenVK by running External/getLatestSPIRVCross, which gets around the unpackHalf2x16 issue. As it doesn't seem like swizzle in MoltenVK will get fixed anytime soon, I see no reason to keep this issue open.

I'll be monitoring MoltenVK, and if changes are implemented that fix rpcs3, I will restest and if it makes RPCS3 work on MacOS, I will make a PR.

Please close this issue.

[kd-11]

There only exists 2 real problems right now that stop even basics from working (Vulkan)

1. X11 integration on mac isn't doing too great, XGetGeometry fails with the X11 window handle passed back from the Qt widget. Don't worry about the metal window integration stuff, its meaningless for our use case right now since it doesnt even get to swap/present without crashing from other issues.

2. The moltenVK limitations. These seem to stem from problems in metal as an API by itself, probably to support mobile devices. I have no idea how problematic these will be but they seem to be rather severe and require ridiculous workarounds, so we just have to be patient.

@hcorion Has access to a mac device and I dont, so he'll be able to tell us more as moltenvk development continues. Opengl also does not work because apple only supports openGL 4.1 core with no extensions for whatever reason.

[oscarbg]

Thanks @hcorion and @kd-11 for comments..
Closing it now and dreaming (altough I will not hold my breath) that WWDC Metal news brings some fixes for this project.. just and idea, perhaps will be nice to ask in Apple Metal dev forums the current swizzle limitation that will help eventually MoltenVK to support Rpcs3..