Metal backend layer for Qt Window

[jbltx]

Hello,

Maybe my question will be out of scope, but i'm trying to integrate MoltenVK to my vulkan application which is currently using Qt to manage windows. The issue comes from the surface creation :

[***MoltenVK ERROR***] VK_ERROR_INITIALIZATION_FAILED: On-screen rendering requires a view that is backed by a layer of type CAMetalLayer.

As far as I understand, my window - which inherits directly from QWindow, and is recognized by the surface create info structure using the QWindow::WId member - seems to not have the requirements to get Metal working on it.

I was looking for some answers about the same issue on the web, but surprisingly i didn't find anything interesting. So i try here.

Did anyone have a chance to get MoltenVK and Qt working in the same application on OSX ?

Thank you.

[zeux]

Yeah - you need to change the layer on the NSView to be a CAMetalLayer. It might be worthwhile to make MoltenVK do this automatically (it's only possible to do on macOS, not iOS). Just run this code:

static void makeViewMetalCompatible(void* handle)
{
    NSView* view = (NSView*)handle;
    assert([view isKindOfClass:[NSView class]]);

    if (![view.layer isKindOfClass:[CAMetalLayer class]])
    {
        [view setLayer:[CAMetalLayer layer]];
        [view setWantsLayer:YES];
    }
}

(note: you need to compile this in an ObjC translation unit; not sure if it's possible to do the same from just C/C++)

[jbltx]

@zeux your method works like a charm, thank you !

I just made a small .mm wrapper file for the surface creation for the moment, it will surely grow later...

I'm a total newbie with ObjC so it's really helpful for me, and I hope it will be also helpful for other people using Qt !

Thank you again.

[mellinoe]

It might be worthwhile to make MoltenVK do this automatically

I'd also like to see that. It can also be done on an iOS UIView by adding the CAMetalLayer as a new sublayer. This is what I'm doing in my code for Metal, at least.

[billhollings]

@zeux & @mellinoe

Hmmm...interesting suggestion...and thanks for the solutions.

But I'm not so sure that MoltenVK...as a graphics driver...should be taking it upon itself to reach into an app and change the app's windowing composition. That seems a bit like a recipe for confusion and complaints from (especially newbie) app devs who are otherwise expecting to be able to do normal app stuff to that window or view.

For instance...what if Qt (or equivalent) has a requirement that it NOT be a CAMetalLayer...maybe its expecting to run OpenGL on that view? It seems to be me better to let the app handle arbitration in those circumstances.

Perhaps as a convenience...the MoltenVK API could include your solutions into some kind of helper function (eg- mvkMakeMeCompatible(NS/UIView* view))?

[zeux]

@billhollings yeah - I'm not sure how I feel about automatically doing this, especially for iOS (not sure what the implications are about adding another layer). Adding a mvk function would be great though - as it stands you basically need to write Objective C code to use MoltenVK since these APIs aren't accessible from C AFAICT.

[mellinoe]

as it stands you basically need to write Objective C code to use MoltenVK since these APIs aren't accessible from C AFAICT.

They are accessible, you just need to be familiar with the Objective-C Runtime and message passing. I'm doing what was described above from C#/.NET, through its C interop capabilities.

[billhollings]

@mellinoe

Similar to @zeux's macOS example...would you mind submitting a code snippet that implements the sublayer approach on iOS...for a future implementation of such a convenience function, please?

Or if anyone wants to take on implementing that function for fun...and submit a PR...please feel free! ;^)

[mellinoe]

@billhollings Sure, here's what I do in my library. It should be understandable although it is in C#:

CAMetalLayer = CAMetalLayer.New(); // Calls alloc + init
UIView uiView = ...; // From swapchain create info

CGSize viewSize = uiView.frame.size;
metalLayer.frame = uiView.frame;
metalLayer.opaque = true;
uiView.layer.addSublayer(metalLayer.NativePtr); // This is just your CAMetalLayer* in C/C++
metalLayer.device = device; // My MTLDevice instance
metalLayer.pixelFormat = MTLPixelFormat.BGRA8Unorm; // Aka MTLPixelFormatBGRA8Unorm
metalLayer.framebufferOnly = true;
metalLayer.drawableSize = viewSize;

Some of the above is probably already done by MoltenVK (like setting the device, etc.).

[oscarbg]

Hi guys thanks to this thread I have been able to add support for surface info to a simple port I had done last week of VulkanCapsViewer to MacOS and IOS..
you can see the diff here:
SaschaWillems/VulkanCapsViewer#45
I unified @zeux and @mellinoe solutions in one function

#ifdef BUILD_FOR_MAC
#import <Cocoa/Cocoa.h>
#else
#import <UIKit/UIKit.h>
#import <Metal/Metal.h>
#endif

extern "C" void makeViewMetalCompatible(void* handle)
{
#ifdef BUILD_FOR_MAC
    NSView* view = (NSView*)handle;
    assert([view isKindOfClass:[NSView class]]);

    if (![view.layer isKindOfClass:[CAMetalLayer class]])
    {
        orilayer=[view layer];
        [view setLayer:[CAMetalLayer layer]];
        //[view setWantsLayer:NO];
    }
#else
    UIView* view = (UIView*)handle;
    assert([view isKindOfClass:[UIView class]]);

    CAMetalLayer *metalLayer = [CAMetalLayer new]; // Calls alloc + init
    //UIView uiView = ...; // From swapchain create info

    CGSize viewSize = view.frame.size;
    metalLayer.frame = view.frame;
   metalLayer.opaque = true;
    metalLayer.framebufferOnly = true;
    metalLayer.drawableSize = viewSize;
    metalLayer.pixelFormat = (MTLPixelFormat)80;//BGRA8Unorm==80
    [view.layer addSublayer:metalLayer];
#endif
}

notes:
as VulkanCapsviewer is special Qt+Vulkan app as calls VkCreateSurface for getting surface info but also doesn't up rendering to it.. so I need after getting the info to call some other function (unmakeViewMetalCompatible below) to restore the original layer (that I saved previously as oriLayer) so I can see the app render correctly..

void unmakeViewMetalCompatible(void* handle)
{
#ifdef BUILD_FOR_MAC
    NSView* view = (NSView*)handle;
    assert([view isKindOfClass:[NSView class]]);
#else
  UIView* view = (UIView*)handle;
    assert([view isKindOfClass:[UIView class]]);
#endif

    if ([view.layer isKindOfClass:[CAMetalLayer class]])
    {
        [view setLayer:orilayer];
    }
}

it seems to work on MacOS without any strange messages with debugging enabled so nice..

code with ifdefs compiles also on IOS and in the form I share doesn't crash on it (as it was if using the original approach shader for MacOS)..
on IOS still MoltenVK complains that the view should be CAMetalLayer..
on IOS, with the sublayer approach, original view still seems to be EAGLLayer but seems the addSublayer should have added the Metal sublayer as shared here..
so seems now the ball is on MoltenVK to support that scenario, right @billhollings?
then will fix the unmakeViewMetalCompatible call on IOS which is not ready or perhaps it's already it is if I save the original layer on IOS case and restore later..
Also note I know on MacOS there is a leak on code as restoring original layer I should delete the CAMetalLayer I created, right?

thanks..

[mellinoe]

I should delete the CAMetalLayer I created, right?

I'm not really an Objective-C expert, but yes, you need to release the object if you created it yourself (with the "new" function).

[ikryukov]

Made simple solution for Qt + Vulkan on OSX using MoltenVK with qmake. Work in progress now, but I think it could be useful for some one: https://github.com/ikryukov/QtVulkan

[msorvig]

We've now also added experimental support in Qt itself, see http://blog.qt.io/blog/2018/05/30/vulkan-for-qt-on-macos .

In Qt 5.12, setting the surface type of the window to QSurface::VulkanSurface will configure the QWindow/NSView to backed by a CAMetalLayer layer which should be usable by MoltenVK.

[billhollings]

@msorvig

Wonderful news! Thanks very much for letting us know!

And with that news...I am going to close this issue here!