WIP: MoltenVK/Vulkan backend for macOS #7482

Akemi · 2020-02-23T14:07:38Z

this is in a very early state. i still need to majorly clean this code up and kill all the code duplications. my goal is to have one code base to use in all the macOS backends (might be a bit tricky with libmpv/cocoa-cb). nothing really has been tested so far. it renders the video and the basic windowing stuff works.

TODO:

though feel free to report any kind of problems you encounter.

simple and dirty way to install the dependencies:

downloaded vulkan sdk from https://vulkan.lunarg.com/sdk/home#mac
installed it with the provided install script python install_vulkan.py
created a vulkan.pc pkg config file with:

#!/bin/bash

export DEPS_ROOT_LOCATON=$(grealpath .)
export VULKAN_SDK_VERSION="1.2.131.2"
export VULKAN_SDK_BASENAME="vulkansdk-macos-$VULKAN_SDK_VERSION"
export VULKAN_SDK="$DEPS_ROOT_LOCATON/$VULKAN_SDK_BASENAME/macOS"
export VULKAN_SDK_PKG_CONFIG_PATH="$VULKAN_SDK/lib/pkgconfig"
export PATH="/usr/local/opt/python/libexec/bin:$VULKAN_SDK/bin:$PATH"

cat > "$VULKAN_SDK_PKG_CONFIG_PATH/vulkan.pc" << EOL
libdir=$VULKAN_SDK/lib
includedir=$VULKAN_SDK/include

Name: vulkan
Description: Vulkan
Version: $VULKAN_SDK_VERSION
Libs: -L\${libdir} -lMoltenVK
Libs.private: -lm
Cflags: -I\${includedir}
EOL

symlinked the vulkan.pc file /usr/local/lib/pkgconfig
ln -s vulkansdk-macos-1.2.131.2/macOS/lib/pkgconfig/vulkan.pc /usr/local/lib/pkgconfig/vulkan.pc
install libplacebo

fixes #5571

Akemi · 2020-02-25T18:21:47Z

i am not recommending anything. even said it's a dirty way. i didn't look into building/linking properly yet.

orion1vi · 2020-05-05T03:29:55Z

video/out/vulkan/context_mac.m

+        .sType = VK_STRUCTURE_TYPE_MACOS_SURFACE_CREATE_INFO_MVK,
+        .pNext = NULL,
+        .flags = 0,
+        .pView = p->vo_macos.video,


segfaulted, until I have changed it to p->vo_macos.video.layer

that's kinda interesting since the docs say something else https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkMacOSSurfaceCreateInfoMVK.html

on the other hand the moltenVK docs say something else and the layer is recommended KhronosGroup/MoltenVK@e546b83

https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkMetalSurfaceCreateInfoEXT.html
this one should be used with the layer directly.

VK_MVK_macos_surface is called obsolete here KhronosGroup/Vulkan-Tools#220, so maybe it's better to just use VK_EXT_metal_surface

interesting. i was planning to use the Metal Layer directly at some point anyway (i looked into metal surface ext beforehand), to reduce some redundancy and make it similar to cocoa-cb with its layer.

no, i don't think i have enough time for that.

Akemi · 2020-07-17T15:40:27Z

a little update. i finished unifying the two backends to get rid of a majority of all the code duplications. the cocoa-cb generalisation is completely done and properly cleaned up, and could possibly be merged independently. though there are no real benefits besides me not having to backport possible fixes from master. the molten backend still has to be cleaned up properly.

Akemi · 2020-07-26T10:10:39Z

some findings:

with interpolation i can reliable reproduce rendering freezes, this can be mitigated by --vulkan-disable-events
initialisation takes quite a lot of time, sometimes more than a second, this can be mitigated by --gpu-shader-cache-dir
setting an icc profile on startup doesn't work properly and needs to be set after the first draw (still needs a proper fix)
vsync jitter is still a bit erratical and needs more testing
live reisze is not as smooth as on cocoa-cb, rendering surface is not fixed in size and updates constantly
drawing in background doesn't work and blocks, like with every layer, needs a mechanism to actively skip
drawing size update still needs testing

for myself building libplacebo so i don't forget it:

meson configure --clearcache $DIR
meson --reconfigure $DIR -Dvulkan-registry=/Users/Akemi/Repositories/vulkansdk-macos-1.2.141.2/macOS/share/vulkan/registry/vk.xml

Akemi · 2020-08-01T16:52:23Z

some more findings:

VK_PRESENT_MODE_IMMEDIATE_KHR sets displaysyncenabled which significantly degrades performance. seems like it can't properly sync the the display and drops frames because of that
using VK_PRESENT_MODE_FIFO_KHR instead and doing a manual sync to the display works a lot better and it seems that performance is at least similar to opengl (if not better, still need to test)
how moltenvk initialises the metal layer https://github.com/KhronosGroup/MoltenVK/blob/f967fb6aebcce02d02ad9bc249f45cb5cf4a11f6/MoltenVK/MoltenVK/GPUObjects/MVKSwapchain.mm#L269-L275
from a quick test, compute shaders seem to work as intended
layer scaling is set to kCAFilterNearest by default and should be set to at least be kCAFilterLinear (moltenvk can only set those too)
using the precise timer test (i did once), could be used here

stats with the follwoing:
VK_PRESENT_MODE_IMMEDIATE_KHR: https://0x0.st/ixIc.png
VK_PRESENT_MODE_FIFO_KHR: https://0x0.st/ixIA.png

Akemi · 2020-08-01T19:06:23Z

includes for the molten stuff needs to be manually installed?

ln -s /Users/Akemi/Repositories/vulkansdk-macos-1.2.141.2/MoltenVK/include/MoltenVK /usr/local/include/MoltenVK
ln -s /Users/Akemi/Repositories/vulkansdk-macos-1.2.141.2/MoltenVK/include/vulkan-portability /usr/local/include/vulkan-portability

https://github.com/haasn/libplacebo/blob/5e517936c15583e3148f46d76135d63537617f9f/src/vulkan/swapchain.c#L164-L172 check why it picks an 8bit format even so 10/16bit formats are available.

Akemi · 2020-08-07T14:00:31Z

some more info:

VK_PRESENT_MODE_IMMEDIATE_KHR causes major tearing and can't be used, VK_PRESENT_MODE_FIFO_KHR needs to be used but this causes bad vsync

some more info from recent testing with moltenvk 1.2.148.0 i updated to:

performance with VK_PRESENT_MODE_FIFO_KHR got a lot better, still needs proper testing
~~after updating libplacebo the bad performance was back, caused by commit haasn/libplacebo@055cdc0~~ can't reproduce this anymore, maybe a hiccup

Akemi · 2020-08-08T11:49:43Z

did some performance tests today. i couldn't find any sample or settings that would indicate any kind of performance difference, neither better nor worse, that weren't bottlenecked by decoding speed.

only thing i noticed is that vsyncing on moltenvk is a tad more unstable so far.

github-actions · 2023-09-30T14:58:28Z

Download the artifacts for this pull request:

Windows

m154k1 · 2023-09-30T17:06:40Z

Nice, thanks! I've did a quick test, looks very promising :)

github-actions · 2023-10-01T23:13:36Z

Download the artifacts for this pull request:

Windows

Akemi · 2023-10-01T23:34:07Z

i fixed/adjusted the precise timer. it's there for testing purpose mostly. when i started with this +2 years ago, moltenvk had some problems where the precise timing worked better than directly using the displaylink callback. it should be tested further, hence the new option to switch between 3 different modes.

in the long run the precise timer should be removed and the proper presentation feedback should be used. this won't be done in the PR most likely.

github-actions · 2023-10-02T00:01:58Z

Download the artifacts for this pull request:

Windows

m154k1 · 2023-10-02T16:13:42Z

Great. All 3 modes are working good. I'm not sure how to properly test this but playback, seeking, --video-sync=audio/display-resample, etc. are working as expected. Tested on built-in display (60Hz) and external display (75Hz) with random content.

The only difference I can find is vsync jitter.

--macos-frame-timer=callback/system has 0.001-0.040 jitter. Usually it is ~0.020.
--macos-frame-timer=precise has very low and stable 0.001 jitter 99% of the time.

But anyway, I can't tell any difference without stats menu, so I'm fine with whatever mode.

Akemi · 2023-10-02T16:38:57Z

if it doesn't make any difference anymore, that's good.

i found one issue with the precise timer. toggling fullscreen several times throws off the vsync in windowed mode on my end. everything fine in fullscreen.

m154k1 · 2023-10-02T16:52:00Z

i found one issue with the precise timer. toggling fullscreen several times throws off the vsync in windowed mode on my end. everything fine in fullscreen.

Hmm, works fine for me...

Akemi · 2023-10-07T19:26:24Z

i am done with this now and i am comfortable with it being merged.

mpv --vulkan-device=help seems broken now
the bundle creation should be fixed in a separate PR since it's generally broken in some cases
render timing should be changed to use the presentation feedback in separate PR

also i am not quite sure about the commit message prefix.

meson.build

add support for vulkan through metal and a translation layer like MoltenVK. also add the possibility to use different render timing modes for testing. i still consider this experimental atm.

github-actions · 2023-10-07T23:59:57Z

Download the artifacts for this pull request:

Windows

Akemi · 2023-10-14T16:40:41Z

have fun and happy testing.

guyman624 · 2023-10-30T00:22:59Z

would this work with libmpv and ios as well? or would more work need to be done first?

Akemi · 2023-10-30T00:37:18Z

libmpv only has opengl support, if you run your own render loop. wid embedding ist not supported. so it won't work.

also see my previous comment #7482 (comment)

rcombs · 2023-10-30T01:00:19Z

wid embedding does work with libmpv, and adding it here likely wouldn't be a huge undertaking. Not sure if there's something else that'd be preferred in the long run.

Akemi · 2023-10-30T06:52:47Z

nothing else was claimed. it does not work currently with the code related to this PR. like you said yourself. the feature is not implemented.

aehlke · 2023-10-30T19:43:39Z

would this work with libmpv and ios as well? or would more work need to be done first?

someone's ported it to ios via flurry which you can embed as a (heavy?) wrapped component inside a non-flurry ios project

Akemi mentioned this pull request Feb 23, 2020

leverage Vulkan as default on MacOS and iOS now that MoltenVK is open-sourced #5571

Closed

Akemi force-pushed the mac_vulkan branch from 3dec5c9 to d44481e Compare May 2, 2020 11:54

orion1vi reviewed May 5, 2020

View reviewed changes

Akemi force-pushed the mac_vulkan branch from d44481e to e7f7ac0 Compare June 6, 2020 13:42

tmm1 mentioned this pull request Jun 24, 2020

video/out/vulkan: add MoltenVK context #7857

Closed

Akemi force-pushed the mac_vulkan branch from 1a17659 to 2b6706a Compare June 27, 2020 09:15

Akemi force-pushed the mac_vulkan branch from 2b6706a to f278b8e Compare July 12, 2020 15:04

Akemi force-pushed the mac_vulkan branch 5 times, most recently from 2c3a49c to 7a3e19f Compare July 18, 2020 15:11

Akemi force-pushed the mac_vulkan branch 5 times, most recently from 5243559 to a9e376e Compare July 26, 2020 09:53

Akemi force-pushed the mac_vulkan branch from d1a15ee to d61d469 Compare July 26, 2020 11:50

Akemi mentioned this pull request Jul 27, 2020

9 to 10 frames dropped every 56 seconds iina/iina#1926

Open

1 task

Akemi force-pushed the mac_vulkan branch from d61d469 to e496dc9 Compare July 31, 2020 11:25

Akemi force-pushed the mac_vulkan branch 3 times, most recently from 20af3e4 to c6bb856 Compare August 2, 2020 12:38

Akemi mentioned this pull request Oct 7, 2023

Changing --video-crop runtime causes temporary window jumping #12589

Closed

Akemi force-pushed the mac_vulkan branch from 300af2b to 902fe8a Compare October 7, 2023 18:35

Dudemanguy reviewed Oct 7, 2023

View reviewed changes

meson.build Show resolved Hide resolved

Dudemanguy reviewed Oct 7, 2023

View reviewed changes

meson.build Outdated Show resolved Hide resolved

Akemi force-pushed the mac_vulkan branch from 902fe8a to b6ce1d8 Compare October 7, 2023 21:42

vo_gpu/vo_gpu_next: add vulkan support for macOS

ffc37ae

add support for vulkan through metal and a translation layer like MoltenVK. also add the possibility to use different render timing modes for testing. i still consider this experimental atm.

Akemi force-pushed the mac_vulkan branch from b6ce1d8 to ffc37ae Compare October 7, 2023 21:47

Dudemanguy mentioned this pull request Oct 10, 2023

Duplicate frames and frames dropping with Option + s #12605

Closed

toooonyy mentioned this pull request Oct 12, 2023

Upstream mpv new Vulkan backend iina/iina#4647

Open

Akemi merged commit 78d4374 into mpv-player:master Oct 14, 2023

Akemi deleted the mac_vulkan branch October 14, 2023 16:40

Jules-A mentioned this pull request Oct 15, 2023

MPV display output performance significantly worse than Firefox on Youtube for same video #12638

Closed

rcombs mentioned this pull request Oct 30, 2023

hwdec: support videotoolbox with libplacebo #12776

Merged

Akemi mentioned this pull request Feb 18, 2024

With gpu-next / macvk switching from / to fullscreen sometimes glitches the video #13505

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: MoltenVK/Vulkan backend for macOS #7482

WIP: MoltenVK/Vulkan backend for macOS #7482

Akemi commented Feb 23, 2020 •

edited

Loading

Akemi commented Feb 25, 2020

orion1vi May 5, 2020

Akemi May 5, 2020

Akemi May 5, 2020

orion1vi May 5, 2020

Akemi May 5, 2020

Akemi May 5, 2020

Akemi commented Jul 17, 2020

Akemi commented Jul 26, 2020 •

edited

Loading

Akemi commented Aug 1, 2020

Akemi commented Aug 1, 2020 •

edited

Loading

Akemi commented Aug 7, 2020 •

edited

Loading

Akemi commented Aug 8, 2020

github-actions bot commented Sep 30, 2023

m154k1 commented Sep 30, 2023

github-actions bot commented Oct 1, 2023

Akemi commented Oct 1, 2023

github-actions bot commented Oct 2, 2023

m154k1 commented Oct 2, 2023

Akemi commented Oct 2, 2023

m154k1 commented Oct 2, 2023

Akemi commented Oct 7, 2023 •

edited

Loading

github-actions bot commented Oct 7, 2023

Akemi commented Oct 14, 2023

guyman624 commented Oct 30, 2023

Akemi commented Oct 30, 2023 •

edited

Loading

rcombs commented Oct 30, 2023

Akemi commented Oct 30, 2023

aehlke commented Oct 30, 2023

WIP: MoltenVK/Vulkan backend for macOS #7482

WIP: MoltenVK/Vulkan backend for macOS #7482

Conversation

Akemi commented Feb 23, 2020 • edited Loading

Akemi commented Feb 25, 2020

orion1vi May 5, 2020

Choose a reason for hiding this comment

Akemi May 5, 2020

Choose a reason for hiding this comment

Akemi May 5, 2020

Choose a reason for hiding this comment

orion1vi May 5, 2020

Choose a reason for hiding this comment

Akemi May 5, 2020

Choose a reason for hiding this comment

Akemi May 5, 2020

Choose a reason for hiding this comment

Akemi commented Jul 17, 2020

Akemi commented Jul 26, 2020 • edited Loading

Akemi commented Aug 1, 2020

Akemi commented Aug 1, 2020 • edited Loading

Akemi commented Aug 7, 2020 • edited Loading

Akemi commented Aug 8, 2020

github-actions bot commented Sep 30, 2023

m154k1 commented Sep 30, 2023

github-actions bot commented Oct 1, 2023

Akemi commented Oct 1, 2023

github-actions bot commented Oct 2, 2023

m154k1 commented Oct 2, 2023

Akemi commented Oct 2, 2023

m154k1 commented Oct 2, 2023

Akemi commented Oct 7, 2023 • edited Loading

github-actions bot commented Oct 7, 2023

Akemi commented Oct 14, 2023

guyman624 commented Oct 30, 2023

Akemi commented Oct 30, 2023 • edited Loading

rcombs commented Oct 30, 2023

Akemi commented Oct 30, 2023

aehlke commented Oct 30, 2023

Akemi commented Feb 23, 2020 •

edited

Loading

Akemi commented Jul 26, 2020 •

edited

Loading

Akemi commented Aug 1, 2020 •

edited

Loading

Akemi commented Aug 7, 2020 •

edited

Loading

Akemi commented Oct 7, 2023 •

edited

Loading

Akemi commented Oct 30, 2023 •

edited

Loading