Clarification on vkGetDeviceProcAddr behavior for non-device functions

[roderickc]

I would like some clarification on the spec for vkGetDeviceProcAddr. There are some ambiguities, which came up in a bug I originally filed for the loader (KhronosGroup/Vulkan-LoaderAndValidationLayers#2323).

During development of a Vulkan ICD for Wine, I encountered shipping games (Doom / Wolfenstein II), which utilize vkGetDeviceProcAddr to load ALL their function pointers including instance function pointers.

I was surprised the loader allowed this and after testing found it indeed works. In the loader related ticket we concluded that the spec of vkGetDeviceProcAddr isn't entirely clear. vkGetDeviceProcAddr is supposed to work for 'core Vulkan commands', it isn't clear whether that just mean device commands or all instance + device commands. Then there is the side note, which states that the returned function pointers should only be used by device or children of device. This area needs more clarification. What is the intended behaviour?

On the other hand as I demonstrated there are major shipping games relying on potentially unintended behaviour.

[critsec]

The spec says:

The overhead of this internal dispatch can be avoided by obtaining device-specific function pointers for any commands that use a device or device-child object as their dispatchable object. Such function pointers can be obtained with the command vkGetDeviceProcAddr

The footnote directly under Table 2 describing vkGetDeviceProcAddr also says, referring to all function pointers returned by vkGetDeviceProcAddr:

The returned function pointer must only be called with a dispatchable object (the first parameter) that is device or a child of device. e.g. VkDevice, VkQueue, or VkCommandBuffer

Thus even if you get a function pointer back for a command that dispatches on VkInstance or VkPhysicalDevice, you couldn't validly call through that pointer. The intent was that vkGetDeviceProcAddr only supports functions of VkDevice or its children, and that function pointers you get from it don't require any dispatch overhead: they jump directly to the code for the "outermost" layer that intercepts the function, or directly into driver code if there are no such layers. This is in contrast with vkGetInstanceProcAddr function pointers, which in many cases must jump through a dispatch table.

I agree the spec could be more explicit about this. I'll follow up.

[krOoze]

I was thinking the intention for it was to return NULL in that case too. I mean, it already is supposed to return NULL in the case of non-enabled extension commands (so the general design of the command seem to be to protect the user by returning NULL).

[critsec]

Sorry, yeah, it should return NULL there. I was just using that as evidence that it was only supposed to support device-level commands. The point was that if it supported instance-level commands, the function pointers you'd get back for them would be unusable -- which was unlikely to be the intent :).

[krOoze]

@critsec Yeah, I get that. I was making similar point at the layers repo original Issue. The biggest ambiguity is if it should return NULL or some fp. But either way the given pointer is unusable.

[krOoze]

@Novum By "unusable" we mean ✨"undefined behavior"✨.

[oddhack]

This should be fixed in the 1.0.69 spec update.

[Zingam]

Do I misunderstand something:

You can query Vulkan Device functions using either vkGetInstanceProcAddr or vkGetDeviceProcAddr. If you choose to use vkGetInstanceProcAddr, it will have an additional level built into the call chain, which will reduce performance slightly. However, the function pointer returned can be used for any device created later, as long as it is associated with the same Vulkan Instance. If, instead you use vkGetDeviceProcAddr, the call chain will be more optimized to the specific device, but it will only work for the device used to query the function function pointer. Also, unlike vkGetInstanceProcAddr, vkGetDeviceProcAddr can only be used on core Vulkan Device functions, or Device extension functions.

Taken from:
https://vulkan.lunarg.com/doc/sdk/1.1.73.0/windows/loader_and_layer_interface.html
Or the document is wrong?

[krOoze]

@Zingam Seems correct. What about that document?

You can get instance and device level commands with vkGetInstanceProcAddr.
You can get device level commands with vkGetDeviceProcAddr.
You cannot get instance (and physical device) level commands with vkGetDeviceProcAddr.

[ghost]

I would like to point out that @krOoze's comment was from April 27, 2018, while on March 5, 2018, two months before his comment, Mesa developers had to guess which commands are instance or device level:

Since the vk.xml file doesn't state if core commands are instance or
device level, we identify device level commands as the ones that take a
VkDevice, VkQueue or VkCommandBuffer as their first parameter.

https://www.mail-archive.com/mesa-dev@lists.freedesktop.org/msg186199.html

[ghost]

On @critsec's comment:

The intent was that vkGetDeviceProcAddr only supports functions of VkDevice or its children

Currently searching for an official list of children or a graph of parent-child relationships of Vulkan objects to confirm Mesa developers' guess...

[ghost]

"VkDevice objects are parents of many object types (all that take a VkDevice as a parameter to their creation)."
https://www.khronos.org/registry/vulkan/specs/1.0/html/chap2.html

[ghost]

"The returned function pointer must only be called with a dispatchable object (the first parameter) that is instance or a child of instance, e.g. VkInstance, VkPhysicalDevice, VkDevice, VkQueue, or VkCommandBuffer."
https://www.khronos.org/registry/vulkan/specs/1.0/html/chap3.html

You can indirectly derive from this VkDevice, VkQueue and VkCommandBuffer (tho 'e.g.' may mean it's incomplete).

[ghost]

"Device-Level Object
Logical device objects and their child objects. For example, VkDevice, VkQueue, and VkCommandBuffer objects are device-level objects."
https://www.khronos.org/registry/vulkan/specs/1.0/html/chap40.html#glossary

"For example", again, may mean the list is incomplete, but good enough as a confirm for now.

[krOoze]

@procedural

Mesa developers had to guess which commands are instance or device level:

That's unfortunate, but it was clear beyond any doubt in 1.0.69 (19 Feb 2018):

The function pointer must: only be called with a dispatchable object (the
first parameter) that is pname:device or a child of pname:device.

And it was strongly suggestive before, and as you quote they arrived to the correct conclusion.

Currently searching for an official list of children or a graph of parent-child relationships of Vulkan objects to confirm Mesa developers' guess...

There's simple rule to this. The parent is always the object you retrieved (i.e. via vkCreate, vkAllocate, or vkGet\Enumerate) the child from.

Besides it is in the vk.xml, e.g.:

<type category="handle" parent="VkInstance"><type>VK_DEFINE_HANDLE</type>(<name>VkPhysicalDevice</name>)</type>

VkQueue and VkCommandBuffer (tho 'e.g.' may mean it's incomplete).

Yea, it is brittle. Someone adding object later may forget to add it to that list at that unexpected place, so "e.g." to be safe. But at this point that is a complete list of dispatchable handle children.

[ghost]

#922 (comment)

[ghost]

There's simple rule to this. The parent is always the object you retrieved (i.e. via vkCreate, vkAllocate, or vkGet\Enumerate) the child from.

That's a good rule, thanks! :) I wouldn't mind if it ended up somewhere in the spec...

[krOoze]

@procedural Yea, the Object chapter could be improved.
You can still consume the vk.xml to get accurate parent-child relationship graph.