Skip to content

Commit

Permalink
[Vulkan][Docs] Minor updates following Vulkan target query. (apache#8151
Browse files Browse the repository at this point in the history
)

- Better error messages, specifying which capability is needed.

- Documentation section outlining the different target capabilities,
  which vulkan properties they correspond to, and which spir-v
  capabilities they are required by.

Co-authored-by: Eric Lunderberg <elunderberg@octoml.ai>
  • Loading branch information
2 people authored and trevor-m committed Jun 17, 2021
1 parent e5d269e commit e186df6
Show file tree
Hide file tree
Showing 4 changed files with 253 additions and 10 deletions.
2 changes: 1 addition & 1 deletion docs/dev/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -245,7 +245,7 @@ for learning-based optimizations.


.. toctree::
:maxdepth: 1
:maxdepth: 2

runtime
debugger
Expand Down
10 changes: 10 additions & 0 deletions docs/dev/runtime.rst
Original file line number Diff line number Diff line change
Expand Up @@ -296,3 +296,13 @@ To support extension types, we used a registry system to register type related i
in C++, see `Extension types`_ for more details.

.. _Extension types: https://github.com/apache/tvm/tree/main/apps/extension


Runtime-Specific Information
============================

.. toctree::
:maxdepth: 1
:glob:

runtimes/*
207 changes: 207 additions & 0 deletions docs/dev/runtimes/vulkan.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,207 @@
.. Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
.. http://www.apache.org/licenses/LICENSE-2.0
.. Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
.. _tvm-runtime-vulkan:

Vulkan Runtime
==============

TVM supports using Vulkan compute shaders to execute queries. Each
computational kernel is compiled into a SPIR-V shader, which can then
be called using the TVM interface.

.. _tvm-runtime-vulkan-features:

Vulkan Features, Limits
-----------------------

.. _Required Limits: https://www.khronos.org/registry/vulkan/specs/1.2-extensions/html/vkspec.html#limits-minmax

Since different Vulkan implementations may enable different optional
features or have different physical limits, the code generation must
know which features are available to use. These correspond to
specific Vulkan capabilities/limits as in
:ref:`Vulkan Capabilities Table <tvm-table-vulkan-capabilities>`.
If unspecified, TVM assumes that a capability is not available, or
that a limit is the minimum guaranteed by the Vulkan spec in the
`Required Limits`_ section.

These parameters can be either explicitly specific when defining a
:ref:`Target <tvm-target-specific-target>`, or can be queried from a
device. To query from a device, the special parameter
``-from_device=N`` can be used to query all vulkan device parameters
from device id ``N``. Any additional parameters explicitly specified
will override the parameters queried from the device.

.. _VkSubgroupFeatureFlagBits: https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VkSubgroupFeatureFlagBits.html

.. list-table:: Vulkan Capabilities
:name: tvm-runtime-table-vulkan-capabilities
:header-rows: 1

* - Target Parameter
- Required Vulkan Version/Extension
- Parameter Queried
- Default Value

* - ``supported_subgroup_operations``
- Vulkan 1.1+
- ``VkPhysicalDeviceSubgroupProperties::supportedOperations``
- 0 (interpreted as `VkSubgroupFeatureFlagBits`_)

* - ``max_push_constants_size``
-
- ``VkPhysicalDeviceLimits::maxPushConstantsSize``
- 128 bytes

* - ``max_uniform_buffer_range``
-
- ``VkPhysicalDeviceLimits::maxUniformBufferRange``
- 16384 bytes


* - ``max_storage_buffer_range``
-
- ``VkPhysicalDeviceLimits::maxStorageBufferRange``
- 2\ :sup:`27`\ bytes


* - ``max_per_stage_descriptor_storage_buffer``
-
- ``VkPhysicalDeviceLimits::maxPerStageDescriptorStorageBuffers``
- 4


* - ``supports_storage_buffer_storage_class``
- VK_KHR_storage_buffer_storage_class
-
- false


* - ``supports_storage_buffer_8bit_access``
- VK_KHR_8bit_storage
- ``VkPhysicalDevice8BitStorageFeaturesKHR::storageBuffer8BitAccess``
- false


* - ``supports_storage_buffer_16bit_access``
- VK_KHR_16bit_storage
- ``VkPhysicalDevice16BitStorageFeaturesKHR::storageBuffer16BitAccess``
- false


* - ``supports_float16``
- VK_KHR_shader_float16_int8
- ``VkPhysicalDeviceShaderFloat16Int8FeaturesKHR::shaderFloat16``
- false


* - ``supports_float64``
-
- ``VkPhysicalDeviceFeatures::shaderFloat64``
- false


* - ``supports_int8``
- VK_KHR_shader_float16_int8
- ``VkPhysicalDeviceShaderFloat16Int8FeaturesKHR::shaderInt8``
- false


* - ``supports_int16``
-
- ``VkPhysicalDeviceFeatures::shaderInt16``
- false


* - ``supports_int64``
-
- ``VkPhysicalDeviceFeatures::shaderInt64``
- false



As of May 2021, not all Vulkan implementations are supported. For
example, support for 64-bit integers is required. If a Vulkan target
is not supported, an error message should be issued during SPIR-V code
generation. Efforts are also underway to remove these requirements
and support additional Vulkan implementations.


.. _tvm-runtime-vulkan-spirv-capabilities:

SPIR-V Capabilities
-------------------

Some of the device-specific capabilities also correspond to SPIR-V
capabilities or extensions that must be declared in the shader, or a
minimum SPIR-V version required in order to use a feature. The
TVM-generated shaders will declare the minimum set of
extensions/capabilities and the minimum allowed version of SPIR-V
that are needed to execute the compiled graph.

If the shader generation requires a capability or extension that is
not enabled in the ``Target``, an exception will be raised.


.. list-table:: Vulkan Capabilities
:name: tvm-table-vulkan-capabilities
:header-rows: 1

* - Target Parameter
- Required SPIR-V Version/Extension
- Declared Capability

* - ``supported_subgroup_operations``
- SPIR-V 1.3+
- Varies, see `VkSubgroupFeatureFlagBits`_

* - ``supports_storage_buffer_storage_class``
- SPV_KHR_storage_buffer_storage_class
-

* - ``supports_storage_buffer_8bit_access``
- SPV_KHR_8bit_storage
- StorageBuffer8BitAccess

* - ``supports_storage_buffer_16bit_access``
- SPV_KHR_16bit_storage
- StorageBuffer16BitAccess

* - ``supports_float16``
-
- Float16


* - ``supports_float64``
-
- Float64


* - ``supports_int8``
-
- Int8


* - ``supports_int16``
-
- Int16


* - ``supports_int64``
-
- Int64
44 changes: 35 additions & 9 deletions src/target/spirv/ir_builder.cc
Original file line number Diff line number Diff line change
Expand Up @@ -449,24 +449,42 @@ void IRBuilder::AddCapabilityFor(const DataType& dtype) {
// Declare appropriate capabilities for int/float types
if (dtype.is_int() || dtype.is_uint()) {
if (dtype.bits() == 8) {
ICHECK(spirv_support_.supports_int8) << "Vulkan target does not support Int8 capability";
ICHECK(spirv_support_.supports_int8)
<< "Vulkan target does not support Int8 capability. "
<< "If your device supports 8-bit int operations, "
<< "please either add -supports_int8=1 to the target, "
<< "or query all device parameters by adding -from_device=0.";
capabilities_used_.insert(spv::CapabilityInt8);
} else if (dtype.bits() == 16) {
ICHECK(spirv_support_.supports_int16) << "Vulkan target does not support Int16 capability";
ICHECK(spirv_support_.supports_int16)
<< "Vulkan target does not support Int16 capability. "
<< "If your device supports 16-bit int operations, "
<< "please either add -supports_int16=1 to the target, "
<< "or query all device parameters by adding -from_device=0.";
capabilities_used_.insert(spv::CapabilityInt16);
} else if (dtype.bits() == 64) {
ICHECK(spirv_support_.supports_int64) << "Vulkan target does not support Int64 capability";
ICHECK(spirv_support_.supports_int64)
<< "Vulkan target does not support Int64 capability. "
<< "If your device supports 64-bit int operations, "
<< "please either add -supports_float64=1 to the target, "
<< "or query all device parameters by adding -from_device=0.";
capabilities_used_.insert(spv::CapabilityInt64);
}

} else if (dtype.is_float()) {
if (dtype.bits() == 16) {
ICHECK(spirv_support_.supports_float16)
<< "Vulkan target does not support Float16 capability";
<< "Vulkan target does not support Float16 capability. "
<< "If your device supports 16-bit float operations, "
<< "please either add -supports_float16=1 to the target, "
<< "or query all device parameters by adding -from_device=0.";
capabilities_used_.insert(spv::CapabilityFloat16);
} else if (dtype.bits() == 64) {
ICHECK(spirv_support_.supports_float64)
<< "Vulkan target does not support Float64 capability";
<< "Vulkan target does not support Float64 capability. "
<< "If your device supports 16-bit float operations, "
<< "please either add -supports_float16=1 to the target, "
<< "or query all device parameters by adding -from_device=0.";
capabilities_used_.insert(spv::CapabilityFloat64);
}
}
Expand All @@ -478,17 +496,25 @@ void IRBuilder::AddCapabilityFor(const DataType& dtype) {
// supports Int8 but doesn't support 8-bit buffer access.
if (dtype.bits() == 8) {
ICHECK(spirv_support_.supports_storage_buffer_8bit_access)
<< "Vulkan target does not support StorageBuffer8BitAccess";
<< "Vulkan target does not support StorageBuffer8BitAccess. "
<< "If your device supports 8-bit buffer access, "
<< "please either add -supports_8bit_buffer=1 to the target, "
<< "or query all device parameters by adding -from_device=0.";
capabilities_used_.insert(spv::CapabilityStorageBuffer8BitAccess);
extensions_used_.insert("SPV_KHR_8bit_storage");

ICHECK(spirv_support_.supports_storage_buffer_storage_class)
<< "Illegal Vulkan target description. "
<< "Vulkan spec requires extension VK_KHR_storage_buffer_storage_class "
<< "if VK_KHR_8bit_storage is supported";
<< "if VK_KHR_8bit_storage is supported. "
<< "Please either add -supports_storage_buffer_storage_class=1 to the target, "
<< "or query all device parameters by adding -from_device=0.";
} else if (dtype.bits() == 16) {
ICHECK(spirv_support_.supports_storage_buffer_8bit_access)
<< "Vulkan target does not support StorageBuffer16BitAccess";
ICHECK(spirv_support_.supports_storage_buffer_16bit_access)
<< "Vulkan target does not support StorageBuffer16BitAccess. "
<< "If your device supports 16-bit buffer access, "
<< "please either add -supports_16bit_buffer=1 to the target, "
<< "or query all device parameters by adding -from_device=0.";

extensions_used_.insert("SPV_KHR_16bit_storage");
if (spirv_support_.supports_storage_buffer_storage_class) {
Expand Down

0 comments on commit e186df6

Please sign in to comment.