Support half precision floating point types #313
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mikepapadim
reviewed
Jan 23, 2024
| @@ -191,7 +191,7 @@ def build_spirv_toolkit_and_level_zero(rebuild=False): | |||
|
|
|||
| if (rebuild or build): | |||
| os.chdir(spirv_tool_kit) | |||
| subprocess.run(["git", "pull", "origin", "master"]) | |||
There was a problem hiding this comment.
I think this needs to be reverted
There was a problem hiding this comment.
We keep this until we merge the code from the Beehive SPIR-V Toolkit
| * The second {@code HalfFloat} input for the multiplication. | ||
| * @return A new {@HalfFloat} containing the results of the multiplication. | ||
| */ | ||
| public static HalfFloat mult(HalfFloat a, HalfFloat b) { |
There was a problem hiding this comment.
nit: any reason we called it mult and mul
| @@ -74,6 +74,8 @@ public OCLAssembler(TargetDescription target) { | |||
| emitLine("#pragma OPENCL EXTENSION cl_khr_fp64 : enable "); | |||
| } | |||
|
|
|||
| emitLine("#pragma OPENCL EXTENSION cl_khr_fp16 : enable "); | |||
There was a problem hiding this comment.
so, now we are going to emit this in all opencl kernels we generate, regardless if it uses half floating point?
There was a problem hiding this comment.
I think we should emit this line only if the platform supports it.
| @@ -74,6 +74,8 @@ public OCLAssembler(TargetDescription target) { | |||
| emitLine("#pragma OPENCL EXTENSION cl_khr_fp64 : enable "); | |||
| } | |||
|
|
|||
| emitLine("#pragma OPENCL EXTENSION cl_khr_fp16 : enable "); | |||
|
|
|||
| if (((OCLTargetDescription) target).supportsInt64Atomics()) { | |||
There was a problem hiding this comment.
can we have a similar condition instead?
jjfumero
requested changes
Jan 24, 2024
|
Unit-test are passing for SPIR-V and PTX. For OpenCL, depending on the device, this passes. In my case, using RTX 2060 and the NVIDIA Drivers 510.54 do not pass. |
|
I included a function that checks if FP16 is supported. However, since on my machine I do not have this capability but I can pass the tests for OpenCL, we decided with @jjfumero to keep the printing of the OpenCL pragma for the time being, regardless of whether it is supported, until we figure out why this is happening. |
|
Yes, we can query the capability. The pragma can always be generated. We need to investigate how to run OpenCL FP16 on modern NVIDIA GPUs. |
jjfumero
approved these changes
Jan 24, 2024
|
Ok, lets merge it then and we can iterate for a fix in codegen when they add the extension in the driver |
mikepapadim
approved these changes
Jan 24, 2024
|
It appears a new conflict. @mairooni, can you take a look? |
jjfumero
added a commit
that referenced
this pull request
Jan 30, 2024
TornadoVM 1.0.1 ---------------- 30/01/2024 Improvements ~~~~~~~~~~~~~~~~~~ - `#305 <https://github.com/beehive-lab/TornadoVM/pull/305>`_: Under-demand data transfer for custom data ranges. - `#305 <https://github.com/beehive-lab/TornadoVM/pull/305>`_: Copy out subregions using the execution plan: - `#313 <https://github.com/beehive-lab/TornadoVM/pull/313>`_: Initial support for Half-Precision (FP16) data types. - `#311 <https://github.com/beehive-lab/TornadoVM/pull/311>`_: Enable Multi-Task Multiple Device (MTMD) model from the ``TornadoExecutionPlan`` API: - `#315 <https://github.com/beehive-lab/TornadoVM/pull/315>`_: Math ``Ceil`` function added Compatibility/Integration ~~~~~~~~~~~~~~~~~~~~~~~~~~~ - `#294 <https://github.com/beehive-lab/TornadoVM/pull/294>`_: Separation of the OpenCL Headers from the code base. - `#297 <https://github.com/beehive-lab/TornadoVM/pull/297>`_: Separation of the LevelZero JNI API in a separate repository. - `#301 <https://github.com/beehive-lab/TornadoVM/pull/301>`_: Temurin configuration supported. - `#304 <https://github.com/beehive-lab/TornadoVM/pull/304>`_: Refactor of the common phases for the JIT compiler. - `#316 <https://github.com/beehive-lab/TornadoVM/pull/316>`_: Beehive SPIR-V Toolkit version updated. Bug Fixes ~~~~~~~~~~~~~~~~~~ - `#298 <https://github.com/beehive-lab/TornadoVM/pull/298>`_: OpenCL Codegen fixed open-close brackets. - `#300 <https://github.com/beehive-lab/TornadoVM/pull/300>`_: Python Dependencies fixed for AWS - `#308 <https://github.com/beehive-lab/TornadoVM/pull/308>`_: Runtime check for Grid-Scheduler names - `#309 <https://github.com/beehive-lab/TornadoVM/pull/309>`_: Fix check-style to support STR templates - `#314 <https://github.com/beehive-lab/TornadoVM/pull/314>`_: emit Vector16 Capability for 16-width vectors
jjfumero
added a commit
that referenced
this pull request
Jan 30, 2024
Improvements ~~~~~~~~~~~~~~~~~~ - `#305 <https://github.com/beehive-lab/TornadoVM/pull/305>`_: Under-demand data transfer for custom data ranges. - `#313 <https://github.com/beehive-lab/TornadoVM/pull/313>`_: Initial support for Half-Precision (FP16) data types. - `#311 <https://github.com/beehive-lab/TornadoVM/pull/311>`_: Enable Multi-Task Multiple Device (MTMD) model from the ``TornadoExecutionPlan`` API. - `#315 <https://github.com/beehive-lab/TornadoVM/pull/315>`_: Math ``Ceil`` function added. Compatibility/Integration ~~~~~~~~~~~~~~~~~~~~~~~~~~~ - `#294 <https://github.com/beehive-lab/TornadoVM/pull/294>`_: Separation of the OpenCL Headers from the code base. - `#297 <https://github.com/beehive-lab/TornadoVM/pull/297>`_: Separation of the LevelZero JNI API in a separate repository. - `#301 <https://github.com/beehive-lab/TornadoVM/pull/301>`_: Temurin configuration supported. - `#304 <https://github.com/beehive-lab/TornadoVM/pull/304>`_: Refactor of the common phases for the JIT compiler. - `#316 <https://github.com/beehive-lab/TornadoVM/pull/316>`_: Beehive SPIR-V Toolkit version updated. Bug Fixes ~~~~~~~~~~~~~~~~~~ - `#298 <https://github.com/beehive-lab/TornadoVM/pull/298>`_: OpenCL Codegen fixed open-close brackets. - `#300 <https://github.com/beehive-lab/TornadoVM/pull/300>`_: Python Dependencies fixed for AWS. - `#308 <https://github.com/beehive-lab/TornadoVM/pull/308>`_: Runtime check for Grid-Scheduler names. - `#309 <https://github.com/beehive-lab/TornadoVM/pull/309>`_: Fix check-style to support STR templates. - `#314 <https://github.com/beehive-lab/TornadoVM/pull/314>`_: emit Vector16 Capability for 16-width vectors.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR provides support for half-float types, i.e. float values represented by 16 bits instead of 32. Since Java does not currently offer a half-float type, in this implementation it is represented by a new class,
HalfFloat.The
HalfFloatclass offers two constructors. The first receives a float value, which is internally converted to float-16, using theFloat.floatToFloat16function. The second directly receives a short value, which represents a half-float.The
HalfFloatclass exposes a set of operations that can be performed between half-float values, specifically addition, subtraction, multiplication, and division.Finally, a new off-heap array type for half-float values (
HalfFloatArrayclass) has been included.Backend/s tested
Mark the backends affected by this PR.
OS tested
Mark the OS where this PR is tested.
Did you check on FPGAs?
If it is applicable, check your changes on FPGAs.
How to test the new patch?
Tests have been added for the following scenarios:
HalfFloatinstance in the kernel:tornado-test -V uk.ac.manchester.tornado.unittests.arrays.TestArrays#testHalfFloatInitializationtornado-test -V uk.ac.manchester.tornado.unittests.arrays.TestArrays#testVectorAdditionHalfFloattornado-test -V uk.ac.manchester.tornado.unittests.arrays.TestArrays#testVectorSubtractionHalfFloattornado-test -V uk.ac.manchester.tornado.unittests.arrays.TestArrays#testVectorMultiplicationHalfFloattornado-test -V uk.ac.manchester.tornado.unittests.arrays.TestArrays#testVectorDivisionHalfFloatfromElementsandfromArrayfunctions of theHalfFloatArrayclass:tornado-test -V uk.ac.manchester.tornado.unittests.api.TestAPI#testSegmentsHalfFloatsfromSegmentfunction of theHalfFloatArrayclass:tornado-test -V uk.ac.manchester.tornado.unittests.api.TestAPI#testBuildWithSegmentsHalfFloat