cl_khr_tensor extension #2

linehill · 2023-10-30T14:59:04Z

No description provided.

pjaaskel

This is great!

ext/cl_khr_tensor.asciidoc

pjaaskel · 2023-10-31T13:07:17Z

ext/cl_khr_tensor.asciidoc

+
+=== Sample Codes
+
+Helper functions used in the follow up tensor code samples:


Let's first define a couple of helper functions which are used in the tensor usage examples:

ext/cl_khr_tensor.asciidoc

pjaaskel · 2023-10-31T13:09:16Z

ext/cl_khr_tensor.asciidoc

+// might not use some of the tensor.
+assert(clEnqueueReadTensor(..., t0, ...) == CL_INVALID_OPERATION);
+----
+


The OpenCL C builtins for acessing the tensor metadata needs to be defined (and after that the SPIR-V extensions).

Could the OpenCL C support for tensors be in a separate extension? I think, there is a lot to be covered.

Built-in kernels should be able to define kernels tensor arguments without the OpenCL C tensor extension.

Yes. Since this will be useful already for the BiKs, it doesn't require the OpenCL C or SPIR-V counterparts, you're right.

pjaaskel · 2023-10-31T13:11:44Z

ext/cl_khr_tensor.asciidoc

+[[cl_khr_tensor]]
+== Tensor Data Type
+
+Purpose of this extension is to provide ...


"This extension provides a new opaque OpenCL datatype called cl_tensor. It is used for storing N-dimensional tensor data in implementation-optimized way. The datatype is designed to be efficiently used within the cl_khr_command_buffers extension to capture task graphs which can utilize tensors as input, output and temporary storage." [etc etc]

Added this and tweaked it a little.

bashbaug

Interesting extension! There are some parts of it that we might want to consider refactoring out, since they could be applicable to non-tensor objects also:

The ability to bind a description of an object to an existing buffer. We already have this slightly, albeit in the other direction, in the form of the "2d image from buffer" extensions.
The ability to attach "temporary" (or "virtual") objects to a command buffer, that are owned by a command buffer but that do not persist beyond command buffer execution.

ext/cl_khr_tensor.asciidoc

bashbaug · 2023-10-31T14:44:10Z

ext/cl_khr_tensor.asciidoc

+
+[source,c]
+----
+cl_int clEnqueueReadTensor(


Presumably these read and write functions are invalid for CL_TENSOR_COMMAND_BUFFER_TEMPORARY tensors?

Yes and the functions should return CL_INVALID_OPERATION. This is attempted to be stated in CL_MEM_COMMAND_BUFFER_TEMPORARY property description (perhaps it's not clear enough?).

Co-authored-by: Ben Ashbaugh <ben.ashbaugh@intel.com> Co-authored-by: Pekka Jääskeläinen <pekka.jaaskelainen@tuni.fi>

* cl_khr_tensor -> cl_exp_tensor. * Remove cl_khr_command_buffer requirement.

* Fix signed -> unsigned. * Single line cl{Retain,Release}TensorObject declaration.

…Tensor. * Clarify in clEnqueue{TranslateFrom,TranslateTo}Tensor that data read from / written to the tensor in opaque manner.

linehill · 2023-11-02T13:01:07Z

@bashbaug, temporary command buffer object property has been refactored out to be a buffer object property. Would this look good to you?

bashbaug · 2023-11-07T00:34:03Z

Would this look good to you?

This seems like a good direction, yes.

A few other things we could consider:

Perhaps the "temporary command buffer object" could be created from and have its object lifetime attached to a specific command buffer? This concept of a "virtual" allocation is something that I really liked from OpenVX (see e.g. vxCreateVirtualImage).
Eventually we may want to refactor the property into its own extension so it could apply to ordinary buffers and images as well, though I think it's fine to keep it as part of this extension for now while proving its usefulness.

Thanks!

pjaaskel · 2023-11-10T15:10:17Z

ext/cl_khr_tensor.asciidoc

+which can utilize tensors as input, output and temporary storage.
+
+=== General information
+


Let's write a motivational text here which is interfacing to the DBKs (refer to the WiP spec) on the backend, and to better interface to higher-level models for neural networks and other libraries/formats dealing with tensors.

* new section for tensor data type * add origin, region and pitch parameters for clEnqueueTranslate*Tensor. * Update code samples. * Add take on accessing tensors in OpenCL C.

pjaaskel

Kiitos. Looks good. Please fix these and then submit the PR to the Khronos repo to hopefully get further feedback.

ext/cl_exp_tensor.asciidoc

pjaaskel · 2023-11-15T11:33:52Z

ext/cl_exp_tensor.asciidoc


 // TODO: add clEnqueueCopyTensor

 // TODO: add clEnqueueFillTensor?

-// TODO: add command buffer variants for clEnqueue{copy,read,write}Tensor.
-
+TODO: add command buffer variants for clEnqueue*Tensor.


Perhaps do this still before sending the Khronos/OpenCL-Docs pull request?

ext/cl_exp_tensor.asciidoc

Co-authored-by: Pekka Jääskeläinen <pekka.jaaskelainen@tuni.fi>

* Add error codes for tensor translation commands. * Tweaked mem_pitch semantics.

pjaaskel · 2023-11-17T09:51:15Z

ext/cl_exp_tensor.asciidoc

@@ -418,11 +421,131 @@ an abstract function that accesses a tensor element in its storage at
 given coordinate. The method how the coordinates translate to tensor
 storage addresses is unspecified.

+*clEnqueueTranslateFsomTensor* and *clEnqueueTranslateToTensor*


Maybe import/export are even better word here instead of 'translate'?

Fixed type and renamed.

pjaaskel · 2023-11-17T09:52:37Z

ext/cl_exp_tensor.asciidoc

 // TODO: add clEnqueueCopyTensor

 // TODO: add clEnqueueFillTensor?

-TODO: add command buffer variants for clEnqueue*Tensor.
+If *cl_khr_command_buffer* is is supported, then the following command


* Fix typos.

linehill · 2023-11-23T11:53:37Z

Closing. This PR has moved to KhronosGroup#1006.

Sketch something for cl_khr_tensor

f3a2bb9

pjaaskel requested changes Oct 31, 2023

View reviewed changes

pjaaskel reviewed Oct 31, 2023

View reviewed changes

bashbaug reviewed Oct 31, 2023

View reviewed changes

linehill and others added 12 commits November 2, 2023 14:16

Apply suggestions from code review

bf94321

Co-authored-by: Ben Ashbaugh <ben.ashbaugh@intel.com> Co-authored-by: Pekka Jääskeläinen <pekka.jaaskelainen@tuni.fi>

* Add brief introduction.

36db4b6

* cl_khr_tensor -> cl_exp_tensor. * Remove cl_khr_command_buffer requirement.

Add contributors

baa7688

* Fix name for add kernel creator

9db1e65

* cl_tensor_type -> cl_tensor _datatype.

141643d

* Fix signed -> unsigned. * Single line cl{Retain,Release}TensorObject declaration.

* clEnqueue(Read,Write)Tensor -> clEnqueue(TranslateFrom,TranslateTo)…

db91aee

…Tensor. * Clarify in clEnqueue{TranslateFrom,TranslateTo}Tensor that data read from / written to the tensor in opaque manner.

Refactor command buffer temporary property out of tensor

6fecc4e

Fix cl_tensor_type -> cl_tensor_datatype

f55a904

Add an open question

6d1c26f

Add CL_INVALID_TENSOR error code

52d8bb3

Require either buffer or host_ptr to be non-NULL

534bcef

Regenerate html for cl_exp_tensor

7447be2

cl_khr_tensor.* -> cl_exp_tensor.*

85edd05

pjaaskel reviewed Nov 10, 2023

View reviewed changes

* Add overview

8fe9046

* new section for tensor data type * add origin, region and pitch parameters for clEnqueueTranslate*Tensor. * Update code samples. * Add take on accessing tensors in OpenCL C.

pjaaskel requested changes Nov 15, 2023

View reviewed changes

linehill and others added 2 commits November 15, 2023 14:00

Apply suggestions from code review

363c69b

Co-authored-by: Pekka Jääskeläinen <pekka.jaaskelainen@tuni.fi>

* Add command buffer counterparts for tensor translation commands

6402f59

* Add error codes for tensor translation commands. * Tweaked mem_pitch semantics.

pjaaskel requested changes Nov 17, 2023

View reviewed changes

* translate -> import/export

57fda1a

* Fix typos.

pjaaskel approved these changes Nov 17, 2023

View reviewed changes

linehill closed this Nov 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cl_khr_tensor extension #2

cl_khr_tensor extension #2

linehill commented Oct 30, 2023

pjaaskel left a comment

pjaaskel Oct 31, 2023

pjaaskel Oct 31, 2023

linehill Nov 2, 2023

pjaaskel Nov 6, 2023

pjaaskel Oct 31, 2023

linehill Nov 2, 2023

bashbaug left a comment

bashbaug Oct 31, 2023

linehill Nov 2, 2023

linehill commented Nov 2, 2023

bashbaug commented Nov 7, 2023

pjaaskel Nov 10, 2023

pjaaskel left a comment

pjaaskel Nov 15, 2023

pjaaskel Nov 17, 2023

pjaaskel Nov 17, 2023

linehill Nov 17, 2023

pjaaskel Nov 17, 2023

linehill Nov 17, 2023

linehill commented Nov 23, 2023


		=== Sample Codes

		Helper functions used in the follow up tensor code samples:

		which can utilize tensors as input, output and temporary storage.

		=== General information

cl_khr_tensor extension #2

cl_khr_tensor extension #2

Conversation

linehill commented Oct 30, 2023

pjaaskel left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bashbaug left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

linehill commented Nov 2, 2023

bashbaug commented Nov 7, 2023

Choose a reason for hiding this comment

pjaaskel left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

linehill commented Nov 23, 2023