-
Notifications
You must be signed in to change notification settings - Fork 2.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] WebGPU EP [skip ci] #21904
base: main
Are you sure you want to change the base?
[WIP] WebGPU EP [skip ci] #21904
Conversation
* | ||
* \since Version 1.20. | ||
*/ | ||
ORT_API2_STATUS(SessionOptionsAppendExecutionProvider_WGPU, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using the generic SessionOptionsAppendExecutionProvider is preferred over adding EP specific functions
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you have any suggestions if I want to use the generic SessionOptionsAppendExecutionProvider
with passing pointer (void*)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A void* is just a number for the address so could you pass the address of the OrtWGPUProviderOptions instance?
Not quite as pure but the advantage of that is you don't need to expose a new function via all the ORT language bindings (which would be C#, Objective-C and Java for mobile).
But that said, why do the various handles have to be created outside of the EP and passed in vs the EP being passed the info to create them. I would think that would be easier to use from other languages,otherwise you're going to have to document how to create OrtWGPUProviderOptions in all those languages.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A void* is just a number for the address so could you pass the address of the OrtWGPUProviderOptions instance?
It's the same question. "How to pass a pointer to SessionOptionsAppendExecutionProvider
" still need to answered.
Not quite as pure but the advantage of that is you don't need to expose a new function via all the ORT language bindings (which would be C#, Objective-C and Java for mobile).
But that said, why do the various handles have to be created outside of the EP and passed in vs the EP being passed the info to create them. I would think that would be easier to use from other languages,otherwise you're going to have to document how to create OrtWGPUProviderOptions in all those languages.
From my understanding, there is not going to be use case of passing any pointers through language bindings. The only use case (MSEdge) with passing pointers is to static link ORT and use C-API, which requires at least to pass the pointer of dawn_proc_table. So, I am totally fine with using SessionOptionsAppendExecutionProvider
everywhere else with string-to-string kvp except a minimal change in C-API to enable the scenario above.
BTW, there is a seperate PR for the API change only -> #21838. Further discussion can be done there.
Co-authored-by: Yulong Wang <7679871+fs-eire@users.noreply.github.com>
### Description <!-- Describe your changes. --> ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> --------- Co-authored-by: Yulong Wang <7679871+fs-eire@users.noreply.github.com>
adds webgpu support for LayerNormalization, SimplifiedLayerNormalization, SkipLayerNormalization, SkipSimplifiedLayerNormalization --------- Co-authored-by: Yulong Wang <7679871+fs-eire@users.noreply.github.com>
1. On Linux path strings are not std::wstring. They should be std::string. 2. "auto i =0" means i will be int. But sometimes we want to it to be size_t.
- support for input_skip_bias_sum in SkipLayerNorm - use GetElementAt in concat --------- Co-authored-by: Yulong Wang <7679871+fs-eire@users.noreply.github.com>
|
||
private: | ||
int64_t axis_; | ||
float epsilon_; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was trying to build WebGPU EP on android, and it is raising -Wunused-private-field error for axis_
, epsilon_
, stash_type_
, x_size_
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thank you for reporting and I will go with a fix. Would you like to help to create a WebGPU EP CI pipeline for Android?
### Description This change introduces the WebGPU EP into ONNX Runtime. To make the PR as simple as possible, this PR excluded the following: - C API changes for WebGPU EP - actual implementation of WebGPU EP. Currently in this PR, WebGPU is a stub implementation that does not register any kernel. - Python IO Binding update - Node.js IO Binding update This PR now contains only 43 file changes (while the working branch contains 130+) and hopefully this makes it easier to review. There is going to be separated PRs for each mentioned above. Current working branch: #21904
Description
For code review purpose only. Offer a place for discussion.
see also:
prerequisite:
related changes:
TODO items