-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
How TorchSharp & Onnx can address the pain points of the ~900 ML.NET Apr2021 survey responses? #5874
Comments
@briacht can you take a look at this? |
Yes! I had a conversation with @GeorgeS2019 about deep learning in .NET. As part of our deep learning plan, we will enable NLP scenarios. I believe these suggestions will start at the TorchSharp level. |
@michaelgsharp @briacht The method 2 involves shapeDictionary, which is particularly useful for working with variable dimension inputs and outputs. All examples provided by OnnxCatalog.ApplyOnnxMode use Image Classification through squeezenet onnx from Onnx zoo models The users of the April survey requested NLP use case. There are more image related examples involving ML.NET than NLP. Image use case, unlike NLP, which often does not involve variable dimension inputs and outputs. I suggest the document provides, in addition to image, NLP examples from e.g. Onnx Zoo model e.g. GPT-2 => which will show HOW TO DEAL with VARIABLE dimension and the need to use ShapeDictionary.
This statement was introduced to the documentation through this PR to address the need for handling of variable axes of ONNX-models often found in NLP use case We need expand the documentation to elaborate how to handle variable axes (e.g. using ShapeDictionary) especially in NLP use case |
Thanks for the suggestion @GeorgeS2019! @luisquintanilla can we add an issue to the docs repo for this? |
As correctly pointed out by @antoniovs1029
This missing feature has been raised by @yaeldekel here Add "Reshape Transform" => I request to look into this "tensor reshaping" and the variable dimension discussed above. Perhaps both are related. Here are NLP cases applying OnnxCatalog.ApplyOnnxModel onnx models Perhaps by implementing the "Reshape Transform", this could address challenges when working with NLP with ML.NET? |
Is your feature request related to a problem? Please describe.
The pain points of the Apr 2021 ML.NET survey and the result discussions
Describe the solution you'd like
It is clear that NLP is high on priority
This means **more deep learning NLP use cases ** e.g. using ML.NET to load pretrained Hugging Face transformer models using OnnxRuntime
We can do that by porting some of the PyTorch NLP Transformer codes to c# to address the pain points of ML.NET!!
The porting process is now more feasible due to [the recent renaming effort of TorchSharp] (dotnet/TorchSharp#308 (comment)) which make TorchSharp codes MORE resemble PyTorch
The text was updated successfully, but these errors were encountered: