Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Samples added to WinML Ext #36

Merged
merged 6 commits into from
Feb 1, 2019
Merged

Samples added to WinML Ext #36

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
4ee9aa3
Samples added to WinML Ext
kiritigowda Feb 1, 2019
e73cd3e
Samples updated
kiritigowda Feb 1, 2019
55019d8
Fix link to labels
kiritigowda Feb 1, 2019
59b12a4
amd_winml.sln - default project winml
kiritigowda Feb 1, 2019
fedfe54
vx_ext_winml.h fix
kiritigowda Feb 1, 2019
1889f17
fix winML images
kiritigowda Feb 1, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion amd_openvx_extensions/amd_winml.sln
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ Global
HideSolutionNode = FALSE
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {3FB58533-A51A-34B5-ACE3-6B1BFD797DB1}
SolutionGuid = {F164F9CB-D02C-4BC0-9122-54E6CEA7DA72}
SolutionGuid = {3FB58533-A51A-34B5-ACE3-6B1BFD797DB1}
EndGlobalSection
EndGlobal
4 changes: 2 additions & 2 deletions amd_openvx_extensions/amd_winml/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# AMD WinML Extension
The AMD WinML (vx_winml) is an OpenVX module that implements a mechanism to access WinML functionality as OpenVX kernels. These kernels can be accessed from within OpenVX framework using OpenVX API call [vxLoadKernels](https://www.khronos.org/registry/vx/specs/1.0.1/html/da/d83/group__group__user__kernels.html#gae00b6343fbb0126e3bf0f587b09393a3)(context, "vx_winml").

<p align="center"><img width="80%" img height="60%"src="../../docs/images/winmlFrameWorks.png" /></p>
<p align="center"><img width="80%" src="../../docs/images/winmlFrameWorks.png" /></p>

WinML extension will allow developers to import a pre-trained ONNX model into an OpenVX graph and add hundreds of different pre & post processing `vision`/`generic`/`user-defined` functions, available in OpenVX and OpenCV interop, to the input and output of the neural net model. This will allow developers to build an end to end application for inference.

<p align="center"><img width="80%" img height="60%"src="../../docs/images/winmlRuntime.png" /></p>
<p align="center"><img width="100%" src="../../docs/images/winmlRuntime.png" /></p>

## List of WinML-interop kernels
The following is a list of WinML functions that have been included in the vx_winml module.
Expand Down
12 changes: 6 additions & 6 deletions amd_openvx_extensions/amd_winml/include/vx_ext_winml.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,12 +54,12 @@ extern "C" {
* \retval vx_node A node reference. Any possible errors preventing a successful creation should be checked using <tt>\ref vxGetStatus</tt>*/
VX_API_ENTRY vx_node VX_API_CALL vxExtWinMLNode_OnnxToMivisionX
(
vx_graph graph,
vx_scalar modelLocation,
vx_scalar inputTensorName,
vx_scalar outputTensorName,
vx_tensor inputTensor,
vx_tensor outputTensor,
vx_graph graph,
vx_scalar modelLocation,
vx_scalar inputTensorName,
vx_scalar outputTensorName,
vx_tensor inputTensor,
vx_tensor outputTensor,
vx_scalar deviceKind
);

Expand Down
68 changes: 65 additions & 3 deletions amd_openvx_extensions/amd_winml/samples/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Get ONNX models from [ONNX Model Zoo](https://github.com/onnx/models)
* Use [Netron](https://lutzroeder.github.io/netron/) to open the model.onnx
* Look at Model Properties to find Input & Output Tensor Name (data_0 - input; softmaxout_1 - output)
* Look at output tensor dimensions (n,c,h,w - [1,1000,1,1] for softmaxout_1)
* Use the label file - Labels.txt and sample image - car.JPEG to run samples
* Use the label file - [data\Labels.txt](data/Labels.txt) and sample image - data\car.JPEG to run samples

### winML-image.gdf - Single Image Inference

Expand All @@ -32,7 +32,7 @@ read input_image FULL_PATH_TO\data\car.JPEG
data modelLocation = scalar:STRING,FULL_PATH_TO\squeezenet\model.onnx:view,resultWindow
````

* Add full path to the data\Labels.txt provided in this folder in line 34
* Add full path to the [data\Labels.txt](data/Labels.txt) provided in this folder in line 34
````
data labelLocation = scalar:STRING,FULL_PATH_TO\data\Labels.txt
````
Expand All @@ -54,7 +54,69 @@ Make the below changes in the `winML-live.gdf` file to run the inference
data modelLocation = scalar:STRING,FULL_PATH_TO\squeezenet\model.onnx:view,resultWindow
````

* Add full path to the data\Labels.txt provided in this folder in line 25
* Add full path to the [data\Labels.txt](data/Labels.txt) provided in this folder in line 25
````
data labelLocation = scalar:STRING,FULL_PATH_TO\data\Labels.txt
````

## Sample - FER+ Emotion Recognition

* Download the [FER+ Emotion Recognition](https://onnxzoo.blob.core.windows.net/models/opset_8/emotion_ferplus/emotion_ferplus.tar.gz) ONNX Model
* Use [Netron](https://lutzroeder.github.io/netron/) to open the model.onnx
* Look at Model Properties to find Input & Output Tensor Name (Input3 - input; Plus692_Output_0 - output)
* Look at output tensor dimensions (n,c,h,w - [1,8] for Plus692_Output_0)
* Use the label file - [data/emotions.txt](data/emotions.txt) to run sample

This sample is in [Graph Description Format](../../../utilities/runvx#amd-runvx) (gdf)

#### usage
````
runvx.exe -frames:LIVE winML-live-emotions.gdf
````

**NOTE:**
Make the below changes in the `winML-live-emotions.gdf` file to run the inference

* Add full path to the FER+ Emotion Recognition ONNX model downloaded in line 16
````
data modelLocation = scalar:STRING,FULL_PATH_TO\emotion_ferplus\model.onnx:view,inputImageWindow
````

* Add full path to the data\emotions.txt provided in this folder in line 25
````
data labelLocation = scalar:STRING,FULL_PATH_TO\data\emotions.txt
````

## Sample - VGG19

* Download the [VGG-19](https://s3.amazonaws.com/download.onnx/models/opset_8/vgg19.tar.gz) ONNX Model
* Use [Netron](https://lutzroeder.github.io/netron/) to open the model.onnx
* Look at Model Properties to find Input & Output Tensor Name (data_0 - input; prob_1 - output)
* Look at output tensor dimensions (n,c,h,w - [1,1000] for prob_1)
* Use the label file - [data/Labels.txt](data/Labels.txt) to run sample

This sample is in [Graph Description Format](../../../utilities/runvx#amd-runvx) (gdf)

#### usage
````
runvx.exe -v winML-image-vgg19.gdf
````

**NOTE:**
Make the below changes in the `winML-live-vgg19.gdf` file to run the inference

* Add full path to the data\bird.JPEG image provided in this folder in line 11
````
read input_image FULL_PATH_TO\data\bird.JPEG
````

* Add full path to the VGG 19 ONNX model downloaded in line 21
````
data modelLocation = scalar:STRING,FULL_PATH_TO\vgg19\model.onnx:view,resultWindow
````

* Add full path to the [data\Labels.txt](data\Labels.txt) provided in this folder in line 33
````
data labelLocation = scalar:STRING,FULL_PATH_TO\data\Labels.txt
````
````
8 changes: 8 additions & 0 deletions amd_openvx_extensions/amd_winml/samples/data/emotions.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
0,neutral
1,happiness
2,surprise
3,sadness
4,anger
5,disgust
6,fear
7,contempt
41 changes: 41 additions & 0 deletions amd_openvx_extensions/amd_winml/samples/winML-image-vgg19.gdf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Get WinML Extension DLL
import vx_winml

data result_image = image:640,150,RGB2
data input_image = image:224,224,RGB2
data input_tensor = tensor:4,{224,224,3,1},VX_TYPE_FLOAT32,0
data a = scalar:FLOAT32,1.0
data b = scalar:FLOAT32,0.0
data reverse_channel_order = scalar:BOOL,0

read input_image FULL_PATH_TO\data\bird.JPEG
view input_image inputImageWindow
view result_image resultWindow

# Use convert to tensor node - convert input image to tensor
node com.winml.convert_image_to_tensor input_image input_tensor a b reverse_channel_order

# write input tensor
write input_tensor input_tensor.f32

data modelLocation = scalar:STRING,FULL_PATH_TO\data\vgg19\model.onnx:view,resultWindow
data modelInputName = scalar:STRING,data_0
data modelOutputName = scalar:STRING,prob_1
data output = tensor:2,{1000,1},VX_TYPE_FLOAT32,0
data deviceKind = scalar:INT32,0

# Use Import ONNX Node to run Inference
node com.winml.onnx_to_mivisionx modelLocation modelInputName modelOutputName input_tensor output deviceKind

# write output tensor
write output output_tensor.f32

data labelLocation = scalar:STRING,FULL_PATH_TO\data\Labels.txt
data top1 = scalar:STRING,TOPK:view,resultWindow
data top2 = scalar:STRING,TOPK:view,resultWindow
data top3 = scalar:STRING,TOPK:view,resultWindow
data top4 = scalar:STRING,TOPK:view,resultWindow
data top5 = scalar:STRING,TOPK:view,resultWindow

# Use get top K label node to display results
node com.winml.get_top_k_labels output labelLocation top1 top2 top3 top4 top5
31 changes: 31 additions & 0 deletions amd_openvx_extensions/amd_winml/samples/winML-live-emotions.gdf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Get WinML Extension DLL
import vx_winml

# Get input from camera: one for display & another for input
data input_view = image:1080,720,RGB2:camera,0:view,inputImageWindow
data input_image = image:64,64,U008:camera,0

data input_tensor = tensor:4,{64,64,1,1},VX_TYPE_FLOAT32,0
data a = scalar:FLOAT32,1.0
data b = scalar:FLOAT32,0.0
data reverse_channel_order = scalar:BOOL,0

# Use convert to tensor node - convert input image to tensor
node com.winml.convert_image_to_tensor input_image input_tensor a b reverse_channel_order

data modelLocation = scalar:STRING,FULL_PATH_TO\emotion_ferplus\model.onnx:view,inputImageWindow
data modelInputName = scalar:STRING,Input3
data modelOutputName = scalar:STRING,Plus692_Output_0
data output = tensor:2,{8,1},VX_TYPE_FLOAT32,0
data deviceKind = scalar:INT32,0

# Use Import ONNX Node to run Inference
node com.winml.onnx_to_mivisionx modelLocation modelInputName modelOutputName input_tensor output deviceKind

data labelLocation = scalar:STRING,FULL_PATH_TO\data\emotions.txt
data top1 = scalar:STRING,INITIALIZE:view,inputImageWindow
data top2 = scalar:STRING,INITIALIZE:view,inputImageWindow
data top3 = scalar:STRING,INITIALIZE:view,inputImageWindow

# Use get top K label node to display results
node com.winml.get_top_k_labels output labelLocation top1 top2 top3