[v1.x] Onnx Support for Dropout #19837
Conversation
|
Hey @Zha0q1 , Thanks for submitting the PR
CI supported jobs: [windows-cpu, unix-cpu, unix-gpu, website, windows-gpu, centos-gpu, sanity, centos-cpu, miscellaneous, edge, clang] Note: |
lanking520
added
pr-awaiting-testing
| return nodes | ||
| else: | ||
| return [make_node("Dropout", input_nodes, [name], ratio=probability, name=name)] | ||
| if axes != [None]: |
There was a problem hiding this comment.
Do you need to check for string value 'None'?
There was a problem hiding this comment.
convert_string_to_list() will return [None] if input is 'None'
| @@ -1125,16 +1125,19 @@ def convert_dropout(node, **kwargs): | |||
| opset_version = kwargs["opset_version"] | |||
|
|
|||
| probability = float(attrs.get("p", 0.5)) | |||
| axes = convert_string_to_list((attrs.get("axes", "None"))) | |||
lanking520
added
pr-awaiting-testing
lanking520
added
pr-awaiting-testing
lanking520
added
pr-awaiting-testing
| return nodes | ||
| else: | ||
| return [make_node("Dropout", input_nodes, [name], ratio=probability, name=name)] | ||
| if mode != 'training': |
There was a problem hiding this comment.
Do we want to check and preserve the Dropout nodes if mode is training? Otherwise if inference mode, return identity op.
There was a problem hiding this comment.
I think dropout is almost never used in inference because the output values will be randomly dropped and theoretically it's designed for training only. I think not supporting 'both' mode, at least for now, can confirm this while we export the models in the model zoo.
There was a problem hiding this comment.
Sorry, just trying to understand the mode parameter and how it applies to these training-only ops. Is mode set to 'training' in the models or are you just depending on the default from attrs.get() above?
Just wondering for future plans to export models for training (if needed.) Do we need to specify whether we are exporting for training or for inference in the export call, or can we make it operator dependent?
There was a problem hiding this comment.
Feel free to merge, we should just continue this conversation for future uses.
There was a problem hiding this comment.
Sorry, just trying to understand the mode parameter and how it applies to these training-only ops. Is mode set to 'training' in the models or are you just depending on the default from attrs.get() above?
Just wondering for future plans to export models for training (if needed.) Do we need to specify whether we are exporting for training or for inference in the export call, or can we make it operator dependent?
mode is a parameter to MXNet operator Dropout only; it's now a parameter for the model :). I think since we targeting inference we might be able to set it to Identity for now
lanking520
added
pr-awaiting-review
For inference we can just use
IdentityforDropout