Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[V1.x] ONNX support for Softmax -- optimize for axis=-1 case #19794

Merged
merged 3 commits into from
Jan 27, 2021

Conversation

Zha0q1
Copy link
Contributor

@Zha0q1 Zha0q1 commented Jan 26, 2021

Before this pr we would manually expand MXNet softmax. This was due to the mismatch between the configuration of mxnet and onnx softmax.

However we can adapt the onnx softmax to behave the same as mxnet softmax for the axis==-1 case. This is exactly how Gluon BERT uses softmax. With this optimization we 1) improve the runtime 2) avoid NaN issue caused by Exp when running inference with TensorRT in fp16 mode.

@Zha0q1 Zha0q1 requested a review from szha as a code owner January 26, 2021 23:13
@mxnet-bot
Copy link

Hey @Zha0q1 , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [miscellaneous, unix-cpu, centos-cpu, clang, website, unix-gpu, edge, sanity, windows-gpu, centos-gpu, windows-cpu]


Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jan 26, 2021
Copy link
Contributor

@josephevans josephevans left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a testcase for axis = -1?
nevermind, I see there already is a test case :)

@lanking520 lanking520 added pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-work-in-progress PR is still work in progress labels Jan 26, 2021
@lanking520 lanking520 added pr-awaiting-review PR is waiting for code review and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jan 27, 2021
Copy link
Contributor

@waytrue17 waytrue17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
pr-awaiting-review PR is waiting for code review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants