Nystromformer ONNX export

[whr778]

This PR implements ONNX export functionality for Nystromformer models.

Fixes # (727)

In addition to running the test cases, I exported a locally built Nystromformer 2048 input sequence model to ONNX and ran prediction. I verified the prediction output.

I decided to use the uw-madison/nystromformer-1024 for testing and not the hf-internal-testing/tiny-random-NystromformerModel... as the inputs are wrong in the hf-internal-testing model.

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[fxmarty]

Thank you for your contribution, very clean! Just to understand, what is the issue with hf-internal-testing/tiny-random-NystromformerModel with the ONNX Runtime integration?

[whr778]

I ran the onnx test suite and was getting an error that the inputs were incorrect. I went back and verified that the coded inputs were correct, and then changed the model to the reference model from UW, and the test suite passed with no issues. I did not investigate the hf-internal-testing/tiny-random-NystromformerModel any further. I concluded there must be an issue with how it was created.

[fxmarty]

I tested locally with "nystromformer": "hf-internal-testing/tiny-random-NystromformerModel" and pytest tests/onnxruntime/test_modeling.py -k "nystromformer" --exitfirst -s and the tests pass.

Could you have a try again with the tiny model? It avoids to slow down too much the tests.

[whr778]

Sure. I'll do that right now and push a change if it passes.

…

On Tue, Jan 31, 2023 at 4:23 AM fxmarty ***@***.***> wrote: I tested locally with "nystromformer": "hf-internal-testing/tiny-random-NystromformerModel" and pytest tests/onnxruntime/test_modeling.py -k "nystromformer" --exitfirst -s and the tests pass. Could you have a try again with the tiny model? It avoids to slow down too much the tests. — Reply to this email directly, view it on GitHub <#728 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNKCQ7UB23F23YIEHS2AJLWVDK2PANCNFSM6AAAAAAUJYFJNE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[whr778]

I cleaned out all of my pycache files etc ```find . | grep -E "(__pycache__|\.pyc|\.pyo$)" | xargs rm -rf``` and the test ran fine. I'll push a change and notify you when it's done.

…

On Tue, Jan 31, 2023 at 6:17 AM William Roe ***@***.***> wrote: Sure. I'll do that right now and push a change if it passes. On Tue, Jan 31, 2023 at 4:23 AM fxmarty ***@***.***> wrote: > I tested locally with "nystromformer": > "hf-internal-testing/tiny-random-NystromformerModel" and pytest > tests/onnxruntime/test_modeling.py -k "nystromformer" --exitfirst -s and > the tests pass. > > Could you have a try again with the tiny model? It avoids to slow down > too much the tests. > > — > Reply to this email directly, view it on GitHub > <#728 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ABNKCQ7UB23F23YIEHS2AJLWVDK2PANCNFSM6AAAAAAUJYFJNE> > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> >

[whr778]

@fxmarty changes are pushed. Thanks for the help!

[fxmarty]

Thanks great! The git history seems broken (see the diff), could you try to fix it? Not sure what's wrong, I already had the issue using git rebase.

Maybe git merge main would do (from upstream main).

[whr778]

@fxmarty I think I have the git history fixed. I also ran make style. What appears to have happened is I inadvertently hit the sync fork button on the forked optimum repository web page.
I fixed it with a git merge upstream/main. E.g.

 1039  make style
 1040  git status
 1041  history | grep git
 1042  git fetch upstream
 1043  git merge upstream/main
 1044  git status
 1045  git push -u origin onnx-export-nystromformer

Sorry about the issues.

[fxmarty]

It's all good now, the failing test is unrelated. Thanks for the addition!