Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(src/layer): add vision_transformer benchmark #3730

Merged
merged 21 commits into from
May 17, 2022

Conversation

tpoisonooo
Copy link
Contributor

@tpoisonooo tpoisonooo commented Apr 24, 2022

支持 vision_transformer。

  • onnx2ncnn.cpp 增加了转换 GELU 和 Crop
  • MultiHeadAttenction opr 加了句注释,冷门 opr 不说来源真的扛不住...
  • benchmark 增加了转好的 ViT.param 。完整的模型已经测过数值对的

速度测试 on 8 核 i5 CPU,330MB 大模型。

vision_transformer min = 3236.20 max = 3541.62 avg = 3403.61

@codecov-commenter
Copy link

codecov-commenter commented Apr 24, 2022

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.35%. Comparing base (bf1109c) to head (6b5b942).
Report is 789 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #3730      +/-   ##
==========================================
+ Coverage   93.33%   93.35%   +0.01%     
==========================================
  Files         672      674       +2     
  Lines      171577   171980     +403     
==========================================
+ Hits       160146   160556     +410     
+ Misses      11431    11424       -7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@tpoisonooo tpoisonooo changed the title [WIP] feat(src/layer): add vision_transformer benchmark and relative layer feat(src/layer): add vision_transformer benchmark and relative layer Apr 25, 2022
@lgtm-com
Copy link

lgtm-com bot commented Apr 25, 2022

This pull request introduces 1 alert when merging eb7d4b2 into f14388a - view on LGTM.com

new alerts:

  • 1 for Multiplication result converted to larger type

@tpoisonooo tpoisonooo changed the title feat(src/layer): add vision_transformer benchmark and relative layer [WIP] feat(src/layer): add vision_transformer benchmark and relative layer Apr 25, 2022
@tpoisonooo tpoisonooo changed the title [WIP] feat(src/layer): add vision_transformer benchmark and relative layer feat(src/layer): add vision_transformer benchmark and relative layer Apr 26, 2022
@tpoisonooo
Copy link
Contributor Author

@nihui test_squeezenet 的错,是要我重造模型? 我也没改涉及 sqz 的东西。

@tpoisonooo
Copy link
Contributor Author

@nihui 我没有要改的了。

@tpoisonooo
Copy link
Contributor Author

@nihui 好人给 review 一下呗。

@nihui
Copy link
Member

nihui commented May 6, 2022

InnerProduct     MatMul_28                1 1 onnx::MatMul_180 onnx::Transpose_182 0=3072 1=0 2=2359296
Permute          Transpose_29             1 1 onnx::Transpose_182 onnx::Reshape_183 0=1
Reshape          Reshape_31               1 1 onnx::Reshape_183 onnx::Add_185 0=1 1=-1 2=3072
BinaryOp         Add_32                   2 1 onnx::Add_185 onnx::Add_653 onnx::Reshape_188 0=0
Reshape          Reshape_34               1 1 onnx::Reshape_188 onnx::Transpose_190 0=-1 1=3072
Permute          Transpose_35             1 1 onnx::Transpose_190 mmdeploy::Gelu_191 0=1

这套东西,其实就是一个 nn.Linear ,被onnx拆散了 😆

@tpoisonooo
Copy link
Contributor Author

@nihui 改好啦,扔掉了 gather; ViT.param 也小了很多。

@tpoisonooo tpoisonooo changed the title feat(src/layer): add vision_transformer benchmark and relative layer feat(src/layer): add vision_transformer benchmark May 16, 2022
@nihui nihui merged commit 6fd801b into Tencent:master May 17, 2022
@nihui
Copy link
Member

nihui commented May 17, 2022

Thanks for your contribution !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants