Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Paddle-TRT] TRT inference support for BERT/Transformer in paddle 2.0 api #31744

Merged
merged 5 commits into from
Mar 30, 2021
Merged

[Paddle-TRT] TRT inference support for BERT/Transformer in paddle 2.0 api #31744

merged 5 commits into from
Mar 30, 2021

Conversation

cryoco
Copy link
Contributor

@cryoco cryoco commented Mar 19, 2021 •
edited
Loading

PR types

Performance optimization

PR changes

Others

Describe

  1. add multihead_matmul_fuse_pass_v3 for multihead attention pattern in 2.0
  2. embedding_elementwise_layernorm_fuse_pass support lookup_table_v2
  3. FFN support FC in matmul op format
  4. support varlen feature in 2.0 model
  5. support 3 * 2 shape in map_matmul_to_mul_pass

@paddle-bot-old
Copy link

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@cryoco cryoco changed the title bert trt support matmul qkv [Paddle-TRT] BERT trt support matmul qkv Mar 22, 2021
@cryoco cryoco force-pushed the multihead-fuse-pass-support-matmul branch from cd87b90 to 23b221f Compare March 22, 2021 08:05
@cryoco cryoco changed the title [Paddle-TRT] BERT trt support matmul qkv [Paddle-TRT] TRT inference support for BERT in paddle 2.0 Mar 24, 2021
@cryoco cryoco changed the title [Paddle-TRT] TRT inference support for BERT in paddle 2.0 [Paddle-TRT] TRT inference support for BERT in paddle 2.0 api Mar 24, 2021
@cryoco cryoco changed the title [Paddle-TRT] TRT inference support for BERT in paddle 2.0 api [Paddle-TRT] TRT inference support for BERT/Transformer in paddle 2.0 api Mar 30, 2021
shangzhizhou
shangzhizhou approved these changes Mar 30, 2021
View reviewed changes
Copy link
Member

@shangzhizhou shangzhizhou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cryoco cryoco merged commit 14b7e3c into PaddlePaddle:develop Mar 30, 2021
@cryoco cryoco deleted the multihead-fuse-pass-support-matmul branch March 30, 2021 07:59
cryoco added a commit to cryoco/Paddle that referenced this pull request Mar 30, 2021
@cryoco
[Paddle-TRT] TRT inference support for BERT/Transformer in paddle 2.0…
309efda 
… api (PaddlePaddle#31744)

* support multihead_matmul_fuse_pass_v3

* fix compile problems

* embedding_eltwise_ln pass support lookup_table_v2

* suppoort matmul and matmul_v2 in qkv matmul
@cryoco cryoco mentioned this pull request Mar 30, 2021
Merged
Superjomn pushed a commit that referenced this pull request Mar 31, 2021
@cryoco
Cherry pick bert transformer 2.0 support (#31959)
967f4c2 
* [Paddle-TRT] TRT inference support for BERT/Transformer in paddle 2.0 api (#31744)

* support multihead_matmul_fuse_pass_v3

* fix compile problems

* embedding_eltwise_ln pass support lookup_table_v2

* suppoort matmul and matmul_v2 in qkv matmul

* map_matmul_to_mul_pass support 3dim
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shangzhizhou shangzhizhou shangzhizhou approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@cryoco @shangzhizhou