extend matmul_int8 and support matmul_with_flatten_int8 #56827
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
你的PR提交成功，感谢你对开源项目的贡献! |
|
|
|
✅ This PR's description meets the template requirements! |
| @@ -43,3 +43,9 @@ PD_REGISTER_KERNEL(matmul_with_flatten, | |||
| double, | |||
| phi::dtype::bfloat16, | |||
| phi::dtype::float16) {} | |||
|
|
|||
There was a problem hiding this comment.
期望不使用matmul_int8和matmul_with_flatten_int8独立于 matmul和matmul_with_flatten，而是通过增加数据类型的方式支持。之前的matmul_int8是一种非常临时的方式
There was a problem hiding this comment.
目前修改了这个部分，按照上述这个方式进行了实现。
chenwhql
approved these changes
Sep 13, 2023
|
|
||
| #if defined(PADDLE_WITH_CUDA) | ||
| #if CUDA_VERSION >= 11060 | ||
| REGISTER_OP_CUDA_KERNEL( |
phlrain
approved these changes
Sep 14, 2023
risemeup1
added a commit
that referenced
this pull request
Sep 14, 2023
…)" This reverts commit 5b6594c.
danleifeng
pushed a commit
to danleifeng/Paddle
that referenced
this pull request
Nov 14, 2023
…#56827) * add matmul_int8 and matmul_with_flatten_int8 * fix api of int8 matmul and matmul_with_flatten * support dy2static * fix static matmul build * code refine * add cuda version limitation in matmulflatten * avoid rocm build error * fix build error * fix build errors: auto_tune and blaslt_impl * fix inference ci error * fix multidefinition error * fix matmul with flatten error * fix unit test --------- Co-authored-by: RichardWooSJTU <37864677+RichardWooSJTU@users.noreply.github.com> Co-authored-by: wufeisheng <wfs1997@163.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
New features
PR changes
Ops
Description