[XPU] Add fast_layernorm + leaky_relu fusion #57113
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
你的PR提交成功，感谢你对开源项目的贡献! |
frank-oops
force-pushed
the
fln_act-1
branch
from
9181ab8 to
33d3a16
Compare
| @@ -291,6 +291,7 @@ if(WITH_XPU) | |||
| ${XPU_PASS_DEPS}) | |||
| pass_library(gather_squeeze_pass inference DIR xpu DEPS ${XPU_PASS_DEPS}) | |||
| pass_library(fast_where_xpu_fuse_pass inference DIR xpu DEPS ${XPU_PASS_DEPS}) | |||
| pass_library(fln_act_xpu_fuse_pass inference DIR xpu DEPS ${XPU_PASS_DEPS}) | |||
There was a problem hiding this comment.
layer_norm_act_xpu_fuse_pass
| @@ -0,0 +1,212 @@ | |||
| // Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved. | |||
| // | |||
| // Licensed under the Apache License, Version 2.0 (the "License"); | |||
There was a problem hiding this comment.
layer_norm_act_fuse_pass.cc
| After the pass is applied: | ||
| x | ||
| | | ||
| fln_act_xpu |
| graph: | ||
| x | ||
| | | ||
| layernorm_xpu |
| const std::string& name_scope, | ||
| const std::string& act_type); | ||
| // declare operator node's name | ||
| PATTERN_DECL_NODE(ln); |
| output | ||
| */ | ||
|
|
||
| struct FLNActXPUPattern : public PatternBase { |
paddle/phi/api/yaml/fused_ops.yaml
Outdated
| @@ -101,6 +101,15 @@ | |||
| data_type : x | |||
| optional : bias, x_max | |||
|
|
|||
| - op : fln_act_xpu | |||
MuShangCC
reviewed
Sep 13, 2023
| ->assert_is_op_input("layer_norm", "Scale"); | ||
| auto ln_out = pattern->NewNode(ln_out_repr()) | ||
| ->AsOutput() | ||
| ->assert_is_op_output("layer_norm", "Y"); |
There was a problem hiding this comment.
->assert_is_op_input(act_type_, "X"); 加到这里来
另外，需要判断 ln_out 只有一个 output
frank-oops
force-pushed
the
fln_act-1
branch
from
33d3a16 to
a73a054
Compare
paddle/phi/infermeta/fusion.cc
Outdated
| @@ -1014,6 +1014,20 @@ void AddCMulXPUInferMeta(const MetaTensor& x, | |||
| out->set_layout(x.layout()); | |||
| } | |||
|
|
|||
| void FLNActXPUInferMeta(const MetaTensor& x, | |||
There was a problem hiding this comment.
LayerNormActInferMeta
paddle/phi/infermeta/fusion.h
Outdated
| @@ -226,6 +226,15 @@ void AddCMulXPUInferMeta(const MetaTensor& x, | |||
| const MetaTensor& w, | |||
| MetaTensor* out); | |||
|
|
|||
| void FLNActXPUInferMeta(const MetaTensor& x, | |||
| namespace fusion { | ||
|
|
||
| template <typename T, typename Context> | ||
| void FLNActXPUKernel(const Context& ctx, |
| act.hard_sigmoid_slope = act_param; | ||
| } | ||
| #ifdef PADDLE_WITH_XPU_PLUGIN | ||
| int r = xpu::plugin::fln_act_fusion(ctx.x_context(), |
There was a problem hiding this comment.
fln 是否能统一调整为 layer_norm_act_fusion
frank-oops
force-pushed
the
fln_act-1
branch
from
a73a054 to
f9582fd
Compare
paddle/phi/api/yaml/fused_ops.yaml
Outdated
| args : (Tensor x, Tensor scale, Tensor bias, int begin_norm_axis, float epsilon, int act_type, float act_param) | ||
| output : Tensor(out) | ||
| infer_meta : | ||
| func : LayerNormActInferMeta |
There was a problem hiding this comment.
LayerNormActInferMeta 改成 LayerNormActXPUInferMeta
函数名称和算子名称保持一致，下同
| namespace patterns { | ||
|
|
||
| /* | ||
| change layernorm op to fast_layernorm op |
| @@ -70,6 +70,8 @@ XPUOpMap& get_kl2_ops() { | |||
| XPUKernelSet({phi::DataType::FLOAT32, phi::DataType::FLOAT16})}, | |||
| {"batch_norm", | |||
| XPUKernelSet({phi::DataType::FLOAT32, phi::DataType::FLOAT16})}, | |||
| {"layer_norm_act_xpu", | |||
There was a problem hiding this comment.
按照字母顺序？或者 fuse 类的算子放一起？
frank-oops
force-pushed
the
fln_act-1
branch
2 times, most recently
from
2305a00 to
cd13575
Compare
frank-oops
force-pushed
the
fln_act-1
branch
from
cd13575 to
d611343
Compare
hong19860320
changed the title
hong19860320
changed the title
Frida-a
pushed a commit
to Frida-a/Paddle
that referenced
this pull request
Oct 14, 2023
danleifeng
pushed a commit
to danleifeng/Paddle
that referenced
this pull request
Nov 14, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
Performance optimization
PR changes
OPs
Description
add fast_layternorm and act(leaky_relu) fusion