-
Notifications
You must be signed in to change notification settings - Fork 766
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Docathon][Add CN Doc No.60-61] #6211
Changes from 3 commits
99432d8
7e35172
f7ddfab
06390ac
2e75eb0
f9d6393
4d46496
119fb1c
1f0da34
0140fe1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
.. _cn_api_paddle_incubate_nn_functional_fused_layer_norm: | ||
|
||
fused_layer_norm | ||
------------------------------- | ||
|
||
.. py:function:: paddle.incubate.nn.functional.fused_layer_norm(x, norm_weight, norm_bias, epsilon, residual_alpha=1.0, begin_norm_axis=1, bias=None, residual=None, quant_scale=- 1, quant_round_type=0, quant_max_bound=0, quant_min_bound=0) | ||
|
||
应用 Fused LayerNorm 内核,提供了更高的 GPU 利用率。同时,支持模式融合 LayerNorm(bias + residual_alpha * residual + x)。 | ||
|
||
当 norm_weight 和 norm_bias 为 None 时,返回融合的 (bias + residual_alpha * residual + x)。 | ||
|
||
fused_layer_norm 算子目前只支持在 GPU 下运行。 | ||
|
||
参数 | ||
:::::::::::: | ||
- **x** (Tensor) - 输入 ``Tensor``。 | ||
- **norm_weight** (Tensor) - 用于仿射输出的权重张量。 | ||
- **norm_bias** (Tensor) - 用于仿射输出的偏置张量。 | ||
- **epsilon** (float) - 一个小的浮点数,用于避免除以零。 | ||
- **residual_alpha** (float) - 用于残差的缩放因子,默认为 1。 | ||
- **begin_norm_axis** (int) - 归一化的起始轴,默认为 1。 | ||
- **bias** (可选|Tensor) - 前一层的偏置,用于融合。 | ||
- **residual** (可选|Tensor) - 输入的残差,用于融合。 | ||
- **quant_scale** (float) - 量化比例。 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 量化缩放因子 |
||
- **quant_round_type** (float) - 量化四舍五入类型。 | ||
- **quant_max_bound** (float) - 量化的最大边界进行裁剪。 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 量化裁剪的最大/最小边界值,下同 |
||
- **quant_min_bound** (float) - 量化的最小边界进行裁剪。 | ||
|
||
|
||
返回 | ||
:::::::::::: | ||
输出``Tensor`` | ||
zade23 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
形状 | ||
:::::::::::: | ||
``Tensor``,形状同 ``x`` 一致。 | ||
|
||
代码示例 | ||
:::::::::::: | ||
|
||
COPY-FROM: paddle.incubate.nn.functional.fused_layer_norm |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
.. _cn_api_paddle_incubate_nn_functional_fused_rms_norm: | ||
|
||
fused_rms_norm | ||
------------------------------- | ||
|
||
.. py:function:: paddle.incubate.nn.functional.fused_rms_norm(x, norm_weight, norm_bias, epsilon, begin_norm_axis, bias=None, residual=None, quant_scale=- 1, quant_round_type=0, quant_max_bound=0, quant_min_bound=0) | ||
|
||
应用 Fused RMSNorm 内核,提供了更高的 GPU 利用率。同时,支持模式融合 RMSNorm(bias + residual + x)。 | ||
|
||
细节可参考论文 `Root Mean Square Layer Normalization <https://arxiv.org/abs/1910.07467>`_ 。 | ||
|
||
fused_rms_norm 算子目前只支持在 GPU 下运行, | ||
|
||
参数 | ||
:::::::::::: | ||
- **x** (Tensor) - 输入 ``Tensor``。 | ||
- **norm_weight** (Tensor) - 用于仿射输出的权重张量。 | ||
- **norm_bias** (Tensor) - 用于仿射输出的偏置张量。 | ||
- **epsilon** (float) - 一个小的浮点数,用于避免除以零。 | ||
- **begin_norm_axis** (int) - 归一化的起始轴,默认为 1。 | ||
- **bias** (可选|Tensor) - 前一层的偏置,用于融合。 | ||
- **residual** (可选|Tensor) - 输入的残差,用于融合。 | ||
- **quant_scale** (float) - 量化比例。 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 量化缩放因子 |
||
- **quant_round_type** (float) - 量化四舍五入类型。 | ||
- **quant_max_bound** (float) - 量化的最大边界进行裁剪。 | ||
- **quant_min_bound** (float) - 量化的最小边界进行裁剪。 | ||
|
||
|
||
返回 | ||
:::::::::::: | ||
输出``Tensor`` | ||
zade23 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
形状 | ||
:::::::::::: | ||
``Tensor``,形状同 ``x`` 一致。 | ||
|
||
代码示例 | ||
:::::::::::: | ||
|
||
COPY-FROM: paddle.incubate.nn.functional.fused_rms_norm |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
用于融合,这句可以都删了