Skip to content

Commit

Permalink
[fix] fix the math style in paddle.static.nn.batch_norm (#58514)
Browse files Browse the repository at this point in the history
  • Loading branch information
ooooo-create authored Oct 31, 2023
1 parent 4cae876 commit 347a565
Showing 1 changed file with 12 additions and 12 deletions.
24 changes: 12 additions & 12 deletions python/paddle/static/nn/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -2803,19 +2803,19 @@ def batch_norm(
Internal Covariate Shift <https://arxiv.org/pdf/1502.03167.pdf>`_
for more details.
:math:input is the input features over a mini-batch.
:math:`input` is the input features over a mini-batch.
.. math::
\\mu_{\\beta} &\\gets \\frac{1}{m} \\sum_{i=1}^{m} x_i \\qquad &//\\
\ mini-batch\ mean \\\\
\\sigma_{\\beta}^{2} &\\gets \\frac{1}{m} \\sum_{i=1}^{m}(x_i - \\
\\mu_{\\beta})^2 \\qquad &//\ mini-batch\ variance \\\\
\\hat{x_i} &\\gets \\frac{x_i - \\mu_\\beta} {\\sqrt{\\
\\sigma_{\\beta}^{2} + \\epsilon}} \\qquad &//\ normalize \\\\
y_i &\\gets \\gamma \\hat{x_i} + \\beta \\qquad &//\ scale\ and\ shift
\mu_{\beta} &\gets \frac{1}{m} \sum_{i=1}^{m} x_i \qquad &//
\ mini-batch\ mean \\
\sigma_{\beta}^{2} &\gets \frac{1}{m} \sum_{i=1}^{m}(x_i -
\mu_{\\beta})^2 \qquad &//\ mini-batch\ variance \\
\hat{x_i} &\gets \frac{x_i - \mu_\beta} {\sqrt{
\sigma_{\beta}^{2} + \epsilon}} \qquad &//\ normalize \\
y_i &\gets \gamma \hat{x_i} + \beta \qquad &//\ scale\ and\ shift
moving\_mean = moving\_mean * momentum + mini-batch\_mean * (1. - momentum) \\\\
moving\_mean = moving\_mean * momentum + mini-batch\_mean * (1. - momentum) \\
moving\_var = moving\_var * momentum + mini-batch\_var * (1. - momentum)
Expand All @@ -2829,9 +2829,9 @@ def batch_norm(
.. math::
\\hat{x_i} &\\gets \\frac{x_i - \\mu_\\beta} {\\sqrt{\\
\\sigma_{\\beta}^{2} + \\epsilon}} \\\\
y_i &\\gets \\gamma \\hat{x_i} + \\beta
\hat{x_i} &\gets \frac{x_i - \mu_\beta} {\sqrt{
\sigma_{\beta}^{2} + \epsilon}} \\
y_i &\gets \gamma \hat{x_i} + \beta
Note:
if build_strategy.sync_batch_norm=True, the batch_norm in network will use
Expand Down

0 comments on commit 347a565

Please sign in to comment.