add new API paddle.nn.initializer.Orthogonal and calculate_gain #37163
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks for your contribution! |
zhwesky2010
force-pushed
the
add_orthogonal
branch
from
8ade9f2 to
74c368e
Compare
zhwesky2010
force-pushed
the
add_orthogonal
branch
from
74c368e to
7937e1b
Compare
zhwesky2010
force-pushed
the
add_orthogonal
branch
from
7937e1b to
c6a8bd2
Compare
jeff41404
reviewed
Nov 17, 2021
| param = 0.01 | ||
| else: | ||
| assert isinstance(param, (bool, int, float)) | ||
| recommended_gain = { |
There was a problem hiding this comment.
shall we support calculate gain of selu?
jeff41404
reviewed
Nov 17, 2021
| @@ -41,6 +41,13 @@ def output_hist(out): | |||
|
|
|||
|
|
|||
| class TestConstantInitializer(unittest.TestCase): | |||
| def test_calculate_gain(self): | |||
There was a problem hiding this comment.
shall we add test case of relu and selu?
zhwesky2010
force-pushed
the
add_orthogonal
branch
from
f03e982 to
bc959ac
Compare
TCChenlong
reviewed
Nov 18, 2021
python/paddle/fluid/initializer.py
Outdated
|
|
||
| Args: | ||
| nonlinearity: nonlinearity function. | ||
| param: optional parameter for somme nonlinearity function |
There was a problem hiding this comment.
param(type, optional);
default is None, means xxx
python/paddle/fluid/initializer.py
Outdated
|
|
||
| import paddle | ||
| gain = paddle.nn.initializer.calculate_gain('tanh') | ||
| initializer = paddle.nn.initializer.Orthogonal(gain) |
zhwesky2010
dismissed stale reviews from XiaoguangHu01 and jeff41404
via
6045fdc
zhwesky2010
force-pushed
the
add_orthogonal
branch
from
6045fdc to
9721672
Compare
zhwesky2010
force-pushed
the
add_orthogonal
branch
from
9721672 to
e1c11e3
Compare
zhwesky2010
force-pushed
the
add_orthogonal
branch
from
e1c11e3 to
06cfcdf
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
New features
PR changes
APIs
Describe
paddle.nn.initializer.Orthogonal
paddle.nn.initializer.calculate_gain