replace the AdagradOptimizer 、adamaxOptimizer、AdadeltaOptimizer、RMSPropOptimizer、LambOptimizer and Momentum #54152

longranger2 · 2023-05-28T09:46:46Z

PR types

Others

PR changes

APIs

Description

remove the AdagradOptimizer in paddle/fluid/optimizer.py and use paddle/optimizer/adagrad.py replace it
remove the AdamaxOptimizer in paddle/fluid/optimizer.py and use paddle/optimizer/adamax.py replace it
remove the AdadeltaOptimizer in paddle/fluid/optimizer.py and use paddle/optimizer/adadelta.py to replace it.
remove the RMSPropOptimizer in paddle/fluid/optimizer.py and use paddle/optimizer/rmsprop.py to replace it.
remove the LambOptimizer in paddle/fluid/optimizer.py and use paddle/optimizer/lamb.py to replace it.
remove the Momentum in paddle/fluid/contrib/optimizer.py and use python/paddle/optimizer/momentum.py to replace it.

…paddle/optimizer/momentum.py

paddle-bot · 2023-05-28T09:46:50Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

paddle-bot · 2023-05-28T09:46:53Z

✅ This PR's description meets the template requirements!
Please wait for other CI results.

… Adadelta

paddle-ci-bot · 2023-06-28T03:14:56Z

Sorry to inform you that 26a1caf's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

…for AdagradOptimizer

zoooo0820 · 2023-06-30T03:58:43Z

test/legacy_test/test_optimizer.py

@@ -249,7 +249,7 @@ def test_nesterov_momentum_optimizer(self):


 class TestAdagradOptimizer(unittest.TestCase):
-    class MockAdagrad(optimizer.AdagradOptimizer):


这个单测测试的均为fluid.Optimizer，实际上2.0的Optimizer已有自己的单测test_xxx_api / test_xxx_op ，这里建议对应删除对应的测试case，而非将测旧优化器的单测改为测新优化器

下同

已修改👌

zoooo0820 · 2023-07-03T03:05:39Z

test/legacy_test/test_adadelta_op.py

@@ -410,9 +410,9 @@ def dygraph_adadelta_mp(self, use_amp, mp):
        paddle.set_device('gpu')
        input = paddle.randn((2, 2))
        model = paddle.nn.Linear(2, 2)
-        optimizer = paddle.fluid.optimizer.Adadelta(


在这几个test_xxx_op.py的单测文件中，目前多数同时存在例如TestAdadeltaMultiPrecision1_0 & TestAdadeltaMultiPrecision2_0 分别对paddle.fluid.Optimizer和paddle.Optimizer进行测试。

目前的改法是将1.0版本的测试替换成2.0，和原2.0版本的测试差别仅是初始化数值上的的差异。考虑到并不能覆盖到更多场景，同时单测的类名将有一定迷惑性（命名1_0，但是测2.0的API），建议把这部分单测都删掉

zoooo0820 · 2023-07-03T03:06:15Z

test/legacy_test/test_adagrad_op.py

@@ -408,7 +408,7 @@ def static_adagrad_mp(self, use_amp, mp):
        exe = paddle.static.Executor('gpu')
        train_program = paddle.static.Program()
        startup_program = paddle.static.Program()
-        optimizer = paddle.fluid.optimizer.Adagrad(learning_rate=0.001)


同test_adadelta_op.py，建议移除单测

zoooo0820 · 2023-07-03T03:06:43Z

test/legacy_test/test_adamax_op.py

@@ -404,8 +404,8 @@ def dygraph_adamax_mp(self, use_amp, mp):
        paddle.set_device('gpu')
        input = paddle.randn((2, 2))
        model = paddle.nn.Linear(2, 2)
-        optimizer = paddle.fluid.optimizer.Adamax(
-            learning_rate=0.001, parameter_list=model.parameters()
+        optimizer = paddle.optimizer.Adamax(


同test_adadelta_op.py，建议移除单测

… Adadelta

longranger2 · 2023-07-05T02:30:47Z

已修改👌

longranger2 · 2023-07-11T01:03:04Z

@zoooo0820 辛苦review下～

zoooo0820

LGTM

jeff41404

LGTM

risemeup1

LGTM

XiaoguangHu01

LGTM

…opOptimizer、LambOptimizer and Momentum (PaddlePaddle#54152) * replace the AdadeltaOptimizer with Adadelta * replace the RMSPropOptimizer with RMSProp * replace the LambOptimizer with lamb * replace the momentum in contrib/optimizer.py with Momentum in python/paddle/optimizer/momentum.py * fix bug * fix bug * fix bug * fix bug of Lamp * fix bug of Lamp * fix bug of import * replace the AdamaxOptimizer with Admax and change the optimizer base for AdagradOptimizer * fix bug * fix bug * Update optimizer.py * fix bug * fix bug

longranger2 added 4 commits May 28, 2023 09:34

replace the AdadeltaOptimizer with Adadelta

557f46f

replace the RMSPropOptimizer with RMSProp

593bb04

replace the LambOptimizer with lamb

997b737

replace the momentum in contrib/optimizer.py with Momentum in python/…

ba2ffdf

…paddle/optimizer/momentum.py

paddle-bot bot added contributor External developers status: proposed labels May 28, 2023

longranger2 added 4 commits May 29, 2023 16:12

fix bug

72e3904

Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…

588acff

… Adadelta

fix bug

cf06d26

fix bug

8a12c26

longranger2 changed the title ~~replace the AdadeltaOptimizer、RMSPropOptimizer、LambOptimizer and Momentum~~ replace the AdadeltaOptimizer、RMSPropOptimizer、LambOptimizer、AdamOptimizer and Momentum Jun 9, 2023

longranger2 added 3 commits June 16, 2023 10:44

fix bug of Lamp

6b9f9f9

fix bug of Lamp

f327122

fix bug of import

26a1caf

Merge branch 'develop' into Adadelta

092ba82

longranger2 changed the title ~~replace the AdadeltaOptimizer、RMSPropOptimizer、LambOptimizer、AdamOptimizer and Momentum~~ replace the AdagradOptimizer 、adamaxOptimizer、AdadeltaOptimizer、RMSPropOptimizer、LambOptimizer and Momentum Jun 29, 2023

replace the AdamaxOptimizer with Admax and change the optimizer base …

e51daa0

…for AdagradOptimizer

zoooo0820 reviewed Jun 30, 2023

View reviewed changes

longranger2 added 3 commits June 30, 2023 13:27

fix bug

934f19c

fix bug

8547661

Update optimizer.py

17b9be7

zoooo0820 reviewed Jul 3, 2023

View reviewed changes

longranger2 added 2 commits July 5, 2023 02:27

fix bug

86d06de

Merge branch 'Adadelta' of https://github.com/longranger2/Paddle into…

1a43987

… Adadelta

longranger2 requested a review from zoooo0820 July 5, 2023 02:31

fix bug

28d1afa

fix conflict

f874c1a

zoooo0820 approved these changes Jul 11, 2023

View reviewed changes

jeff41404 approved these changes Jul 11, 2023

View reviewed changes

risemeup1 approved these changes Jul 11, 2023

View reviewed changes

luotao1 approved these changes Jul 11, 2023

View reviewed changes

YuanRisheng approved these changes Jul 11, 2023

View reviewed changes

XiaoguangHu01 approved these changes Jul 11, 2023

View reviewed changes

jeff41404 merged commit 9436585 into PaddlePaddle:develop Jul 11, 2023

USTCKAY mentioned this pull request Jul 12, 2023

[NPU]fix uts PaddlePaddle/PaddleCustomDevice#679

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

replace the AdagradOptimizer 、adamaxOptimizer、AdadeltaOptimizer、RMSPropOptimizer、LambOptimizer and Momentum #54152

replace the AdagradOptimizer 、adamaxOptimizer、AdadeltaOptimizer、RMSPropOptimizer、LambOptimizer and Momentum #54152

longranger2 commented May 28, 2023 •

edited

Loading

paddle-bot bot commented May 28, 2023

paddle-bot bot commented May 28, 2023 •

edited

Loading

paddle-ci-bot bot commented Jun 28, 2023

zoooo0820 Jun 30, 2023

longranger2 Jul 3, 2023 •

edited

Loading

zoooo0820 Jul 3, 2023

zoooo0820 Jul 3, 2023

zoooo0820 Jul 3, 2023

longranger2 commented Jul 5, 2023

longranger2 commented Jul 11, 2023 •

edited

Loading

zoooo0820 left a comment

jeff41404 left a comment

risemeup1 left a comment

XiaoguangHu01 left a comment

		@@ -249,7 +249,7 @@ def test_nesterov_momentum_optimizer(self):


		class TestAdagradOptimizer(unittest.TestCase):
		class MockAdagrad(optimizer.AdagradOptimizer):

replace the AdagradOptimizer 、adamaxOptimizer、AdadeltaOptimizer、RMSPropOptimizer、LambOptimizer and Momentum #54152

replace the AdagradOptimizer 、adamaxOptimizer、AdadeltaOptimizer、RMSPropOptimizer、LambOptimizer and Momentum #54152

Conversation

longranger2 commented May 28, 2023 • edited Loading

PR types

PR changes

Description

paddle-bot bot commented May 28, 2023

paddle-bot bot commented May 28, 2023 • edited Loading

paddle-ci-bot bot commented Jun 28, 2023

zoooo0820 Jun 30, 2023

Choose a reason for hiding this comment

longranger2 Jul 3, 2023 • edited Loading

Choose a reason for hiding this comment

zoooo0820 Jul 3, 2023

Choose a reason for hiding this comment

zoooo0820 Jul 3, 2023

Choose a reason for hiding this comment

zoooo0820 Jul 3, 2023

Choose a reason for hiding this comment

longranger2 commented Jul 5, 2023

longranger2 commented Jul 11, 2023 • edited Loading

zoooo0820 left a comment

Choose a reason for hiding this comment

jeff41404 left a comment

Choose a reason for hiding this comment

risemeup1 left a comment

Choose a reason for hiding this comment

XiaoguangHu01 left a comment

Choose a reason for hiding this comment

longranger2 commented May 28, 2023 •

edited

Loading

paddle-bot bot commented May 28, 2023 •

edited

Loading

longranger2 Jul 3, 2023 •

edited

Loading

longranger2 commented Jul 11, 2023 •

edited

Loading