feature(nyz): add PPOF new interface support #567

PaParaZz1 · 2023-01-02T15:17:42Z

Description

from ding.bonus import PPOF

agent = PPOF(env='lunarlander_discrete', exp_name='lunarlander_discrete_demo')
agent.train(step=int(1e5))
agent.deploy(enable_save_replay=True)

Related Issue

TODO

Check List

merge the latest version source branch/repo, and resolve all the conflicts
pass style check
pass all the tests

puyuan1996 · 2023-01-03T03:39:37Z

dizoo/rocket/README.md

+# Install
+
+```shell
+pip install git+https://github.com/nighood/rocket-recycling@master#egg=rocket_recycling


是否可以把这个修改版的 rocket-recycling 移到 opendilab 下面？

这个暂时不用吧，等稳定版

puyuan1996 · 2023-01-03T03:42:33Z

ding/rl_utils/sampler.py

+import treetensor.torch as ttorch
+from torch.distributions import Normal, Independent
+
+


增加 Overview 注释？

puyuan1996 · 2023-01-03T03:45:17Z

ding/policy/ppof.py

+        deterministic_eval=True,
+        # model
+        model=dict(),
+    )


增加 overview 介绍以及 ppof 与 ppo 的区别，介绍为什么需要单独写一个 ppof.py

ding/policy/common_utils.py

ding/envs/env_manager/subprocess_env_manager.py

ding/bonus/ppof.py

puyuan1996 · 2023-01-03T03:56:40Z

ding/bonus/ppof.py

+            task.run()
+
+    def deploy(self, ckpt_path: str = None, enable_save_replay: bool = False, debug: bool = False) -> None:
+        if debug:


deploy 与 batch_evaluate 的异同是？是否可以添加一下注释

puyuan1996 · 2023-01-03T04:00:23Z

ding/bonus/ppof.py

+        'drone_fly',
+        'hybrid_moving',
+    ]
+


添加 bonus 路径下的文件说明？简介这里的ppof与policy/下面的ppof的异同？

ding/bonus/ppof.py

codecov · 2023-01-03T11:54:22Z

Codecov Report

Merging #567 (cabd45b) into main (0a25e46) will decrease coverage by 1.13%.
The diff coverage is 33.93%.

❗ Current head cabd45b differs from pull request most recent head 93ddd76. Consider uploading reports for the commit 93ddd76 to get more accurate results

@@            Coverage Diff             @@
##             main     #567      +/-   ##
==========================================
- Coverage   84.59%   83.45%   -1.14%     
==========================================
  Files         555      562       +7     
  Lines       45197    45489     +292     
==========================================
- Hits        38233    37964     -269     
- Misses       6964     7525     +561

Flag	Coverage Δ
unittests	`83.45% <33.93%> (-1.14%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
ding/bonus/__init__.py	`0.00% <0.00%> (ø)`
ding/bonus/config.py	`0.00% <0.00%> (ø)`
ding/bonus/model.py	`0.00% <0.00%> (ø)`
ding/bonus/ppof.py	`0.00% <0.00%> (ø)`
ding/config/__init__.py	`100.00% <ø> (ø)`
ding/envs/env_manager/subprocess_env_manager.py	`74.47% <0.00%> (-1.48%)`	⬇️
ding/model/common/__init__.py	`100.00% <ø> (ø)`
ding/framework/middleware/functional/evaluator.py	`42.66% <4.16%> (-10.71%)`	⬇️
...ework/middleware/functional/advantage_estimator.py	`78.26% <12.50%> (-13.85%)`	⬇️
ding/policy/ppof.py	`16.47% <16.47%> (ø)`
... and 237 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

feature(nyz): add basic ppof in discrete/continuous action space

5d7d9e8

PaParaZz1 added enhancement New feature or request algo Add new algorithm or improve old one labels Jan 2, 2023

PaParaZz1 mentioned this pull request Jan 2, 2023

Roadmap for DI-engine #548

Open

PaParaZz1 added 4 commits January 3, 2023 00:22

feature(nyz): add ppof hybrid implementation

96d7d16

demo(nyz): add ppof rocket landing demo

2dd108d

fix(nyz): fix compatibility bugs

3b549a3

demo(nyz): add drone fly demo

8eaf2fd

puyuan1996 reviewed Jan 3, 2023

View reviewed changes

ding/bonus/ppof.py Outdated Show resolved Hide resolved

PaParaZz1 added 2 commits January 3, 2023 16:55

fix(nyz): fix typo and add more comments

5bece63

fix(nyz): fix CI and demo bugs

649ed77

PaParaZz1 and others added 3 commits January 3, 2023 20:01

fix(nyz): fix rocket context bug

f0bf6d0

Merge branch 'main' into dev-ppof

cabd45b

style(nyz): fix flake8 style

93ddd76

PaParaZz1 merged commit 886285d into main Jan 3, 2023

PaParaZz1 deleted the dev-ppof branch January 3, 2023 12:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feature(nyz): add PPOF new interface support #567

feature(nyz): add PPOF new interface support #567

PaParaZz1 commented Jan 2, 2023

puyuan1996 Jan 3, 2023

PaParaZz1 Jan 3, 2023

puyuan1996 Jan 3, 2023

puyuan1996 Jan 3, 2023

puyuan1996 Jan 3, 2023

puyuan1996 Jan 3, 2023

codecov bot commented Jan 3, 2023 •

edited

Loading

		import treetensor.torch as ttorch
		from torch.distributions import Normal, Independent

feature(nyz): add PPOF new interface support #567

feature(nyz): add PPOF new interface support #567

Conversation

PaParaZz1 commented Jan 2, 2023

Description

Related Issue

TODO

Check List

puyuan1996 Jan 3, 2023

Choose a reason for hiding this comment

PaParaZz1 Jan 3, 2023

Choose a reason for hiding this comment

puyuan1996 Jan 3, 2023

Choose a reason for hiding this comment

puyuan1996 Jan 3, 2023

Choose a reason for hiding this comment

puyuan1996 Jan 3, 2023

Choose a reason for hiding this comment

puyuan1996 Jan 3, 2023

Choose a reason for hiding this comment

codecov bot commented Jan 3, 2023 • edited Loading

Codecov Report

codecov bot commented Jan 3, 2023 •

edited

Loading