Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: KeyError: 'reward' #24

Closed
4 tasks done
hlwang98 opened this issue May 4, 2023 · 0 comments · Fixed by #27
Closed
4 tasks done

[Bug]: KeyError: 'reward' #24

hlwang98 opened this issue May 4, 2023 · 0 comments · Fixed by #27
Labels
bug Something isn't working

Comments

@hlwang98
Copy link

hlwang98 commented May 4, 2023

🐛 Bug

base_reward.py中 rewards = data["reward"].copy(),没有reward,reward_wrapper.py是rewards,修改之后还有其他问题。。。
image
image

To Reproduce

# train_ppo.py
from openrl.envs.common import make
from openrl.modules.common import PPONet as Net
from openrl.runners.common import PPOAgent as Agent

env = make("CartPole-v1", env_num=9) # 创建环境,并设置环境并行数为9
net = Net(env) # 创建神经网络
agent = Agent(net) # 初始化训练器
agent.train(total_time_steps=20000) # 开始训练,并设置环境运行总步数为20000

Relevant log output / Error message

Traceback (most recent call last):
  File "/home/user/code/python/train_ppo.py", line 9, in <module>
    agent.train(total_time_steps=20000) # 开始训练,并设置环境运行总步数为20000
  File "/home/user/anaconda3/envs/OpenRL/lib/python3.9/site-packages/openrl/runners/common/ppo_agent.py", line 134, in train
    driver.run()
  File "/home/user/anaconda3/envs/OpenRL/lib/python3.9/site-packages/openrl/drivers/onpolicy_driver.py", line 227, in run
    self._inner_loop()
  File "/home/user/anaconda3/envs/OpenRL/lib/python3.9/site-packages/openrl/drivers/onpolicy_driver.py", line 112, in _inner_loop
    rollout_infos = self.actor_rollout()
  File "/home/user/anaconda3/envs/OpenRL/lib/python3.9/site-packages/openrl/drivers/onpolicy_driver.py", line 189, in actor_rollout
    obs, rewards, dones, infos = self.envs.step(actions, extra_data)
  File "/home/user/anaconda3/envs/OpenRL/lib/python3.9/site-packages/openrl/envs/vec_env/wrappers/vec_monitor_wrapper.py", line 37, in step
    returns = self.env.step(action, extra_data)
  File "/home/user/anaconda3/envs/OpenRL/lib/python3.9/site-packages/openrl/envs/vec_env/wrappers/reward_wrapper.py", line 46, in step
    rewards, new_infos = self.reward_class.step_reward(data=extra_data)
  File "/home/user/anaconda3/envs/OpenRL/lib/python3.9/site-packages/openrl/rewards/base_reward.py", line 18, in step_reward
    rewards = data["reward"].copy()
KeyError: 'reward'

System Info

No response

Checklist

  • I have checked that there is no similar issues in the repo
  • I have read the documentation
  • I have provided a minimal working example to reproduce the bug
  • I have version numbers, operating system and environment, where applicable
@hlwang98 hlwang98 added the bug Something isn't working label May 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant