Make the Agent reset immediately after Done #3291
Conversation
| _AgentReset(); | ||
| m_RequestAction = false; | ||
| m_RequestDecision = false; | ||
| m_Reward = 0f; | ||
| m_CumulativeReward = 0f; |
There was a problem hiding this comment.
Feels like this could be moved into NotifyAgentDone() (or maybe combine Done and NotifyAgentDone, unless you don't want to the user to set maxStepReached)
There was a problem hiding this comment.
Moved some things around
| @@ -44,7 +44,7 @@ public struct AgentInfo | |||
| /// Unique identifier each agent receives at initialization. It is used | |||
| /// to separate between different agents in the environment. | |||
| /// </summary> | |||
| public int id; | |||
| public int episodeId; | |||
There was a problem hiding this comment.
Update this comment too.
| @@ -144,7 +144,7 @@ def add_experiences( | |||
| ) | |||
| for traj_queue in self.trajectory_queues: | |||
| traj_queue.put(trajectory) | |||
| self.experience_buffers[global_id] = [] | |||
| del self.experience_buffers[global_id] | |||
There was a problem hiding this comment.
Need to del last_step_result and last_take_action_outputs as well. Probably can do it right after the experience buffer del
There was a problem hiding this comment.
hmmm, unfortunately, it seems last_step_result as well as policy.previous_actions are modified after the check for done. (So even if I delete them, they will be re-added). I need to do more experiments...
| for terminated_id in terminated_agents: | ||
| self._clean_agent_data(terminated_id) | ||
|
|
||
| def _clean_agent_data(self, global_id: str) -> None: |
There was a problem hiding this comment.
@ervteng Tell me what you think
No description provided.