Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Episodes support #11

Open
andreagentili opened this issue Mar 14, 2017 · 6 comments
Open

Episodes support #11

andreagentili opened this issue Mar 14, 2017 · 6 comments

Comments

@andreagentili
Copy link

andreagentili commented Mar 14, 2017

Is it possible to train this network with episodes? E.g. in Karpathy Waterworld, one can add a rule "the character dies if it eats a green pill". After death the "character" shall be revived, how do I tell the network to switch to a new episode? The same could be done for cars example, if a car "explodes" when it touches a wall.

Great lib anyway, and code is well written.

@janhuenermann
Copy link
Owner

Currently there is no support for episodes and I don't plan to add it. But I think, what you can maybe do, is try to just remove the character from the map and then put it in a kind of "death" state, which should lead to low reward. In this way, the character tries to avoid this state. I'll mark this issue as enhancement, so someone can implement this in the future. Maybe this helps.

@andreagentili
Copy link
Author

Thanks for your reply!
But I did't exactly get what you mean :) What do you mean by a "death state"?
Maybe:

  1. character executes agent's actions and hits a green ball
  2. agent gets a bad reward (-1000) and learns
  3. character and world is reset (e.g. position)
  4. goto 1

Is this it?

@janhuenermann
Copy link
Owner

Yes about right ;)

Instead of step 2 you could also just remove the agent and give it like -10 reward for a few timesteps (1-2 seconds, I don't know just guessing). But really, I never tried this and it's just a suggestion.

@js08
Copy link

js08 commented Mar 20, 2017

@janhuenermann can you tell the file name where we need to make the changes

@andreagentili
Copy link
Author

Fixed the link, was just missing the starting http://.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants