Reinforcement Learning Examples

[ozanarkancan]

Hi,

I am starting this pull request for reinforcement learning examples. I have implemented followings:

• Value Iteration, Policy Evaluation, Policy Iteration
• REINFORCE for both discrete and continuous action spaces

My TODO list

• DQN
• Actor Critic

[CarloLucibello]

o["render"] && render(env) for conciseness?

[CarloLucibello]

done && break

[CarloLucibello]

predict_probs or probs or predict instead of predict_linear?

[CarloLucibello]

randn(size(μ)) instead of randn()

[CarloLucibello]

also use .+ and .*

[CarloLucibello]

the package Gym should be submitted to METADATA before merging this

[CarloLucibello]

I coded here some actor-critic examples https://github.com/CarloLucibello/DeepRLexamples.jl, feel free to utilize any piece of code in case you find it of some use

[ereday]

There is a typo in all examples (like this and this) . I guess pkb should be pkg

[ozanarkancan]

fixed

[ereday]

Now, actor_critic_discrete.jl complains about Gym initialization.

ERROR: LoadError: InitError: PyError (ccall(@pysym(:PyImport_ImportModule), PyPtr, (Cstring,), name)
Stacktrace:
 [1] pyerr_check at /home/erenay/.julia/v0.6/PyCall/src/exception.jl:56 [inlined]
 [2] pyerr_check at /home/erenay/.julia/v0.6/PyCall/src/exception.jl:61 [inlined]
 [3] macro expansion at /home/erenay/.julia/v0.6/PyCall/src/exception.jl:81 [inlined]
 [4] pyimport(::String) at /home/erenay/.julia/v0.6/PyCall/src/PyCall.jl:374
 [5] __init__() at /home/erenay/.julia/v0.6/Gym/src/Gym.jl:10
 [6] _include_from_serialized(::String) at ./loading.jl:157
 [7] _require_from_serialized(::Int64, ::Symbol, ::String, ::Bool) at ./loading.jl:200
 [8] _require(::Symbol) at ./loading.jl:498
 [9] require(::Symbol) at ./loading.jl:405
 [10] include_from_node1(::String) at ./loading.jl:576
 [11] include(::String) at ./sysimg.jl:14
 [12] process_options(::Base.JLOptions) at ./client.jl:305
 [13] _start() at ./client.jl:371
during initialization of module Gym
while loading /home/erenay/.julia/v0.6/Knet/examples/actor_critic_discrete.jl, in expression starting on line 22

[ereday]

It works now. @denizyuret examples work. It can be merged.

[denizyuret]

• Notebooks get too big with plots, in Knet I am committing versions of all notebooks with text outputs but without plots (2.5M -> 28K). I did a fresh copy into master as we don't want the 2M files in git history either. (Another alternative is to save with plots under its own package in KnetML). Please continue development under current master.
• In the dp notebook GYM_ENVS should be in quotes -- fixed.
• In the dp notebook I get a "WARNING: special characters "#{}()[]<>|&*?~;" should now be quoted in commands" during Pkg.build("Gym"), don't know if it hurts anything.
• In the dp notebook env = GymEnv("FrozenLake-v0") gives error 'No module named scipy.stats', is there any way scipy can be automatically installed during Pkg.build? Please test with fresh python without preinstalled packages.
• In reinforce I get the following in the last box: 'WARN: gym.spaces.Box autodetected dtype as <type 'numpy.float32'>. Please provide explicit dtype.', is this something we can fix?
• I haven't been able to run dqn yet, it hangs at DQN.main("--help"), maybe I am not giving it enough memory?

[ozanarkancan]

• I guess "WARNING: special characters "#{}()[]<>|&*?~;" is related with a julia issue
• scipy is in the requirements of gym but somehow it is not triggered during the installation. I will investigate it.
• 'WARN: gym.spaces.Box autodetected dtype as <type 'numpy.float32'> is coming from gym even if you use it in python.
• The last one might be a notebook issue, julia dqn.jl --printinfo should work.

[denizyuret]

OK, please check the dqn notebook, I can run all the others.

…

On Sun, Mar 18, 2018 at 9:10 PM Ozan Arkan Can ***@***.***> wrote: - I guess "WARNING: special characters "#{}()[]<>|&*?~;" is related with a julia issue <JuliaLang/julia#20482> - scipy is in the requirements of gym but somehow it is not triggered during the installation. I will investigate it. - 'WARN: gym.spaces.Box autodetected dtype as <type 'numpy.float32'> is coming from gym even if you use it in python. - The last one might be a notebook issue, julia dqn.jl --printinfo should work. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#284 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABvNpp5INLGNxnzCdb21jorgTvDr--UDks5tfqL-gaJpZM4SmJ3_> .