Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tkurth/rl ppo #12

Merged
merged 53 commits into from
Jun 12, 2024
Merged

Tkurth/rl ppo #12

merged 53 commits into from
Jun 12, 2024

Commits on Apr 25, 2024

  1. starting to implement rollout buffer

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    22991f1 View commit details
    Browse the repository at this point in the history

  2. added rollout buffer

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    a1c4a4c View commit details
    Browse the repository at this point in the history

  3. refactoring existing rl part in preparation for on_policy methods

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    fe69873 View commit details
    Browse the repository at this point in the history

  4. fixing small issues in ppo

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    e59520a View commit details
    Browse the repository at this point in the history

  5. fixing device assignment

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    e7364a6 View commit details
    Browse the repository at this point in the history

  6. fixing PPO

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    ed1dfd2 View commit details
    Browse the repository at this point in the history

  7. fixing PPO

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    114bc67 View commit details
    Browse the repository at this point in the history

  8. adding PPO algorithm to training

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    792f6e4 View commit details
    Browse the repository at this point in the history

  9. adding ppo.h

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    56e0aba View commit details
    Browse the repository at this point in the history

  10. exposing finalizing rollout buffer

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    7db9bf9 View commit details
    Browse the repository at this point in the history

  11. distributed advantage normalization

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    9c80bf4 View commit details
    Browse the repository at this point in the history

  12. finished implementing fortran wrappers for on policy systems

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    e8c943f View commit details
    Browse the repository at this point in the history

  13. enhancing ignre_tkr

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    fd6c24f View commit details
    Browse the repository at this point in the history

  14. more selektive ignore pragma

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    8f0dbf2 View commit details
    Browse the repository at this point in the history

  15. adding tkr

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    7bd6919 View commit details
    Browse the repository at this point in the history

  16. fixing device assignments

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    f9d315b View commit details
    Browse the repository at this point in the history

  17. fixing device assignment issues and memory leaks

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    b3dc323 View commit details
    Browse the repository at this point in the history

  18. fixed ppo as well

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    2f36d66 View commit details
    Browse the repository at this point in the history

  19. rebased onto new main

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    268b455 View commit details
    Browse the repository at this point in the history

  20. slight modularization

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    21afa0e View commit details
    Browse the repository at this point in the history

  21. changing update rollout buffer in a way, that q and log_p is computed… 

    … internally from the RL system and the user does not have to bother with it
    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    605d660 View commit details
    Browse the repository at this point in the history

  22. restructuring how rollout buffer works

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    09baa0b View commit details
    Browse the repository at this point in the history

  23. refactoring rollout buffer behavious

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    6f65d83 View commit details
    Browse the repository at this point in the history

  24. fixed device placement of ppo models

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    235f19c View commit details
    Browse the repository at this point in the history

  25. dbg

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    7133c0b View commit details
    Browse the repository at this point in the history

  26. removing debug prints

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    2d34ed4 View commit details
    Browse the repository at this point in the history

  27. fixing policy clipping/scaling in PPO and also making it more flexible

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    cc12ad0 View commit details
    Browse the repository at this point in the history

  28. fixing ordering in replay buffer and streamlining sampling

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    5d7321d View commit details
    Browse the repository at this point in the history

  29. adding linear LR

    @azrael417
    azrael417 committed Apr 25, 2024
    Configuration menu
    Copy the full SHA
    e072b71 View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2024

  1. fixing some of the logprob calculation

    @azrael417
    azrael417 committed Apr 26, 2024
    Configuration menu
    Copy the full SHA
    dd09a5f View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2024

  1. fixing problems in SAC and PPO where the policy checkpoint subfolder … 

    …was not created
    @azrael417
    azrael417 committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    1654432 View commit details
    Browse the repository at this point in the history

Commits on May 2, 2024

  1. adding system comm init to rl system

    @azrael417
    azrael417 committed May 2, 2024
    Configuration menu
    Copy the full SHA
    87afa8b View commit details
    Browse the repository at this point in the history

Commits on May 10, 2024

  1. slight rewrite

    @azrael417
    azrael417 committed May 10, 2024
    Configuration menu
    Copy the full SHA
    2d0919b View commit details
    Browse the repository at this point in the history

  2. simplifying td3 with shadow networks

    @azrael417
    azrael417 committed May 10, 2024
    Configuration menu
    Copy the full SHA
    14ab13e View commit details
    Browse the repository at this point in the history

Commits on May 14, 2024

  1. more specializations

    @azrael417
    azrael417 committed May 14, 2024
    Configuration menu
    Copy the full SHA
    64b627e View commit details
    Browse the repository at this point in the history

Commits on May 16, 2024

  1. adding gradient clipping to PPO

    @azrael417
    azrael417 committed May 16, 2024
    Configuration menu
    Copy the full SHA
    b6ab52f View commit details
    Browse the repository at this point in the history

  2. implementing joint actor-critic network for PPO

    @azrael417
    azrael417 committed May 16, 2024
    Configuration menu
    Copy the full SHA
    0dd888f View commit details
    Browse the repository at this point in the history

  3. fixing clipping

    @azrael417
    azrael417 committed May 16, 2024
    Configuration menu
    Copy the full SHA
    a0aa97e View commit details
    Browse the repository at this point in the history

  4. renamed action section to actor section

    @azrael417
    azrael417 committed May 16, 2024
    Configuration menu
    Copy the full SHA
    253228d View commit details
    Browse the repository at this point in the history

  5. some cleanup

    @azrael417
    azrael417 committed May 16, 2024
    Configuration menu
    Copy the full SHA
    d1db013 View commit details
    Browse the repository at this point in the history

Commits on May 21, 2024

  1. enabling distributed support for fancy quantities

    @azrael417
    azrael417 committed May 21, 2024
    Configuration menu
    Copy the full SHA
    45000f9 View commit details
    Browse the repository at this point in the history

Commits on May 28, 2024

  1. printing more detailed checkpoint save errrors

    @azrael417
    azrael417 committed May 28, 2024
    Configuration menu
    Copy the full SHA
    aa63110 View commit details
    Browse the repository at this point in the history

  2. more detailed errors

    @azrael417
    azrael417 committed May 28, 2024
    Configuration menu
    Copy the full SHA
    564afde View commit details
    Browse the repository at this point in the history

  3. adding kl divergence based step skipping

    @azrael417
    azrael417 committed May 28, 2024
    Configuration menu
    Copy the full SHA
    c67e30a View commit details
    Browse the repository at this point in the history

Commits on Jun 4, 2024

  1. removing debug prints

    @azrael417
    azrael417 committed Jun 4, 2024
    Configuration menu
    Copy the full SHA
    40b98ba View commit details
    Browse the repository at this point in the history

Commits on Jun 6, 2024

  1. adding global function to set manual seed

    @azrael417
    azrael417 committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    9955dff View commit details
    Browse the repository at this point in the history

  2. adding more support of more array shapes

    @azrael417
    azrael417 committed Jun 6, 2024
    Configuration menu
    Copy the full SHA
    a0692e1 View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2024

  1. first set of fixes

    @azrael417
    azrael417 committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    156af63 View commit details
    Browse the repository at this point in the history

  2. reformatting

    @azrael417
    azrael417 committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    46307b8 View commit details
    Browse the repository at this point in the history

  3. adding additional structure to docs

    @azrael417
    azrael417 committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    a26a27f View commit details
    Browse the repository at this point in the history

  4. updated documentation, now including the on-policy routines

    @azrael417
    azrael417 committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    7d2d8a8 View commit details
    Browse the repository at this point in the history

  5. update reset rollout buffer syntax

    @azrael417
    azrael417 committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    b08f560 View commit details
    Browse the repository at this point in the history

  6. cleaned up Dockerfile

    @azrael417
    azrael417 committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    9aa9c0c View commit details
    Browse the repository at this point in the history