Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RLlib] Chaining Models in RLModules #31469

Merged
merged 52 commits into from
Feb 7, 2023
Merged

[RLlib] Chaining Models in RLModules #31469

merged 52 commits into from
Feb 7, 2023

Commits on Jan 24, 2023

  1. initial 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    b1ddd73 View commit details
    Browse the repository at this point in the history

  2. tests complete 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    5489c89 View commit details
    Browse the repository at this point in the history

  3. wip 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    0dc91e2 View commit details
    Browse the repository at this point in the history

  4. wip 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    a9a9498 View commit details
    Browse the repository at this point in the history

  5. mutually exclusive encoders, tests passing 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    9930d79 View commit details
    Browse the repository at this point in the history

  6. add lstm code 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    109e56d View commit details
    Browse the repository at this point in the history

  7. better docs for get expected model config 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    532c8c0 View commit details
    Browse the repository at this point in the history

  8. kourosh's comments 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    31ae2a0 View commit details
    Browse the repository at this point in the history

  9. lstm fixed, tests working 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    018e223 View commit details
    Browse the repository at this point in the history

  10. add state out 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    b38d501 View commit details
    Browse the repository at this point in the history

  11. add __main__ to test 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    462fc4d View commit details
    Browse the repository at this point in the history

  12. change lstm testing according to kourosh's comment 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    f93795a View commit details
    Browse the repository at this point in the history

  13. fix get_initial_state 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    6ae3443 View commit details
    Browse the repository at this point in the history

  14. remove useless forward_exploration/forward_inference branch 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    548f42a View commit details
    Browse the repository at this point in the history

  15. revert changes to test_ppo_with_rl_module.py 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    92ae510 View commit details
    Browse the repository at this point in the history

  16. remove pass 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    04228db View commit details
    Browse the repository at this point in the history

  17. fix gym incompatability 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    0ab7809 View commit details
    Browse the repository at this point in the history

  18. add missing ray. 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    0b7536c View commit details
    Browse the repository at this point in the history

  19. test_ppo_rl_module working 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    77f991a View commit details
    Browse the repository at this point in the history

  20. ppo_torch_rl_module tests working 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    096a612 View commit details
    Browse the repository at this point in the history

  21. feedback from kourosh from last week 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    a373538 View commit details
    Browse the repository at this point in the history

  22. solution 3 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    c2ba97c View commit details
    Browse the repository at this point in the history

  23. some larger refactors 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    6ef042e View commit details
    Browse the repository at this point in the history

  24. rename PPOModuleConfig for upcoming release 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    d8d8c72 View commit details
    Browse the repository at this point in the history

  25. cleanup 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    503b7dd View commit details
    Browse the repository at this point in the history

  26. rename configs 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    3c30b42 View commit details
    Browse the repository at this point in the history

  27. remove rebase artifacts 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    5687454 View commit details
    Browse the repository at this point in the history

  28. minor fixes 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    bec7f47 View commit details
    Browse the repository at this point in the history

  29. fix import 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    da887b6 View commit details
    Browse the repository at this point in the history

  30. Merge remote-tracking branch 'upstream/master' into solution2

    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    dbd7c0a View commit details
    Browse the repository at this point in the history

  31. fix misspelling 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    42e0d49 View commit details
    Browse the repository at this point in the history

  32. fix import 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    aef9875 View commit details
    Browse the repository at this point in the history

  33. also fix other imports 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    d0d5277 View commit details
    Browse the repository at this point in the history

  34. typo 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    3011262 View commit details
    Browse the repository at this point in the history

  35. delete unneeded configs 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    3591fab View commit details
    Browse the repository at this point in the history

  36. remove unneeded model configs 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 24, 2023
    Configuration menu
    Copy the full SHA
    ab2c301 View commit details
    Browse the repository at this point in the history

Commits on Jan 26, 2023

  1. sven's nits 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 26, 2023
    Configuration menu
    Copy the full SHA
    e7aa528 View commit details
    Browse the repository at this point in the history

  2. sven's nits 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 26, 2023
    Configuration menu
    Copy the full SHA
    2d709d5 View commit details
    Browse the repository at this point in the history

  3. more refactors 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 26, 2023
    Configuration menu
    Copy the full SHA
    871fe75 View commit details
    Browse the repository at this point in the history

  4. another nit 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 26, 2023
    Configuration menu
    Copy the full SHA
    0bafdd7 View commit details
    Browse the repository at this point in the history

  5. some more renaming 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 26, 2023
    Configuration menu
    Copy the full SHA
    69d9655 View commit details
    Browse the repository at this point in the history

  6. delete vectorencoder 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 26, 2023
    Configuration menu
    Copy the full SHA
    bd687ab View commit details
    Browse the repository at this point in the history

  7. add back torch folder 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 26, 2023
    Configuration menu
    Copy the full SHA
    6085e05 View commit details
    Browse the repository at this point in the history

Commits on Jan 27, 2023

  1. fix model names and some nits 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 27, 2023
    Configuration menu
    Copy the full SHA
    f46b569 View commit details
    Browse the repository at this point in the history

  2. renaming and lint 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 27, 2023
    Configuration menu
    Copy the full SHA
    868d442 View commit details
    Browse the repository at this point in the history

  3. self-review 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 27, 2023
    Configuration menu
    Copy the full SHA
    8326719 View commit details
    Browse the repository at this point in the history

  4. output activations 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 27, 2023
    Configuration menu
    Copy the full SHA
    cede51f View commit details
    Browse the repository at this point in the history

  5. remove useless init and add comment to torch encoder 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 27, 2023
    Configuration menu
    Copy the full SHA
    8bc5c08 View commit details
    Browse the repository at this point in the history

  6. remove useless constructor 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 27, 2023
    Configuration menu
    Copy the full SHA
    7846d1b View commit details
    Browse the repository at this point in the history

Commits on Jan 30, 2023

  1. lint 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 30, 2023
    Configuration menu
    Copy the full SHA
    d2b4aee View commit details
    Browse the repository at this point in the history

Commits on Jan 31, 2023

  1. kourohs's nits 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 31, 2023
    Configuration menu
    Copy the full SHA
    166e4eb View commit details
    Browse the repository at this point in the history

  2. unify torch + tf 

    Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com>
    @ArturNiederfahrenhorst
    ArturNiederfahrenhorst committed Jan 31, 2023
    Configuration menu
    Copy the full SHA
    c70224f View commit details
    Browse the repository at this point in the history