Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RLlib] Cast types for new configurations in PB2. #42181

Conversation

simonsays1980
Copy link
Collaborator

@simonsays1980 simonsays1980 commented Jan 4, 2024

Fixed a bug in generating a new config in 'PB2'. The type for floats were actually 'numpy.dtype.float64' and resulted so far in error in 'ray.rllib.utils.schedule.Scheduler.validate()' where this type is not allowed for the learning rate,

Why are these changes needed?

PB2 as one of Ray major hyperparameter tuning algorithms for ML and specifically RL was erroring out when tuning the learning rate. This was due to the fact that ray.rllib.utils.schedule.Scheduler.validate is allowing only specific types for the learning rate. Whenever PB2 created a new config it assigned the values received from ray.tune.schedulers.pb2._explore and that was type numpy.dtype.float64. So, basically it was not possible to tune the learning parameter in experiments.

This PR fixes this shortcoming and casts types from a new config explored via Bayesian Optimization to the ones from the old config. Like this configs will keep types over the course of an experiment and PB2does not error out anymore.

Related issue number

Closes #42180

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

…were actually 'numpy.dtype.float64' and resulted so far in error in 'ray.rllib.utils.schedule.Scheduler.validate()' where this type is not allowed for the learning rate,

Signed-off-by: Simon Zehnder <simon.zehnder@gmail.com>
@simonsays1980 simonsays1980 marked this pull request as ready for review January 4, 2024 16:22
@sven1977 sven1977 self-assigned this Jan 4, 2024
@sven1977 sven1977 changed the title Cast types for new configurations in PB2. [RLlib] Cast types for new configurations in PB2. Jan 4, 2024
Signed-off-by: Sven Mika <sven@anyscale.io>
Copy link
Contributor

@sven1977 sven1977 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (added one minor change); Thanks for this fix @simonsays1980 !

@sven1977
Copy link
Contributor

sven1977 commented Jan 4, 2024

Looks great. Waiting for tests to pass, then merge.

@sven1977 sven1977 merged commit 2b57ee9 into ray-project:master Jan 4, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Tune|RLlib] PB2 scheduler runs into error when learning rate is used as hyperparameter.
2 participants