[rllib] Add support for vectorize modes in SingleAgentEnvRunner.make_env
#58410
+167
−64
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Mark Towers <mark@anyscale.com>
pseudo-rnd-thoughts
added
rllib
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request adds support for pre-vectorized environments in SingleAgentEnvRunner by utilizing gymnasium.VectorizeMode.VECTOR_ENTRY_POINT. The changes in SingleAgentEnvRunner.make_env are well-structured to handle different environment registration types and correctly pass configurations. The accompanying tests are thorough, especially the new test_vectorize_mode and test_env_context, and the improvements to test_sample make it more robust.
I've identified a few instances where the dictionary union operator (|) is used, which is incompatible with Python 3.8, a supported version for Ray. I've left comments with suggestions for backward-compatible alternatives. Apart from that, the changes look solid.
8 tasks
added 4 commits
November 5, 2025 15:11
Signed-off-by: Mark Towers <mark@anyscale.com>
Signed-off-by: Mark Towers <mark@anyscale.com>
Signed-off-by: Mark Towers <mark@anyscale.com>
Signed-off-by: Mark Towers <mark@anyscale.com>
simonsays1980
approved these changes
Nov 14, 2025
Contributor
simonsays1980
left a comment
simonsays1980
left a comment
There was a problem hiding this comment.
LGTM. Thanks for the tuning @pseudo-rnd-thoughts !
added 4 commits
November 17, 2025 11:59
# Conflicts: # rllib/env/tests/test_single_agent_env_runner.py
Signed-off-by: Mark Towers <mark@anyscale.com>
Signed-off-by: Mark Towers <mark@anyscale.com>
Signed-off-by: Mark Towers <mark@anyscale.com>
Aydin-ab
pushed a commit
to Aydin-ab/ray-aydin
that referenced
this pull request
Nov 19, 2025
…_env` (ray-project#58410) ## Description Allow users to use environments that are already vectorized for `SingleAgentEnvRunner` With `gymnasium.make_vec`, users have the option to either use the `SyncVectorEnv` to vectorize a base environment or to directly create a vector environment using the `vectorize_mode: gymnasium.VectorizeMode`. This PR utilises the `env_runners(gym_env_vectorize_mode=...)` argument to support `VectorizeMode.VECTOR_ENTRY_POINT` ``` import gymnasium as gym config = ... config.env_runners( gym_env_vectorize_mode=gym.VectorizeMode.VECTOR_ENTRY_POINT, ) ``` An important change related to this PR is that the values accepted for the vectorize mode is either the enum (`VectorizedMode.ASYNC`, etc) or the enum values (`"async"`, etc) as before it was the string version was the enum name (`"ASYNC"`) rather than the enum value itself. ## Related issues Completion of ray-project#57643, ## Additional information ray-project#58397 must be merged first We should apply a similar change to the `MultiAgentEnvRunner.make_env` --------- Signed-off-by: Mark Towers <mark@anyscale.com> Co-authored-by: Mark Towers <mark@anyscale.com> Signed-off-by: Aydin Abiar <aydin@anyscale.com>
ykdojo
pushed a commit
to ykdojo/ray
that referenced
this pull request
Nov 27, 2025
…_env` (ray-project#58410) ## Description Allow users to use environments that are already vectorized for `SingleAgentEnvRunner` With `gymnasium.make_vec`, users have the option to either use the `SyncVectorEnv` to vectorize a base environment or to directly create a vector environment using the `vectorize_mode: gymnasium.VectorizeMode`. This PR utilises the `env_runners(gym_env_vectorize_mode=...)` argument to support `VectorizeMode.VECTOR_ENTRY_POINT` ``` import gymnasium as gym config = ... config.env_runners( gym_env_vectorize_mode=gym.VectorizeMode.VECTOR_ENTRY_POINT, ) ``` An important change related to this PR is that the values accepted for the vectorize mode is either the enum (`VectorizedMode.ASYNC`, etc) or the enum values (`"async"`, etc) as before it was the string version was the enum name (`"ASYNC"`) rather than the enum value itself. ## Related issues Completion of ray-project#57643, ## Additional information ray-project#58397 must be merged first We should apply a similar change to the `MultiAgentEnvRunner.make_env` --------- Signed-off-by: Mark Towers <mark@anyscale.com> Co-authored-by: Mark Towers <mark@anyscale.com> Signed-off-by: YK <1811651+ykdojo@users.noreply.github.com>
SheldonTsen
pushed a commit
to SheldonTsen/ray
that referenced
this pull request
Dec 1, 2025
…_env` (ray-project#58410) ## Description Allow users to use environments that are already vectorized for `SingleAgentEnvRunner` With `gymnasium.make_vec`, users have the option to either use the `SyncVectorEnv` to vectorize a base environment or to directly create a vector environment using the `vectorize_mode: gymnasium.VectorizeMode`. This PR utilises the `env_runners(gym_env_vectorize_mode=...)` argument to support `VectorizeMode.VECTOR_ENTRY_POINT` ``` import gymnasium as gym config = ... config.env_runners( gym_env_vectorize_mode=gym.VectorizeMode.VECTOR_ENTRY_POINT, ) ``` An important change related to this PR is that the values accepted for the vectorize mode is either the enum (`VectorizedMode.ASYNC`, etc) or the enum values (`"async"`, etc) as before it was the string version was the enum name (`"ASYNC"`) rather than the enum value itself. ## Related issues Completion of ray-project#57643, ## Additional information ray-project#58397 must be merged first We should apply a similar change to the `MultiAgentEnvRunner.make_env` --------- Signed-off-by: Mark Towers <mark@anyscale.com> Co-authored-by: Mark Towers <mark@anyscale.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Allow users to use environments that are already vectorized for
SingleAgentEnvRunnerWith
gymnasium.make_vec, users have the option to either use theSyncVectorEnvto vectorize a base environment or to directly create a vector environment using thevectorize_mode: gymnasium.VectorizeMode.This PR utilises the
env_runners(gym_env_vectorize_mode=...)argument to supportVectorizeMode.VECTOR_ENTRY_POINTAn important change related to this PR is that the values accepted for the vectorize mode is either the enum (
VectorizedMode.ASYNC, etc) or the enum values ("async", etc) as before it was the string version was the enum name ("ASYNC") rather than the enum value itself.Related issues
Completion of #57643,
Additional information
#58397 must be merged first
We should apply a similar change to the
MultiAgentEnvRunner.make_env