[2/n] move autoscaling control to application state from deployment s… #57548
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request is a solid step in refactoring the autoscaling logic. The movement of the control loop to the application state is well-executed, and the introduction of ApplicationAutoscalingState provides a clear structure for future application-level autoscaling policies. The new tests are comprehensive and cover many edge cases, which is great to see.
I've identified a few areas for improvement:
- A bug in a test helper method that could lead to incorrect test behavior.
- An opportunity to simplify some logic using a dictionary comprehension for better readability.
- A docstring that could be updated to more accurately reflect the current implementation.
Overall, this is a high-quality contribution. Addressing these points will further enhance the clarity and robustness of the code.
abrarsheikh
commented
Oct 8, 2025
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Contributor
akyang-anyscale
left a comment
akyang-anyscale
left a comment
There was a problem hiding this comment.
the code lgtm, have not reviewed the tests yet
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Kishanthan
reviewed
Oct 10, 2025
arcyleung
reviewed
Oct 11, 2025
arcyleung
reviewed
Oct 11, 2025
arcyleung
reviewed
Oct 11, 2025
Contributor
|
LGTM. |
akyang-anyscale
approved these changes
Oct 14, 2025
zcin
approved these changes
Oct 14, 2025
arcyleung
approved these changes
Oct 14, 2025
abrarsheikh
force-pushed
the
SERVE-1216-abrar-auto
branch
from
dbf3f53 to
d7d25d2
Compare
Signed-off-by: abrar <abrar@anyscale.com>
abrarsheikh
force-pushed
the
SERVE-1216-abrar-auto
branch
from
d7d25d2 to
9c3b266
Compare
justinyeh1995
pushed a commit
to justinyeh1995/ray
that referenced
this pull request
Oct 20, 2025
ray-project#57548) part 2 of ray-project#56149, a significant portion of the code is taken from the original PR. This PR does not introduce any change in functionality. Autoscaling is still performed at the deployment level. This will help us make the transition towards application level autoscaling. The only change in this PR 1. is moving the autoscaling control loop from the deployment state to the application state. 2. adding application autoscaling state class, in the new design autoscaling state manager will manage a list of application autoscaling states and each application autoscaling state will manage a list of deployment autoscaling states Signed-off-by: abrar <abrar@anyscale.com>
xinyuangui2
pushed a commit
to xinyuangui2/ray
that referenced
this pull request
Oct 22, 2025
ray-project#57548) part 2 of ray-project#56149, a significant portion of the code is taken from the original PR. This PR does not introduce any change in functionality. Autoscaling is still performed at the deployment level. This will help us make the transition towards application level autoscaling. The only change in this PR 1. is moving the autoscaling control loop from the deployment state to the application state. 2. adding application autoscaling state class, in the new design autoscaling state manager will manage a list of application autoscaling states and each application autoscaling state will manage a list of deployment autoscaling states Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: xgui <xgui@anyscale.com>
elliot-barn
pushed a commit
that referenced
this pull request
Oct 23, 2025
#57548) part 2 of #56149, a significant portion of the code is taken from the original PR. This PR does not introduce any change in functionality. Autoscaling is still performed at the deployment level. This will help us make the transition towards application level autoscaling. The only change in this PR 1. is moving the autoscaling control loop from the deployment state to the application state. 2. adding application autoscaling state class, in the new design autoscaling state manager will manage a list of application autoscaling states and each application autoscaling state will manage a list of deployment autoscaling states Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>
landscapepainter
pushed a commit
to landscapepainter/ray
that referenced
this pull request
Nov 17, 2025
ray-project#57548) part 2 of ray-project#56149, a significant portion of the code is taken from the original PR. This PR does not introduce any change in functionality. Autoscaling is still performed at the deployment level. This will help us make the transition towards application level autoscaling. The only change in this PR 1. is moving the autoscaling control loop from the deployment state to the application state. 2. adding application autoscaling state class, in the new design autoscaling state manager will manage a list of application autoscaling states and each application autoscaling state will manage a list of deployment autoscaling states Signed-off-by: abrar <abrar@anyscale.com>
Aydin-ab
pushed a commit
to Aydin-ab/ray-aydin
that referenced
this pull request
Nov 19, 2025
ray-project#57548) part 2 of ray-project#56149, a significant portion of the code is taken from the original PR. This PR does not introduce any change in functionality. Autoscaling is still performed at the deployment level. This will help us make the transition towards application level autoscaling. The only change in this PR 1. is moving the autoscaling control loop from the deployment state to the application state. 2. adding application autoscaling state class, in the new design autoscaling state manager will manage a list of application autoscaling states and each application autoscaling state will manage a list of deployment autoscaling states Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: Aydin Abiar <aydin@anyscale.com>
Future-Outlier
pushed a commit
to Future-Outlier/ray
that referenced
this pull request
Dec 7, 2025
ray-project#57548) part 2 of ray-project#56149, a significant portion of the code is taken from the original PR. This PR does not introduce any change in functionality. Autoscaling is still performed at the deployment level. This will help us make the transition towards application level autoscaling. The only change in this PR 1. is moving the autoscaling control loop from the deployment state to the application state. 2. adding application autoscaling state class, in the new design autoscaling state manager will manage a list of application autoscaling states and each application autoscaling state will manage a list of deployment autoscaling states Signed-off-by: abrar <abrar@anyscale.com> Signed-off-by: Future-Outlier <eric901201@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
part 2 of #56149, a significant portion of the code is taken from the original PR.
This PR does not introduce any change in functionality. Autoscaling is still performed at the deployment level. This will help us make the transition towards application level autoscaling.
The only change in this PR