Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SkyServe] Check positive for target_qps_per_replica #2995

Merged
merged 1 commit into from
Jan 18, 2024

Conversation

cblmemo
Copy link
Collaborator

@cblmemo cblmemo commented Jan 17, 2024

Our autoscaling metric target_qps_per_replica should be a positive number. This PR adds a check on the constraint.

Tested (run the relevant ones):

  • Code formatting: bash format.sh
  • Any manual or new tests for this PR (please specify below)
# neg-tar-qps.yaml
service:
  readiness_probe: /
  replica_policy:
    min_replicas: 2
    max_replicas: 10
    target_qps_per_replica: -2.5 # also tested with 0

resources:
  ports: 5000
$ sky serve up neg-tar-qps.yaml
Service from YAML spec: neg-tar-qps.yaml
ValueError: target_qps_per_replica must be greater than 0
  • All smoke tests: pytest tests/test_smoke.py
  • Relevant individual smoke tests: pytest tests/test_smoke.py::test_fill_in_the_name
  • Backward compatibility tests: bash tests/backward_comaptibility_tests.sh

Copy link
Collaborator

@MaoZiming MaoZiming left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cblmemo cblmemo merged commit f0bb6ba into master Jan 18, 2024
19 checks passed
@cblmemo cblmemo deleted the serve-check-target-qps branch January 18, 2024 01:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants