Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Priority-based exclusive placement #687

Merged
merged 2 commits into from
Oct 21, 2024
Merged

Priority-based exclusive placement #687

merged 2 commits into from
Oct 21, 2024

Conversation

ahg-g
Copy link
Contributor

@ahg-g ahg-g commented Oct 20, 2024

/kind feature

What this PR does / why we need it:

Limit exclusive placement to the same priority level.

Which issue(s) this PR fixes:

Fixes ##682

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Limit exclusive placement to jobs within the same priority level.

@k8s-ci-robot k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Oct 20, 2024
@k8s-ci-robot k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Oct 20, 2024
@netlify Netlify
Copy link

netlify bot commented Oct 20, 2024 •
edited
Loading

✅ Deploy Preview for kubernetes-sigs-jobset canceled.

Name Link
🔨 Latest commit 293a253
🔍 Latest deploy log https://app.netlify.com/sites/kubernetes-sigs-jobset/deploys/67158c44b32a760008aeda67

@ahg-g ahg-g force-pushed the exclusive-priority branch 4 times, most recently from d0d1a75 to b279a60 Compare October 20, 2024 02:58
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Oct 20, 2024
@ahg-g ahg-g force-pushed the exclusive-priority branch from b279a60 to fe31bec Compare October 20, 2024 03:06
@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Oct 20, 2024
@danielvegamyhre
Copy link
Contributor

By limiting exclusive placement to jobs within the same priority level, couldn't this cause partial scheduling of jobs with different priority levels onto the same TPU slice?

@ahg-g
Copy link
Contributor Author

ahg-g commented Oct 20, 2024 •
edited
Loading

By limiting exclusive placement to jobs within the same priority level, couldn't this cause partial scheduling of jobs with different priority levels onto the same TPU slice?

On the same slice, only one workload should be running, the assumption is that this workload will occupy all resources on that slice. If a race happens, then the higher priority slice will eventually preempt the lower priority one.

We should move to a proper API for exclusive placement in the next release, and have this configured as a knob, call it ExclusivityScope which defines the set of labels against which anti-affinity should be implemented.

@ahg-g
Copy link
Contributor Author

ahg-g commented Oct 20, 2024

@danielvegamyhre the test is failing due to golang version, I updated it in kubernetes/test-infra#33689, can you please take a look?

@danielvegamyhre
Copy link
Contributor

@danielvegamyhre the test is failing due to golang version, I updated it in kubernetes/test-infra#33689, can you please take a look?

Done

@ahg-g ahg-g force-pushed the exclusive-priority branch from fe31bec to 77c86ea Compare October 20, 2024 19:10
danielvegamyhre
danielvegamyhre reviewed Oct 20, 2024
View reviewed changes
Copy link
Contributor

@danielvegamyhre danielvegamyhre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall, just one minor comment

pkg/constants/constants.go Outdated Show resolved Hide resolved
@ahg-g @danielvegamyhre
Update pkg/constants/constants.go
293a253 
Co-authored-by: Daniel Vega-Myhre <105610547+danielvegamyhre@users.noreply.github.com>
@ahg-g ahg-g added the tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges. label Oct 20, 2024
@ahg-g
Copy link
Contributor Author

ahg-g commented Oct 20, 2024

LGTM overall, just one minor comment

done, thanks

@danielvegamyhre
Copy link
Contributor

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 21, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahg-g, danielvegamyhre

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [ahg-g,danielvegamyhre]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 17685c5 into kubernetes-sigs:main Oct 21, 2024
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@danielvegamyhre danielvegamyhre danielvegamyhre left review comments

@kannon92 kannon92 Awaiting requested review from kannon92

Assignees

@danielvegamyhre danielvegamyhre

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ahg-g @danielvegamyhre @k8s-ci-robot