-
Notifications
You must be signed in to change notification settings - Fork 123
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
vine: check worker has committable resources #4039
Merged
btovar
merged 11 commits into
cooperative-computing-lab:master
from
JinZhou5042:worker_tag_has_free_resources
Feb 6, 2025
Merged
Changes from 9 commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
7a9de27
init
JinZhou5042 be82591
move empty libraries substract
JinZhou5042 06417c1
rename to is_resource_fully_allocated
JinZhou5042 6ec5688
handle has_cores && has_gpus
JinZhou5042 2baf535
Merge branch 'cooperative-computing-lab:master' into worker_tag_has_f…
JinZhou5042 bb34b00
check libraries
JinZhou5042 76421a2
simplify the code
JinZhou5042 dbf2360
remove from the data structure
JinZhou5042 8a5e07b
revert
JinZhou5042 41a2008
remove is_resource_committable
JinZhou5042 dc014c7
Merge branch 'cooperative-computing-lab:master' into worker_tag_has_f…
JinZhou5042 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -149,6 +149,46 @@ int check_worker_have_enough_disk_with_inputs(struct vine_manager *q, struct vin | |
return ok; | ||
} | ||
|
||
/* Check if any of the resources is defined and committable on a given worker. | ||
* @param q Manager info structure | ||
* @param resource @vine_resources.h:struct vine_resources | ||
* @return 1 if the resource type is defined and can be allocated to a task, 0 otherwise. | ||
*/ | ||
static int is_resource_committable(struct vine_manager *q, struct vine_resource resource) | ||
{ | ||
return resource.total > 0 && resource.inuse < overcommitted_resource_total(q, resource.total); | ||
} | ||
|
||
/* Check if this worker has committable resources for any type of task. | ||
* If it returns false, neither a function task, library task nor a regular task can run on this worker. | ||
* If it returns true, the worker has either free slots for function calls or sufficient resources for regular tasks. | ||
* @param q Manager info structure | ||
* @param w The worker info structure. | ||
*/ | ||
static int check_worker_have_committable_resources(struct vine_manager *q, struct vine_worker_info *w) | ||
{ | ||
/* If any slots are committable */ | ||
uint64_t task_id; | ||
struct vine_task *t; | ||
ITABLE_ITERATE(w->current_libraries, task_id, t) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe first check that |
||
{ | ||
if (t->function_slots_inuse < t->function_slots_total) { | ||
return 1; | ||
} | ||
} | ||
|
||
/* A regular task has to use both memory and disk */ | ||
if (is_resource_committable(q, w->resources->memory) && is_resource_committable(q, w->resources->disk)) { | ||
/* If either cores or gpus are committable. */ | ||
if (is_resource_committable(q, w->resources->cores) || is_resource_committable(q, w->resources->gpus)) { | ||
return 1; | ||
} | ||
} | ||
|
||
/* If reach here, no free resources on this worker */ | ||
return 0; | ||
} | ||
|
||
/* Check if this task is compatible with this given worker by considering | ||
* resources availability, features, blocklist, and all other relevant factors. | ||
* Used by all scheduling methods for basic compatibility. | ||
|
@@ -172,6 +212,7 @@ int check_worker_against_task(struct vine_manager *q, struct vine_worker_info *w | |
if (w->draining) { | ||
return 0; | ||
} | ||
|
||
// if worker's end time has not been received | ||
if (w->end_time < 0) { | ||
return 0; | ||
|
@@ -195,6 +236,11 @@ int check_worker_against_task(struct vine_manager *q, struct vine_worker_info *w | |
return 0; | ||
} | ||
|
||
/* if worker has free resources to use */ | ||
if (!check_worker_have_committable_resources(q, w)) { | ||
return 0; | ||
} | ||
|
||
/* Compute the resources to allocate to this task. */ | ||
struct rmsummary *l = vine_manager_choose_resources_for_task(q, w, t); | ||
|
||
|
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are unnecessarily copying the
struct vine_resource
data. In this case, I think it will be clearer if you do theresource.total > 0 && resource.inuse < overcommitted_resource_total(q, resource.total);
per resource insidecheck_worker_have_committable_resources
.