Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Deploy] Try to convert the gpu_topology value type to int. #2132

Merged
merged 1 commit into from
May 29, 2024
Merged

[Deploy] Try to convert the gpu_topology value type to int. #2132

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion python/fedml/computing/scheduler/model_scheduler/device_replica_controller.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,9 @@ def __init__(self, master_id, request_json: dict):
def calc_total_gpu_num(self):
total_gpu_num = 0
for device_id, gpu_num in self.devices_avail_gpus.items():
total_gpu_num += gpu_num
if type(gpu_num) is not int:
logging.warning(f"The value in gpu_topology should be int, but got {type(gpu_num)}. Try to convert it.")
total_gpu_num += int(gpu_num)
return total_gpu_num

def init_id_replica_num(self):
Expand All @@ -77,6 +79,11 @@ def init_id_replica_num(self):
"""
id_replica_num = {}
for id, avail_num in self.devices_avail_gpus.items():
if type(avail_num) is not int:
logging.warning(f"The value in gpu_topology should be int, "
f"but got {type(avail_num)}. Try to convert it.")
avail_num = int(avail_num)

if avail_num % self.gpu_per_replica != 0:
raise ValueError("The number of gpus for each device should be divisible by gpu_per_replica")
id_replica_num[str(id)] = avail_num // self.gpu_per_replica
Expand Down
Loading