Skip to content

[FIX] Fix high GPU memory usage #351

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
rapids-bot merged 5 commits into from
Sep 10, 2025
Merged

[FIX] Fix high GPU memory usage #351

rapids-bot merged 5 commits into from
Sep 10, 2025

Conversation

@aliceb-nv
Copy link
Contributor

@aliceb-nv aliceb-nv commented Aug 26, 2025 •
edited
Loading

closes #349

compute_related_variables was heuristically allocating memory based on A100/H100 with >=40GB of VRAM.
This is now automatically adjusted based on the total VRAM of the device

A command-line option has also been added to solve_MPS_file to specify device memory allocation limits for ease of testing.

@aliceb-nv aliceb-nv added non-breaking Introduces a non-breaking change improvement Improves an existing functionality labels Aug 26, 2025
@copy-pr-bot
Copy link

copy-pr-bot bot commented Aug 26, 2025

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@aliceb-nv
Copy link
Contributor Author

aliceb-nv commented Aug 26, 2025

/ok to test a8ebd6a

@aliceb-nv
Copy link
Contributor Author

aliceb-nv commented Aug 26, 2025

/ok to test 94d4f96

rg20
rg20 reviewed Aug 26, 2025
View reviewed changes
cpp/src/utilities/cuda_helpers.cuh
{
// Otherwise, we need to get the free memory from the device
size_t free_mem, total_mem;
cudaMemGetInfo(&free_mem, &total_mem);
Copy link
Contributor

@rg20 rg20 Aug 26, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In pool memory allocator, we would have probably allocated most of the memory, but the allocator itself might have available memory.

Copy link
Contributor Author

@aliceb-nv aliceb-nv Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only used to get the total device memory :) free_mem is a placeholder here, only total_mem is used

@anandhkb anandhkb added this to the 25.10 milestone Aug 26, 2025
@aliceb-nv aliceb-nv marked this pull request as ready for review August 27, 2025 09:30
@aliceb-nv aliceb-nv requested a review from a team as a code owner August 27, 2025 09:30
@aliceb-nv aliceb-nv requested review from hlinsen and rg20 August 27, 2025 09:30
@aliceb-nv aliceb-nv changed the title [DRAFT] Fix high GPU memory usage [FIX] Fix high GPU memory usage Aug 27, 2025
@github-actions
Copy link

github-actions bot commented Sep 6, 2025

🔔 Hi @anandhkb, this pull request has had no activity for 7 days. Please update or let us know if it can be closed. Thank you!

If this is an "epic" issue, then please add the "epic" label to this issue.
If it is a PR and not ready for review, then please convert this to draft.
If you just want to switch off this notification, then use the "skip inactivity reminder" label.

@nguidotti
Copy link
Contributor

nguidotti commented Sep 9, 2025 •
edited
Loading

Looks good to me. This patch has been working great for running some MIP instances on my laptop.
Can I update the branch and rerun the test suite?

@aliceb-nv
Copy link
Contributor Author

aliceb-nv commented Sep 9, 2025

Sure @nguidotti, you can go ahead, thanks!

@akifcorduk
Copy link
Contributor

akifcorduk commented Sep 10, 2025

/merge

@rapids-bot rapids-bot bot merged commit 4e8bdae into branch-25.10 Sep 10, 2025
102 checks passed
aliceb-nv added a commit that referenced this pull request Sep 22, 2025
@aliceb-nv
[FIX] Fix high GPU memory usage (#351)
83fa6db 
closes #349

compute_related_variables was heuristically allocating memory based on A100/H100 with >=40GB of VRAM.
This is now automatically adjusted based on the total VRAM of the device

A command-line option has also been added to solve_MPS_file to specify device memory allocation limits for ease of testing.

Authors:
  - Alice Boucher (https://github.com/aliceb-nv)
  - Nicolas L. Guidotti (https://github.com/nguidotti)

Approvers:
  - Nicolas L. Guidotti (https://github.com/nguidotti)

URL: #351
aliceb-nv added a commit that referenced this pull request Sep 22, 2025
@aliceb-nv
[FIX] Fix high GPU memory usage (#351)
7a9e7bf 
closes #349

compute_related_variables was heuristically allocating memory based on A100/H100 with >=40GB of VRAM.
This is now automatically adjusted based on the total VRAM of the device

A command-line option has also been added to solve_MPS_file to specify device memory allocation limits for ease of testing.

Authors:
  - Alice Boucher (https://github.com/aliceb-nv)
  - Nicolas L. Guidotti (https://github.com/nguidotti)

Approvers:
  - Nicolas L. Guidotti (https://github.com/nguidotti)

URL: #351
@rgsl888prabhu rgsl888prabhu deleted the lower-mem-usage branch October 29, 2025 16:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nguidotti nguidotti nguidotti approved these changes

@rg20 rg20 Awaiting requested review from rg20 rg20 is a code owner automatically assigned from NVIDIA/cuopt-engine-codeowners

@hlinsen hlinsen Awaiting requested review from hlinsen hlinsen is a code owner automatically assigned from NVIDIA/cuopt-engine-codeowners

Assignees

No one assigned

Labels

improvement Improves an existing functionality non-breaking Introduces a non-breaking change

Projects

None yet

Milestone

25.10

Development

Successfully merging this pull request may close these issues.

[BUG] High GPU memory usage for MIP solver

6 participants

@aliceb-nv @nguidotti @akifcorduk @rg20 @anandhkb