out of memory issue

[TemryL]

should fix:
RuntimeError: MPS backend out of memory (MPS allocated: 16.56 GB, other allocations: 128.70 MB, max allowed: 18.13 GB). Tried to allocate 3.00 GB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).

same issue on cuda with small gpu

[marcojoao]

also got out of ram:

`Error occurred when executing PipelineLoader:

Allocation on device 0 would exceed allowed memory. (out of memory)
Currently allocated : 2.44 GiB
Requested : 5.00 MiB
Device limit : 22.19 GiB
Free (according to CUDA): 6.50 MiB
PyTorch limit (set by user-supplied memory fraction)
: 17179869184.00 GiB

File "/home/runner/ComfyUI/execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/ComfyUI/execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/ComfyUI/execution.py", line 69, in map_node_over_list
results.append(getattr(obj, func)())
^^^^^^^^^^^^^^^^^^^^
File "/home/runner/ComfyUI/custom_nodes/ComfyUI-IDM-VTON/src/nodes/pipeline_loader.py", line 46, in load_pipeline
).requires_grad_(False).eval().to(DEVICE)
^^^^^^^^^^
File "/usr/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1152, in to
return self._apply(convert)
^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 802, in _apply
module._apply(fn)
File "/usr/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 802, in _apply
module._apply(fn)
File "/usr/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 802, in _apply
module._apply(fn)
[Previous line repeated 6 more times]
File "/usr/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 825, in _apply
param_applied = fn(param)
^^^^^^^^^
File "/usr/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1150, in convert
return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^`

[TemryL]

Yes it is a known issue that needs to be fixed, I only managed to make it work on gpu with 32gb+ of vram. Don't hesitate to make PRs if you find a way to fix it :)

[marcojoao]

Yes it is a known issue that needs to be fixed, I only managed to make it work on gpu with 32gb+ of vram. Don't hesitate to make PRs if you find a way to fix it :)

sure I'll take a look

[pranay0318]

Is there a way we can solve out of memory issue now?

[pranay0318]

Here's the Error I got!

Some weights of the model checkpoint were not used when initializing UNet2DConditionModel:
['add_embedding.linear_1.bias, add_embedding.linear_1.weight, add_embedding.linear_2.bias, add_embedding.linear_2.weight']
!!! Exception during processing!!! Allocation on device
Traceback (most recent call last):
File "/home/ubuntu/Comfy/ComfyUI/execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "/home/ubuntu/Comfy/ComfyUI/execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "/home/ubuntu/Comfy/ComfyUI/execution.py", line 69, in map_node_over_list
results.append(getattr(obj, func)())
File "/home/ubuntu/Comfy/ComfyUI/custom_nodes/ComfyUI-IDM-VTON/src/nodes/pipeline_loader.py", line 58, in load_pipeline
).requires_grad_(False).eval().to(DEVICE)
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1173, in to
return self._apply(convert)
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 779, in _apply
module._apply(fn)
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 779, in _apply
module._apply(fn)
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 779, in _apply
module._apply(fn)
[Previous line repeated 4 more times]
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 804, in _apply
param_applied = fn(param)
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1159, in convert
return t.to(
torch.cuda.OutOfMemoryError: Allocation on device

[SuperMasterBlasterLaser]

I've encountered this error too.

When I launch your nodes with your example. It works good. It uses preloaded image and applies garment.

The problem is when I add SD Checkpoint and try to send generated image into Run IDM-VTON Inference node, where it throws this error.

My Specs:

1. RAM - 32GB
2. VRAM (GPU) - 24GB

I think we need some kind of memory unloading?

[HashedViking]

Yes it is a known issue that needs to be fixed, I only managed to make it work on gpu with 32gb+ of vram. Don't hesitate to make PRs if you find a way to fix it :)

Hi, @TemryL , PipelineLoader node crashes ComfyUI no matter what I choose f32/f16/b
My spec: VRAM 24 Gb, RAM 16 GB, wsl2
Have you managed to lower VRAM requirements?
Any advice on how to debug this crash would be really appreciated.
Thank you in advance.

[HashedViking]

Yes it is a known issue that needs to be fixed, I only managed to make it work on gpu with 32gb+ of vram. Don't hesitate to make PRs if you find a way to fix it :)

Hi, @TemryL , PipelineLoader node crashes ComfyUI no matter what I choose f32/f16/b My spec: VRAM 24 Gb, RAM 16 GB, wsl2 Have you managed to lower VRAM requirements? Any advice on how to debug this crash would be really appreciated. Thank you in advance.

fixed by adding 16 GB more RAM to the wsl, so 24 GB VRAM + 32 GB RAM seems enough

[kunkun-zhu]

是的，这是一个需要修复的已知问题，我只能让它在具有 32gb+ vram 的 gpu 上工作。如果你找到修复它的方法，不要犹豫，做 PR :)

你好，@TemryL，无论我选择 f32/f16/b，PipelineLoader 节点都会使 ComfyUI 崩溃。我的规格：VRAM 24 Gb、RAM 16 GB、wsl2 您是否已设法降低 VRAM 要求？任何有关如何调试此崩溃的建议都将不胜感激。提前谢谢您。

通过向 wsl 添加 16 GB 的 RAM 来修复此问题，因此 24 GB VRAM + 32 GB RAM 似乎足够了

Yes it is a known issue that needs to be fixed, I only managed to make it work on gpu with 32gb+ of vram. Don't hesitate to make PRs if you find a way to fix it :)

Hi, @TemryL , PipelineLoader node crashes ComfyUI no matter what I choose f32/f16/b My spec: VRAM 24 Gb, RAM 16 GB, wsl2 Have you managed to lower VRAM requirements? Any advice on how to debug this crash would be really appreciated. Thank you in advance.

fixed by adding 16 GB more RAM to the wsl, so 24 GB VRAM + 32 GB RAM seems enough

hello,My server also reported out of memory with 24 GB VRAM and 240 GB RAM. Are you sure you have resolved this issue？

[HashedViking]

@kunkun-zhu hi,
I've only tried the example workflow.png. Maybe you have an advanced one that causes additional RAM/VRAM load.
My server shows ~ 17 GB RAM and ~20 GB VRAM utilisation.