ComfyUI-Zluda issues with Navi 21 (RDNA2, gfx 1030, Radeon RX6800 XT) on Windows

[timm1k]

Package

ComfyUI-Zluda

When did the issue occur?

Running the Package

What GPU / hardware type are you using?

AMD Radeon RX 6800 XT

What happened?

After playing a bit with generation on default Comfy UI install with DirectML backend I decied to test others in hope to gain some performance. ROCm was out of the picture due to no support for Navi 21 on Windows, so I went ZLUDA way.

In the end I managed to set it up and performance improvement was huge, however it required a lot of debugging and messing around with options, to a point that normal Comfy UI ZLUDA directly from their repo was just easier and worked out of the box.

Problems:

1. Out of the box experience with trying to generate txt to img (Illustrious) ended with cuDNN Frontend error: [cudnn_frontend] Error: No execution plans support the graph). Both for Interference UI in Stability Matrix and own workflow from ComfyUI webUI.

• in Comfy UI i tried with different setups of CFZ CUDNN Toggle node, but only thing I got was getting different error CUDNN_STATUS_INTERNAL_ERROR
• solution was to switch Cross Attention Method from default --use-pytorch-cross-attention to --use-quad-cross-attention. I also added --disable-async-offload --disable-pinned-memory as extra options.
• launch options mentioned above are default part of launch bat script for comfyUI-Zluda - comfyui-n.bat, which is the way to launch it mentioned by Readme in comfyUI-Zluda repo, it also contains debugging and checks of environmental variables like HIP_PATH. Stability Matrix just launches the zluda directly and defaults are not what they should be.

2. After first reinstall in trying to solve first (I switched also python from 3.12.11 to selected by default 3.11.13). Zluda started to default to my iGPU (Ryzen 7700) which wasn't supported. No idea why it didn't happen at first time.

• I needed to add HCC_AMDGPU_TARGET, HIP_VISIBLE_DEVICES and ROCR_VISIBLE_DEVICES all set to 1 as environmental variables (it was enough to set them in SM settings, no need to system-wide in windows properties)

3. I added ROCm bin/ directory to system path env variable manually but no idea if it was needed. I was going with whole ComfyUI-Zluda readme and verifying everything. Could be supplied by Stability Matrix.

After all tweaks it also runs from Stability Matrix, ust Text to Image workflow in Stability Matrix Interference UI fails on VAE Decode, as CFZ CUDNN Toggle node is required to disable CUDNN for this step.

Working workflow for use in Comfy UI directly, with CFZ nodes attached - cudnn-toggle-txt-to-img.json

Console output

No response

Version

v2.15.5

What Operating System are you using?

Windows

[timm1k]

CUDNN issue seems to be related to BF16 vae, forcing FP32 vae fixes it.

[NeuralFault]

On the thought of launching comfyui-n.bat, that is not advisable as it would cause issues with SM's process handling and needs to be re-implemented in C# (as it is currently, which it then calls zluda.exe).

The BAT file also causes an issue with SM's process tracking. It can/will cause the batch file to never actually stop and the user would have to kill the process manually.

The BAT file also does a git-pull on every launch, which will cause conflicts with SM's package version management.

The C# implementation of comfyui-n.bat's functions just needs to be updated. The '--disable-async-offload --disable-pinned-memory' arguments are new thing added since last update to SM's comfyui-zluda's package configuration.
And the usage of quad-cross-attention by default was a miss on my part.

I am implementing these changes to update it in my fork and will submit a PR.

As for it using the GPU inside your Ryzen APU, a lot of these packages grab the first GPU reported to it by the system unless you specify it first via EnVar (not just a ComfyUI-Zluda thing). The HIP_VISIBLE_DEVICES in SM Settings has to be set to the second reported GPU for that (along with sometimes CUDA_VISIBLE_DEVICES on zluda setups). Not everyone has an iGPU, so implementing a check to pass it globally in SM might be an idea for certain CPU models... but right now it has to be manually set.

One thing though, can you try forcing FP16 VAE? or does that not work for you?

[timm1k]

One thing though, can you try forcing FP16 VAE? or does that not work for you?

It works mostly, but some VAE just don't work in FP16 mode, ending with black output, e.g https://huggingface.co/stabilityai/sdxl-vae