SD WebUI not working on Framework 16, RX 7700S (gfx1102), Fedora 40

[tgitnh]

Context / Goal

I have the Framework Laptop 16 with the RX 7700S (gfx1102) and would like to use the Stable Diffusion Web UI on Fedora 40.

What I've done

This is one example of what I tried:

• Installing all rocm* packages: sudo dnf in rocm*
• Making sure my GPU is supported and OpenCL can see it:

rocminfo | egrep -C 3 gfx
rocm-clinfo

• Setting environment variables for my GPU:

export HSA_OVERRIDE_GFX_VERSION=11.0.2
export PYTORCH_ROCM_ARCH=gfx1102
export HIP_VISIBLE_DEVICES=0

• Using a Miniconda environment with Python 3.10:

conda create --name sd python=3.10
conda activate sd

• Uninstalling standard torch packages:
  pip uninstall torch torchvision torchaudio torchtext functorch xformers
• Installing PyTorch ROCm 6.0 Stable:
  pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/rocm6.0
• Making sure PyTorch ROCm is installed and supports the GPU:

python3 -c 'import torch' 2> /dev/null & echo 'Success' || echo 'Failure'
python3 -c 'import torch; print(torch.cuda.is_available())'

• Cloning Stable Diffusion WebUI: git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
• Installing its requirements: pip install -r requirements.txt
• Starting the WebUI shell script: ./webui.sh

Error messages

These are the error messages the webui script is showing at the end (this is when I tried Python venv instead of Miniconda but the result is the same):

Launching Web UI with arguments: --listen --enable-insecure-extension-access --opt-sdp-attention --no-half-vae
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
Downloading: "https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.safetensors" to /home/tobias/apps/stable-diffusion-webui/models/Stable-diffusion/v1-5-pruned-emaonly.safetensors

100%|...| 3.97G/3.97G [2:01:17<00:00, 586kB/s]
Calculating sha256 for /home/tobias/apps/stable-diffusion-webui/models/Stable-diffusion/v1-5-pruned-emaonly.safetensors: Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 7625.9s (prepare environment: 343.1s, import torch: 2.7s, import gradio: 0.4s, setup paths: 0.6s, other imports: 0.5s, list SD models: 7278.2s, load scripts: 0.2s, create ui: 0.2s).
6ce0161689b3853acaa03779ec93eafe75a02f4ced659bee03f50797806fa2fa
Loading weights [6ce0161689] from /home/tobias/apps/stable-diffusion-webui/models/Stable-diffusion/v1-5-pruned-emaonly.safetensors
Creating model from config: /home/tobias/apps/stable-diffusion-webui/configs/v1-inference.yaml
/home/tobias/apps/python-venv/stabdiff/lib64/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
loading stable diffusion model: RuntimeError
Traceback (most recent call last):
  File "/usr/lib64/python3.10/threading.py", line 973, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib64/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib64/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/home/tobias/apps/stable-diffusion-webui/modules/initialize.py", line 149, in load_model
    shared.sd_model  # noqa: B018
  File "/home/tobias/apps/stable-diffusion-webui/modules/shared_items.py", line 175, in sd_model
    return modules.sd_models.model_data.get_sd_model()
  File "/home/tobias/apps/stable-diffusion-webui/modules/sd_models.py", line 620, in get_sd_model
    load_model()
  File "/home/tobias/apps/stable-diffusion-webui/modules/sd_models.py", line 748, in load_model
    load_model_weights(sd_model, checkpoint_info, state_dict, timer)
  File "/home/tobias/apps/stable-diffusion-webui/modules/sd_models.py", line 393, in load_model_weights
    model.load_state_dict(state_dict, strict=False)
  File "/home/tobias/apps/stable-diffusion-webui/modules/sd_disable_initialization.py", line 223, in <lambda>
    module_load_state_dict = self.replace(torch.nn.Module, 'load_state_dict', lambda *args, **kwargs: load_state_dict(module_load_state_dict, *args, **kwargs))
  File "/home/tobias/apps/stable-diffusion-webui/modules/sd_disable_initialization.py", line 221, in load_state_dict
    original(module, state_dict, strict=strict)
  File "/home/tobias/apps/python-venv/stabdiff/lib64/python3.10/site-packages/torch/nn/modules/module.py", line 2175, in load_state_dict
    load(self, state_dict)
  File "/home/tobias/apps/python-venv/stabdiff/lib64/python3.10/site-packages/torch/nn/modules/module.py", line 2163, in load
    load(child, child_state_dict, child_prefix)  # noqa: F821
  File "/home/tobias/apps/python-venv/stabdiff/lib64/python3.10/site-packages/torch/nn/modules/module.py", line 2163, in load
    load(child, child_state_dict, child_prefix)  # noqa: F821
  File "/home/tobias/apps/python-venv/stabdiff/lib64/python3.10/site-packages/torch/nn/modules/module.py", line 2163, in load
    load(child, child_state_dict, child_prefix)  # noqa: F821
  [Previous line repeated 1 more time]
  File "/home/tobias/apps/python-venv/stabdiff/lib64/python3.10/site-packages/torch/nn/modules/module.py", line 2157, in load
    module._load_from_state_dict(
  File "/home/tobias/apps/stable-diffusion-webui/modules/sd_disable_initialization.py", line 225, in <lambda>
    linear_load_from_state_dict = self.replace(torch.nn.Linear, '_load_from_state_dict', lambda *args, **kwargs: load_from_state_dict(linear_load_from_state_dict, *args, **kwargs))
  File "/home/tobias/apps/stable-diffusion-webui/modules/sd_disable_initialization.py", line 191, in load_from_state_dict
    module._parameters[name] = torch.nn.parameter.Parameter(torch.zeros_like(param, device=device, dtype=dtype), requires_grad=param.requires_grad)
  File "/home/tobias/apps/python-venv/stabdiff/lib64/python3.10/site-packages/torch/_meta_registrations.py", line 4964, in zeros_like
    res.fill_(0)
RuntimeError: HIP error: invalid device function
HIP kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing AMD_SERIALIZE_KERNEL=3.
Compile with `TORCH_USE_HIP_DSA` to enable device-side assertions.



Stable diffusion model failed to load
Applying attention optimization: sdp... done.

rocBLAS error: Cannot read /home/tobias/apps/python-venv/stabdiff/lib64/python3.10/site-packages/torch/lib/rocblas/library/TensileLibrary.dat: No such file or directory for GPU arch : gfx1102
 List of available TensileLibrary Files : 
"/home/tobias/apps/python-venv/stabdiff/lib64/python3.10/site-packages/torch/lib/rocblas/library/TensileLibrary_lazy_gfx1030.dat"
"/home/tobias/apps/python-venv/stabdiff/lib64/python3.10/site-packages/torch/lib/rocblas/library/TensileLibrary_lazy_gfx1100.dat"
"/home/tobias/apps/python-venv/stabdiff/lib64/python3.10/site-packages/torch/lib/rocblas/library/TensileLibrary_lazy_gfx900.dat"
"/home/tobias/apps/python-venv/stabdiff/lib64/python3.10/site-packages/torch/lib/rocblas/library/TensileLibrary_lazy_gfx906.dat"
"/home/tobias/apps/python-venv/stabdiff/lib64/python3.10/site-packages/torch/lib/rocblas/library/TensileLibrary_lazy_gfx908.dat"
"/home/tobias/apps/python-venv/stabdiff/lib64/python3.10/site-packages/torch/lib/rocblas/library/TensileLibrary_lazy_gfx90a.dat"
"/home/tobias/apps/python-venv/stabdiff/lib64/python3.10/site-packages/torch/lib/rocblas/library/TensileLibrary_lazy_gfx942.dat"
launch.sh: Zeile 6: 50759 Abgebrochen             (Speicherabzug geschrieben) python3 launch.py --listen --enable-insecure-extension-access --opt-sdp-attention --no-half-vae

How to interpret this?

As I understand it, the last error messages mean that the needed gfx1102 Tensile library has not been found. According to https://rocm.docs.amd.com/projects/install-on-linux/en/latest/reference/system-requirements.html , there seems to be no official AMD ROCm support for my GPU yet. But many people are also using e.g. their RX 6900XT which also does not appear in that list but which features a compatible library (gfx1030).
I have also seen the missing library that I probably need („gfx1102“) being listed in some discussions, e.g. ROCm/ROCm#2631. The gfx1102 is also mentioned in various ROCm release notes, e.g. ROCm/ROCm#3061

Other attempts

I have also tried several other things mentioned in tutorials with similar steps, e.g.:

• Using ROCm 5.7 and ROCm 6.1 Nightly instead of ROCm 6.0
• Trying out Stable Diffusion WebUI Forge → resulting in similar error messages similar to those above.
• Using Python venv instead of Miniconda env
• Using Python 3.11 instead of 3.10
• Setting the environment variables to 11.0.0 and gfx1100 → this also results in errors, probably as the gfx1100 (7900XT) has different specs than the mobile 7700S.
• Using a launch script to set specific variables and then start the WebUI: nano launch.sh →

#!/bin/sh
export HSA_OVERRIDE_GFX_VERSION=11.0.2
export PYTORCH_ROCM_ARCH=gfx1102
export HIP_VISIBLE_DEVICES=0
export PYTORCH_HIP_ALLOC_CONF=garbage_collection_threshold:0.8,max_split_size_mb:512
python3 launch.py --listen --enable-insecure-extension-access --opt-sdp-attention --no-half-vae

Remaining clueless

So, what I’m asking myself (and now you :-) ) at this point is:

• Is it possible to get Stable Diffusion working on my system? (Is anyone else out there able to successfully run it on the Framework 16?)
• Do I just need the missing library files for my GPU and copy them in the folder shown in the rocBLAS error message above? If so, how do I get or create them?

I am really very grateful for any help. Maybe it's just a step I did wrong or overlooked. Maybe you're telling me that it's just not working on my system because ROCm or Stable Diffusion or rocBLAS or whatever doesn't support my GPU yet.
Whatever, I'm open to any suggestions and answers.

Thanks!
Tobias

[tgitnh]

I finally got it working!

I created a new Python 3.10 venv and followed the steps described above, but...

• I installed PyTorch ROCm 5.7 (as this was recommended for better overall compatibility in several posts/tutorials)
• I set the HSA_OVERRIDE... environment variable to 11.0.0 (instead of 11.0.2) and added the AMDGPU_TARGETS variable. So my launch.sh script now looks like this:

#!/bin/sh
export HSA_OVERRIDE_GFX_VERSION=11.0.0
export PYTORCH_ROCM_ARCH=gfx1102
export AMDGPU_TARGETS=gfx1102
export HIP_VISIBLE_DEVICES=0
export PYTORCH_HIP_ALLOC_CONF=garbage_collection_threshold:0.8,max_split_size_mb:512
python3 launch.py --listen --enable-insecure-extension-access --opt-sdp-attention --no-half-vae

Thank you, also just for reading my post. I'm glad it's working now.

[Sloogs]

Thank you so much, I spent days trying to work this out until I read your post.