[Issue]: crash using Stablity Matrix on Fedora 41 with AMD GPU

[panton41]

Issue Description

I had SD.Next running on Bazzite Linux (which is based on Fedora 41) until about three weeks ago when this current error began. I changed distros to Linux Mint and had no problem with SD.Next, but there were issues in other programs, so I bounced around until I was on Fedora 41 itself and the previous problem continues. I am unsure whether his has to do with a Fedora update or an update to SD.Next, but I've been unable to get previous versions installed as well with the same error.

During the installation, an exception is thrown, and the process stops.

Stability Matrix is an App Image that creates an isolated environment to install various Stable Diffusion programs without worrying too much about system libraries.

Version Platform Description

Fedora Linux 41 x86_64
AMD Ryzen 5600X
64GB DDR4
AMD Radeon 7600 GT

No logs are created that contain the application version because it fails to install.

Relevant log output

Unpacking resources
Unpacking resources
Downloading package...
/home/harasurya/AppImages/Stable-Diffusion/Data/Packages/automatic/venv/lib/python3.10/site-packages/_distutils_hack/__init__.py:53: UserWarning: Reliance on distutils from stdlib is deprecated. Users must rely on setuptools to provide the distutils module. Avoid importing distutils or import setuptools first, and avoid setting SETUPTOOLS_USE_DISTUTILS=stdlib. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml
  warnings.warn(
17:34:21-631477 INFO     Starting SD.Next                                       
17:34:21-633871 INFO     Logger:                                                
                         file="/home/harasurya/AppImages/Stable-Diffusion/Data/P
                         ackages/automatic/sdnext.log" level=DEBUG size=64      
                         mode=create                                            
17:34:21-634866 INFO     Python: version=3.10.11 platform=Linux                 
                         bin="/home/harasurya/AppImages/Stable-Diffusion/Data/Pa
                         ckages/automatic/venv/bin/python3"                     
                         venv="/home/harasurya/AppImages/Stable-Diffusion/Data/P
                         ackages/automatic/venv"                                
17:34:21-647326 INFO     Version: app=sd.next updated=2025-01-29 hash=46464c4c  
                         branch=master                                          
                         url=https://github.com/vladmandic/automatic/tree/master
                         ui=main                                                
17:34:21-902721 INFO     Platform: arch=x86_64 cpu= system=Linux                
                         release=6.12.10-200.fc41.x86_64 python=3.10.11         
                         docker=False                                           
17:34:21-904451 DEBUG    Packages: venv=venv                                    
                         site=['venv/lib/python3.10/site-packages']             
17:34:21-905501 INFO     Args: ['--use-zluda', '--debug', '--test']             
17:34:21-906413 DEBUG    Setting environment tuning                             
17:34:21-907385 DEBUG    Torch allocator:                                       
                         "garbage_collection_threshold:0.80,max_split_size_mb:51
                         2"                                                     
17:34:22-036702 DEBUG    Torch overrides: cuda=False rocm=False ipex=False      
                         directml=False openvino=False zluda=True               
17:34:22-038510 INFO     Python: version=3.10.11 platform=Linux                 
                         bin="/home/harasurya/AppImages/Stable-Diffusion/Data/Pa
                         ckages/automatic/venv/bin/python3"                     
                         venv="/home/harasurya/AppImages/Stable-Diffusion/Data/P
                         ackages/automatic/venv"                                
17:34:22-040073 INFO     ROCm: AMD toolkit detected                             
17:34:22-064771 INFO     ROCm: agents=['gfx1102']                               
17:34:22-065675 INFO     ROCm: version=6.2, using agent gfx1102                 
17:34:22-066485 DEBUG    Install: package="onnxruntime-training" install        
                         required                                               
17:34:49-947027 DEBUG    Install: package="torch" install required              
17:34:49-948371 INFO     Torch: download and install in progress...             
                         cmd="torch==2.4.1+rocm6.1 torchvision==0.19.1+rocm6.1  
                         --index-url https://download.pytorch.org/whl/rocm6.1"  
17:37:08-311994 DEBUG    Install: package="onnxruntime" install required        
17:37:10-374875 INFO     Diffusers install:                                     
                         commit=7b100ce589b917d4c116c9e61a6ec46d4f2ab062        
17:37:19-574236 INFO     Install requirements: this may take a while...         
17:38:08-911568 DEBUG    Install: package="torch" install required              
17:38:08-912451 INFO     Install: verifying requirements                        
17:38:08-914168 DEBUG    Install: package="psutil" install required             
17:38:08-933648 DEBUG    Install: package="rich" install required               
17:38:08-952566 DEBUG    Install: package="requests" install required           
17:38:12-699276 INFO     Startup: standard                                      
17:38:12-700099 INFO     Verifying submodules                                   
17:38:19-247781 DEBUG    Git detached head detected:                            
                         folder="extensions-builtin/sd-extension-chainner"      
                         reattach=main                                          
17:38:19-248911 DEBUG    Git submodule: extensions-builtin/sd-extension-chainner
                         / main                                                 
17:38:19-263832 DEBUG    Git detached head detected:                            
                         folder="extensions-builtin/sd-extension-system-info"   
                         reattach=main                                          
17:38:19-264790 DEBUG    Git submodule:                                         
                         extensions-builtin/sd-extension-system-info / main     
17:38:19-276643 DEBUG    Git detached head detected:                            
                         folder="extensions-builtin/sd-webui-agent-scheduler"   
                         reattach=main                                          
17:38:19-277551 DEBUG    Git submodule:                                         
                         extensions-builtin/sd-webui-agent-scheduler / main     
17:38:19-295833 DEBUG    Git detached head detected:                            
                         folder="extensions-builtin/sdnext-modernui"            
                         reattach=main                                          
17:38:19-296800 DEBUG    Git submodule: extensions-builtin/sdnext-modernui /    
                         main                                                   
17:38:19-318016 DEBUG    Git detached head detected:                            
                         folder="extensions-builtin/stable-diffusion-webui-rembg
                         " reattach=master                                      
17:38:19-319135 DEBUG    Git submodule:                                         
                         extensions-builtin/stable-diffusion-webui-rembg /      
                         master                                                 
17:38:19-331743 DEBUG    Git detached head detected:                            
                         folder="modules/k-diffusion" reattach=master           
17:38:19-332722 DEBUG    Git submodule: modules/k-diffusion / master            
17:38:19-344228 DEBUG    Git detached head detected: folder="wiki"              
                         reattach=master                                        
17:38:19-345571 DEBUG    Git submodule: wiki / master                           
17:38:19-353227 DEBUG    Register paths                                         
17:38:19-372312 DEBUG    Installed packages: 159                                
17:38:19-373075 DEBUG    Extensions all: ['Lora', 'sd-extension-chainner',      
                         'sd-extension-system-info', 'sd-webui-agent-scheduler',
                         'sdnext-modernui', 'stable-diffusion-webui-rembg']     
17:38:19-424322 DEBUG    Extension installer:                                   
                         /home/harasurya/AppImages/Stable-Diffusion/Data/Package
                         s/automatic/extensions-builtin/sd-webui-agent-scheduler
                         /install.py                                            
17:38:21-946561 INFO     Extension installed packages: sd-webui-agent-scheduler 
                         ['SQLAlchemy==2.0.37', 'greenlet==3.1.1']              
17:38:21-965942 DEBUG    Extension installer:                                   
                         /home/harasurya/AppImages/Stable-Diffusion/Data/Package
                         s/automatic/extensions-builtin/stable-diffusion-webui-r
                         embg/install.py                                        
17:38:27-292529 INFO     Extension installed packages:                          
                         stable-diffusion-webui-rembg ['PyMatting==1.1.13',     
                         'opencv-python-headless==4.11.0.86', 'pooch==1.8.2',   
                         'rembg==2.0.62']                                       
17:38:27-293639 DEBUG    Extensions all: []                                     
17:38:27-294378 INFO     Extensions enabled: ['Lora', 'sd-extension-chainner',  
                         'sd-extension-system-info', 'sd-webui-agent-scheduler',
                         'sdnext-modernui', 'stable-diffusion-webui-rembg']     
17:38:27-295204 INFO     Install requirements: this may take a while...         
17:38:27-295798 INFO     Install: verifying requirements                        
17:38:27-296449 DEBUG    Setup complete without errors: 1738276707              
17:38:27-298651 DEBUG    Extension preload: {'extensions-builtin': 0.0,         
                         'extensions': 0.0}                                     
17:38:27-299685 INFO     Command line args: ['--use-zluda', '--debug', '--test']
                         use_zluda=True test=True debug=True args=[]            
17:38:27-300754 DEBUG    Env flags: []                                          
17:38:27-301572 DEBUG    Starting module: <module 'webui' from                  
                         '/home/harasurya/AppImages/Stable-Diffusion/Data/Packag
                         es/automatic/webui.py'>                                
amdgpu.ids: No such file or directory
17:38:30-477426 DEBUG    Reading failed: config.json [Errno 2] No such file or  
                         directory: 'config.json'                               
17:38:30-981965 INFO     Device detect: memory=16.0 default=balanced            
17:38:30-984551 DEBUG    Settings: fn="config.json" created                     
17:38:30-986301 INFO     Engine: backend=Backend.DIFFUSERS compute=rocm         
                         device=cuda attention="Scaled-Dot-Product" mode=no_grad
17:38:30-987478 DEBUG    Save: file="config.json" json=19 bytes=732 time=0.002  
17:38:30-988489 DEBUG    Migrated styles: file="styles.csv"                     
                         folder="models/styles"                                 
17:38:30-993767 INFO     Available Styles: folder="models/styles" items=288     
                         time=0.00                                              
17:38:30-994804 DEBUG    Read: file="html/reference.json" json=63 bytes=32978   
                         time=0.000 fn=_call_with_frames_removed:<module>       
Error: StabilityMatrix.Core.Exceptions.ProcessException: install script failed with code 139: '/home/harasurya/AppImages/Stable-Diffusion/Data/Packages/automatic/venv/lib/python3.10/site-packages/_distutils_hack/__init__.py:53: UserWarning: Reliance on distutils from stdlib is deprecated. Users must rely on setuptools to provide the distutils module. Avoid importing distutils or import setuptools first, and avoid setting SETUPTOOLS_USE_DISTUTILS=stdlib. Register concerns at https://github.com/pypa/setuptools/issues/new?template=distutils-deprecation.yml\n  warnings.warn(\n17:34:21-631477 INFO     Starting SD.Next                                       \n17:34:21-633871 INFO     Logger:                                                \n                         file="/home/harasurya/AppImages/Stable-Diffusion/Data/P\n                         ackages/automatic/sdnext.log" level=DEBUG size=64      \n                         mode=create                                            \n17:34:21-634866 INFO     Python: version=3.10.11 platform=Linux                 \n                         bin="/home/harasurya/AppImages/Stable-Diffusion/Data/Pa\n                         ckages/automatic/venv/bin/python3"                     \n                         venv="/home/harasurya/AppImages/Stable-Diffusion/Data/P\n                         ackages/automatic/venv"                                \n17:34:21-647326 INFO     Version: app=sd.next updated=2025-01-29 hash=46464c4c  \n                         branch=master                                          \n                         url=https://github.com/vladmandic/automatic/tree/master\n                         ui=main                                                \n17:34:21-902721 INFO     Platform: arch=x86_64 cpu= system=Linux                \n                         release=6.12.10-200.fc41.x86_64 python=3.10.11         \n                         docker=False                                           \n17:34:21-904451 DEBUG    Packages: venv=venv                                    \n                         site=['venv/lib/python3.10/site-packages']             \n17:34:21-905501 INFO     Args: ['--use-zluda', '--debug', '--test']             \n17:34:21-906413 DEBUG    Setting environment tuning                             \n17:34:21-907385 DEBUG    Torch allocator:                                       \n                         "garbage_collection_threshold:0.80,max_split_size_mb:51\n                         2"                                                     \n17:34:22-036702 DEBUG    Torch overrides: cuda=False rocm=False ipex=False      \n                         directml=False openvino=False zluda=True               \n17:34:22-038510 INFO     Python: version=3.10.11 platform=Linux                 \n                         bin="/home/harasurya/AppImages/Stable-Diffusion/Data/Pa\n                         ckages/automatic/venv/bin/python3"                     \n                         venv="/home/harasurya/AppImages/Stable-Diffusion/Data/P\n                         ackages/automatic/venv"                                \n17:34:22-040073 INFO     ROCm: AMD toolkit detected                             \n17:34:22-064771 INFO     ROCm: agents=['gfx1102']                               \n17:34:22-065675 INFO     ROCm: version=6.2, using agent gfx1102                 \n17:34:22-066485 DEBUG    Install: package="onnxruntime-training" install        \n                         required                                               \n17:34:49-947027 DEBUG    Install: package="torch" install required              \n17:34:49-948371 INFO     Torch: download and install in progress...             \n                         cmd="torch==2.4.1+rocm6.1 torchvision==0.19.1+rocm6.1  \n                         --index-url https://download.pytorch.org/whl/rocm6.1"  \n17:37:08-311994 DEBUG    Install: package="onnxruntime" install required        \n17:37:10-374875 INFO     Diffusers install:                                     \n                         commit=7b100ce589b917d4c116c9e61a6ec46d4f2ab062        \n17:37:19-574236 INFO     Install requirements: this may take a while...         \n17:38:08-911568 DEBUG    Install: package="torch" install required              \n17:38:08-912451 INFO     Install: verifying requirements                        \n17:38:08-914168 DEBUG    Install: package="psutil" install required             \n17:38:08-933648 DEBUG    Install: package="rich" install required               \n17:38:08-952566 DEBUG    Install: package="requests" install required           \n17:38:12-699276 INFO     Startup: standard                                      \n17:38:12-700099 INFO     Verifying submodules                                   \n17:38:19-247781 DEBUG    Git detached head detected:                            \n                         folder="extensions-builtin/sd-extension-chainner"      \n                         reattach=main                                          \n17:38:19-248911 DEBUG    Git submodule: extensions-builtin/sd-extension-chainner\n                         / main                                                 \n17:38:19-263832 DEBUG    Git detached head detected:                            \n                         folder="extensions-builtin/sd-extension-system-info"   \n                         reattach=main                                          \n17:38:19-264790 DEBUG    Git submodule:                                         \n                         extensions-builtin/sd-extension-system-info / main     \n17:38:19-276643 DEBUG    Git detached head detected:                            \n                         folder="extensions-builtin/sd-webui-agent-scheduler"   \n                         reattach=main                                          \n17:38:19-277551 DEBUG    Git submodule:                                         \n                         extensions-builtin/sd-webui-agent-scheduler / main     \n17:38:19-295833 DEBUG    Git detached head detected:                            \n                         folder="extensions-builtin/sdnext-modernui"            \n                         reattach=main                                          \n17:38:19-296800 DEBUG    Git submodule: extensions-builtin/sdnext-modernui /    \n                         main                                                   \n17:38:19-318016 DEBUG    Git detached head detected:                            \n                         folder="extensions-builtin/stable-diffusion-webui-rembg\n                         " reattach=master                                      \n17:38:19-319135 DEBUG    Git submodule:                                         \n                         extensions-builtin/stable-diffusion-webui-rembg /      \n                         master                                                 \n17:38:19-331743 DEBUG    Git detached head detected:                            \n                         folder="modules/k-diffusion" reattach=master           \n17:38:19-332722 DEBUG    Git submodule: modules/k-diffusion / master            \n17:38:19-344228 DEBUG    Git detached head detected: folder="wiki"              \n                         reattach=master                                        \n17:38:19-345571 DEBUG    Git submodule: wiki / master                           \n17:38:19-353227 DEBUG    Register paths                                         \n17:38:19-372312 DEBUG    Installed packages: 159                                \n17:38:19-373075 DEBUG    Extensions all: ['Lora', 'sd-extension-chainner',      \n                         'sd-extension-system-info', 'sd-webui-agent-scheduler',\n                         'sdnext-modernui', 'stable-diffusion-webui-rembg']     \n17:38:19-424322 DEBUG    Extension installer:                                   \n                         /home/harasurya/AppImages/Stable-Diffusion/Data/Package\n                         s/automatic/extensions-builtin/sd-webui-agent-scheduler\n                         /install.py                                            \n17:38:21-946561 INFO     Extension installed packages: sd-webui-agent-scheduler \n                         ['SQLAlchemy==2.0.37', 'greenlet==3.1.1']              \n17:38:21-965942 DEBUG    Extension installer:                                   \n                         /home/harasurya/AppImages/Stable-Diffusion/Data/Package\n                         s/automatic/extensions-builtin/stable-diffusion-webui-r\n                         embg/install.py                                        \n17:38:27-292529 INFO     Extension installed packages:                          \n                         stable-diffusion-webui-rembg ['PyMatting==1.1.13',     \n                         'opencv-python-headless==4.11.0.86', 'pooch==1.8.2',   \n                         'rembg==2.0.62']                                       \n17:38:27-293639 DEBUG    Extensions all: []                                     \n17:38:27-294378 INFO     Extensions enabled: ['Lora', 'sd-extension-chainner',  \n                         'sd-extension-system-info', 'sd-webui-agent-scheduler',\n                         'sdnext-modernui', 'stable-diffusion-webui-rembg']     \n17:38:27-295204 INFO     Install requirements: this may take a while...         \n17:38:27-295798 INFO     Install: verifying requirements                        \n17:38:27-296449 DEBUG    Setup complete without errors: 1738276707              \n17:38:27-298651 DEBUG    Extension preload: {'extensions-builtin': 0.0,         \n                         'extensions': 0.0}                                     \n17:38:27-299685 INFO     Command line args: ['--use-zluda', '--debug', '--test']\n                         use_zluda=True test=True debug=True args=[]            \n17:38:27-300754 DEBUG    Env flags: []                                          \n17:38:27-301572 DEBUG    Starting module: <module 'webui' from                  \n                         '/home/harasurya/AppImages/Stable-Diffusion/Data/Packag\n                         es/automatic/webui.py'>                                \namdgpu.ids: No such file or directory\n17:38:30-477426 DEBUG    Reading failed: config.json [Errno 2] No such file or  \n                         directory: 'config.json'                               \n17:38:30-981965 INFO     Device detect: memory=16.0 default=balanced            \n17:38:30-984551 DEBUG    Settings: fn="config.json" created                     \n17:38:30-986301 INFO     Engine: backend=Backend.DIFFUSERS compute=rocm         \n                         device=cuda attention="Scaled-Dot-Product" mode=no_grad\n17:38:30-987478 DEBUG    Save: file="config.json" json=19 bytes=732 time=0.002  \n17:38:30-988489 DEBUG    Migrated styles: file="styles.csv"                     \n                         folder="models/styles"                                 \n17:38:30-993767 INFO     Available Styles: folder="models/styles" items=288     \n                         time=0.00                                              \n17:38:30-994804 DEBUG    Read: file="html/reference.json" json=63 bytes=32978   \n                         time=0.000 fn=_call_with_frames_removed:<module>       \n'
   at StabilityMatrix.Core.Python.PyVenvRunner.CustomInstall(ProcessArgs args, Action`1 outputDataReceived)
   at StabilityMatrix.Core.Models.Packages.VladAutomatic.InstallPackage(String installLocation, InstalledPackage installedPackage, InstallPackageOptions options, IProgress`1 progress, Action`1 onConsoleOutput, CancellationToken cancellationToken)
   at StabilityMatrix.Core.Models.Packages.VladAutomatic.InstallPackage(String installLocation, InstalledPackage installedPackage, InstallPackageOptions options, IProgress`1 progress, Action`1 onConsoleOutput, CancellationToken cancellationToken)
   at StabilityMatrix.Core.Models.PackageModification.InstallPackageStep.ExecuteAsync(IProgress`1 progress, CancellationToken cancellationToken)
   at StabilityMatrix.Core.Models.PackageModification.PackageModificationRunner.ExecuteSteps(IEnumerable`1 steps)

Backend

Diffusers

UI

Standard

Branch

Master

Model

Other

Acknowledgements

• I have read the above and searched for existing issues
• I confirm that this is classified correctly and its not an extension issue

[Disty0]

gfx1102 / RX 7600 series are unsupported by ROCm, you have to downgrade to ROCm 5.7.

Also why are you using --use-zluda on Linux? That is mainly for Windows because native PyTorch with ROCm isn't available on Windows.

[panton41]

Zulda is turned on because Stability Matrix turns it on by default. Switching to ROCm has identical results with not installing with the exact same error message.

This exact hardware has worked before on this operating system in December - and works on other Linux distros, even with ROCm 6.2 - and doesn't anymore. Something changed with an update to either Fedora or SD.Next around the New Year, and it broke. if it's Fedora I can figure it out on my end, but I need to know what the problem is in order to fix it and Error 139, from what I recall, is "segfault" which is a catchall for "Whelp, it just crashed" and from what I can see the log doesn't actually say anything useful.

The thing is, I've read the exact opposite and that you MUST have a 6.x branch of ROCm to get the hardware to work right. I honestly don't think anyone actually know what they're talking about, everyone has contradictory messages, and I include the engineers at AMD developing ROCm.

[vladmandic]

first thing that happens after last line in the log is setting some torch parameters.
i've just added a bit extra logging in that part of the code in dev branch, so can you try to install/use that and report updated log?
also, as soon as crash happens, run dmesg -ku so we can see the kernel log - it should show where actual crash happend.

[vladmandic]

@panton41 any updates?