Segmentation fault with mkv (av1 codec)

[kanehekili]

Information:

• mpv version: mpv v0.37.0
• Manjaro XFCE latest
• Source of the mpv binary: Arch Linux Repository
• If known which version of mpv introduced the problem: UNKNOWN
• Window Manager and version: XfWM4 4.18.0-2
• GPU model, driver and version: Nvidia GeForce 760 Driver: 470.239.06 Cuda Version 11-4

Reproduction steps

The error can be reproduced with:
mpv ---hwdec=auto-safe file-withAV1.mkv
to enforce hardware decoding. Without config it works, but software decoding only.

Expected behavior

mpv does not segfault, but uses software decoding. E.g. the flag --hwdec-codecs=nvdec-copy prevents any segfaults.
Obviously my old GPU can't decode AV1.

Actual behavior

Segmentation fault. The testFile used to work with mpv and HW acceleration.
When running mpv in IDE (using libmpv in eclipse) I can see:

 >mkv: | + muxing app: Lavf58.29.100
 >mkv: | + writing app: Lavf58.29.100

and code continues, using software decoding. Running in a terminal :

>mkv: | + muxing app: Lavf59.27.100
>mkv: | + writing app: Lavf59.27.100
Fatal Python error: Segmentation fault

(If relevant )

Log file

output.log

Error message:

 (+) Video --vid=1 (*) (av1 3840x2160 29.970fps)
 (+) Audio --aid=1 --alang=eng (*) (opus 2ch 48000Hz)
[ffmpeg/video] av1: Hardware is lacking required capabilities
[ffmpeg/video] av1: Failed setup for format cuda: hwaccel initialisation returned error.
[ffmpeg/video] av1: Your platform doesn't support hardware accelerated AV1 decoding.
[ffmpeg/video] av1: Failed to get pixel format.
Error while decoding frame (hardware decoding)!
[vaapi] libva: vaGetDriverNames() failed with unknown libva error
[ffmpeg/video] av1: Hardware is lacking required capabilities
[ffmpeg/video] av1: Failed setup for format cuda: hwaccel initialisation returned error.
[ffmpeg/video] av1: Your platform doesn't support hardware accelerated AV1 decoding.
[ffmpeg/video] av1: Failed to get pixel format.
Error while decoding frame (hardware decoding)!
Segmentation fault (core dumped)

Sample files

I've donwloaded that file from here using yt-dl

[CounterPillow]

Please obtain a backtrace with e.g. gdb, see https://wiki.archlinux.org/title/Debugging/Getting_traces#Getting_the_trace

Also there is no way you ever had working AV1 hardware decoding with your GT 760 as that card is significantly older than AV1.

[kanehekili]

Thanks @CounterPillow for the fast reply
The gdb backtrace can be found here.
I do not expect a hardware decoding with that card. (I've edited my first post) The same code runs (in softwaremode) perfectly on my old intel T410 - no segfault.
My expectation is: Switch to software decoding if the hardware does not support it. Do not segfault.

So it seems there is a different handling between old intel and old NVIDIA cards.
Thank you guys for maintaining this fantastic piece of software!

[CounterPillow]

Thread 1 (Thread 0x7fffe188af40 (LWP 1053267) "mpv"):
#0  XDisplayString (dpy=0x0) at /usr/src/debug/libx11/libX11-1.8.7/src/Macros.c:119
#1  0x00007fffd8cb1a1d in __vaDriverInit_1_13 () at /usr/lib/dri/nvidia_drv_video.so
#2  0x00007ffff42ce27c in vaInitialize () at /usr/lib/libva.so.2
#3  0x00005555556cca28 in va_initialize (display=0x555555945ac0, plog=plog@entry=0x5555559392b0, probing=true) at > ../mpv/video/vaapi.c:116

Is the segfault happening in thread 1 here? gdb should tell you before the backtrace somewhere. If it is, then the problem looks to be that you have a buggy VA-API-on-nvidia wrapper installed on your system, which might either be libva-nvidia-driver or libva-vdpau-driver. Uninstall those.

Not a bug in mpv as said buggy wrapper then calls into libX11 with what looks to be a nullpointer.

[kanehekili]

Thanks for the analysis. I'll look into it. As I mentioned above, the code works, if I'm running it in an IDE - but not in terminal - something I haven't seen yet. Why doesn't it fail if I add --hwdec-codecs=nvdec-copy ? I just checked: same problem with a VP8 codec.
Update: removing "libva-vdpau-driver" solves the problem. Closing this issue