Crash 0x406d1388 with .NET bindings (for ffmpeg >= 6 because of ff_thread_setname) - mingw bug

[SuRGeoNix]

Hi, it seems there are some compilation flags that causing this issue (which is related with thread naming). It mainly happens when I try to set the tread_count > 1 and then opening the codec. I've tested same ffmpeg version (7.1) with gyan's releases and was working fine.

Possible related compilation flags here?
https://github.com/FFmpeg/FFmpeg/blob/master/libavutil/thread.h

[BtbN]

Not sure what you're asking. Those codes mean nothing. Looks like a "good old crash" to me, so you're probably doing something wrong in your code.
But impossible to tell, and this is also not the place to debug your code.

[SuRGeoNix]

I'm pretty sure the issue is with the compilation as Gyan's same version works.
Sorry for not having an easy reproducible way yet. I might have tomorrow.
Leave this open and I will try to create a PR when I will find some time to compile it manually.

One more thought ... possible the SDL version?

[BtbN]

Do you have a commandline that reproduces the crash?

[SuRGeoNix]

It happens only when using the libraries as native from Visual Studio (eg. with FFmpeg.AutoGen) but I'm using a custom generator for now with 7.1 that FFmpeg.AutoGen does not support yet. Not sure if it happens with v6.x. I might have a git repo for generator and with a reproducible project to test tomorrow.

[BtbN]

I can't really make any promises when using them from Visual Studio.
Not even sure if all type sizes are the same between MSVC and gcc/mingw. Though they normally should be.
But some kind of semantics might be slightly different. Would highly recommend not using MSVC together with non-MSVC-built libraries, specially if those libraries bundle in a ton of libraries.

[SuRGeoNix]

One reason that I'm using a custom Generator is that. To fix all the type sizes issues that currently FFmpeg. Autogen has. But I'm pretty sure that the issue is not with the type sizes.

[BtbN]

What do you mean by "custom Generator", and how does ffmpeg have type size issues?
Also, what is "ffmpeg autogen"? FFmpeg does not use autotools. And the build system has no influence on type sizes whatsoever anyway.

[SuRGeoNix]

FFmpeg.Autogen is an FFmpeg auto generated unsafe bindings for C#/.NET and Core (Linux, MacOS and Mono).
Underneath uses CppSharp package which generates the C# code which I also use with my custom generator.
FFmpeg.Autogen (and CppSharp) has the issue with the type sizes and not the ffmpeg itself.
Hope I clarify things for you :)

[BtbN]

Ah, yes. autogen.sh is often the script to generate an autotools based build system.
And yeah, there definitely will be type size issues between C# and C/C++ types and structs. Independently of MSVC vs. gcc.

[SuRGeoNix]

It takes too long to build the docker and compile so until then ...
I suspect this is related with zmq < 4.3.4 (or 4.3.5) zeromq/libzmq#3586

Update 1: Building a fresh image docker win64 and compiling 7.1 gave me the same issue so far
Update 2: Trying to downgrade to libzmq to 4.3.2 (as the default installed 4.3.6 git master latest) the compilation failed. Trying to completely remove libzmq the issue still exists... I will give a try with mingw..
Update 3: Success... downgrade to 0795443#diff-471713abf7918faa3680336a0530cbe2473618b749188a309f1a9cdded9d991cR4 for mingw and mingw-std-threads worked but I think I did something wrong with step/Update 2 still think that the issue is zmq ... so I will give one more try with clean system only by removing zmq
Update 4: It's mingw and/or mingw-std-threads and a possible related patch https://sourceforge.net/p/mingw-w64/mailman/mingw-w64-public/thread/267a495a-8310-cf90-5b83-c15019445bf0%40martin.st/#msg58829891

Update 5: Confirmed that by downgrading just mingw to f6b0870e4de11fd04155511606bb6e6564d38c71 commit fixes the issue

[SuRGeoNix]

Here is a minimal reproducible example (.NET 8 x64 C#)

using System.Runtime.InteropServices;
using System.Security;

namespace BtbNThreadNamingCrash;

unsafe internal partial class Program
{
    static void Main(string[] args)
    {
        LoadFFmpegLibraries(@"c:\FFmpeg\BtbN\x64");

        var codec = avcodec_find_decoder_by_name("h264");
        var avctx = avcodec_alloc_context3(codec);
        void* dict = null;
        //av_dict_set(&dict, "threads", "1", 0); // success
        av_dict_set(&dict, "threads", "2", 0); // failed
        var ret = avcodec_open2(avctx, null, &dict);

        Thread.Sleep(3000);
        Console.WriteLine("Success");
    }

    public const string AVCODEC    = "avcodec";
    public const string AVDEVICE   = "avdevice";
    public const string AVFILTER   = "avfilter";
    public const string AVFORMAT   = "avformat";
    public const string AVUTIL     = "avutil";
    public const string POSTPROC   = "postproc";
    public const string SWRESAMPLE = "swresample";
    public const string SWSCALE    = "swscale";
    public static Dictionary<string, nint> libToPtr = [];

    public static void LoadFFmpegLibraries(string ffmpegPath)
    {
        var files = Directory.GetFiles(ffmpegPath, $"{AVUTIL}*.dll");
        libToPtr[AVUTIL] = LoadLibraryW(files[0]);

        files = Directory.GetFiles(ffmpegPath, $"{SWSCALE}*.dll");
        libToPtr[SWSCALE] = LoadLibraryW(files[0]);

        files = Directory.GetFiles(ffmpegPath, $"{SWRESAMPLE}*.dll");
        libToPtr[SWRESAMPLE] = LoadLibraryW(files[0]);

        files = Directory.GetFiles(ffmpegPath, $"{POSTPROC}*.dll");
        libToPtr[POSTPROC] = LoadLibraryW(files[0]);

        files = Directory.GetFiles(ffmpegPath, $"{AVCODEC}*.dll");
        libToPtr[AVCODEC] = LoadLibraryW(files[0]);

        files = Directory.GetFiles(ffmpegPath, $"{AVFORMAT}*.dll");
        libToPtr[AVFORMAT] = LoadLibraryW(files[0]);

        files = Directory.GetFiles(ffmpegPath, $"{AVFILTER}*.dll");
        libToPtr[AVFILTER] = LoadLibraryW(files[0]);

        files = Directory.GetFiles(ffmpegPath, $"{AVDEVICE}*.dll");
        libToPtr[AVDEVICE] = LoadLibraryW(files[0]);

        NativeLibrary.SetDllImportResolver(typeof(Program).Assembly, DllImportResolver);
    }

    private static IntPtr DllImportResolver(string libraryName, System.Reflection.Assembly assembly, DllImportSearchPath? searchPath)
    {
        libToPtr.TryGetValue(libraryName, out var ptr);
        
        return ptr;
    }

    [DllImport("kernel32", ExactSpelling = true, CharSet = CharSet.Unicode, SetLastError = true), SuppressUnmanagedCodeSecurity]
    public static extern IntPtr LoadLibraryW(string dllToLoad);

    [DllImport(AVCODEC, CallingConvention = CallingConvention.Cdecl, ExactSpelling = true), SuppressUnmanagedCodeSecurity]
    public static extern void* avcodec_alloc_context3(void* codec);

    [DllImport(AVCODEC, CallingConvention = CallingConvention.Cdecl, ExactSpelling = true), SuppressUnmanagedCodeSecurity]
    public static extern int avcodec_open2(void* avctx, void* codec, void** options);

    [DllImport(AVCODEC, CallingConvention = CallingConvention.Cdecl, ExactSpelling = true), SuppressUnmanagedCodeSecurity]
    public static extern void* avcodec_find_decoder_by_name([MarshalAs(UnmanagedType.LPUTF8Str)] string name);

    [DllImport(AVUTIL, CallingConvention = CallingConvention.Cdecl, ExactSpelling = true), SuppressUnmanagedCodeSecurity]
    public static extern int av_dict_set(void** pm, [MarshalAs(UnmanagedType.LPUTF8Str)] string key, [MarshalAs(UnmanagedType.LPUTF8Str)] string value, int flags);
}

[SuRGeoNix]

BtbNThreadNamingCrash.zip

Quick VS solution zip

Testing the latest Auto-Builds 7.1 fails, 6.1 fails, 5.1 works fine (checking FFmpeg's code they don't use ff_thread_setname on 5.1)

[BtbN]

This doesn't seem like an issue with the builds to me still. Or I'm not sure what you're asking from me.
I can't help you with the usage of third party wrappers. So unless you can point out a specific issue with these builds, I'll close this ticket.

[SuRGeoNix]

There are a lot of people using FFmpeg libraries with .NET bindings that they have this issue and they don't even know it.

I'm pretty sure I've provided enough information, so you can see that the issue is with your builds as it does not happen with Gyan's builds (just run the project I've attached). I've even provided a workaround by downgrading MinGW (couldn't investigate this further, at least for now).

The issue is critical and you should leave it open, even if you don't care, so other people will be informed and possible contribute resolving this.

[BtbN]

I don't doubt that there is an issue. But I have no experience with the .NET bindings and their inner working, or developing with C# native bindings in general.
So I can't debug this further. And from "it doesn't work" I can't really deduce any possible issue.

I generally don't understand what issue you are trying to point to. What does the thread name have to do with it? If the C# bindings crash depending on a thread name, that looks like a bug in the bindings to me?

[SuRGeoNix]

It all starts from a stupid MS way to set the thread name from native/managed code (which raises DWORD MS_VC_EXCEPTION = 0x406D1388).

MS - Tips for debugging threads

I'm not sure what exactly happens, if it thinks that there is a debugger attached or doesn't check at all so even at release mode it throws that exception and the app crashes. It is also possible that FFmpeg should catch that exception, but I think probably the issue is down to mingw. Not even sure if you could 'configure' the threading from your builds (eg. you have some configs for pthreads/winpthreads)