Skyrim: Special Edition

[kMaiSmith]

I've built the tip of master (98d2bbd) and applied imported the build libraries into my wine (proton 3.7-7beta) context using the wine_setup_native script. When I run Skyrim: Special Edition I get the following error dialogues on launch:

Assertion failure at FAudio_CreateSourceVoice (src/FAudio.c:260), triggered 1 time: '0 && "Unsupported format tag!"'

Assertion failure at FAudioSourceVoice_SubmitSourceBuffer (src/FAudio.c:1274), triggered 1 time: 'pBufferWMA == ((void *)0)'

I can hit ignore on those errors and get to the main menu for the game. I hear the initial baseline for the title screen, but there is no menu music, or any other menu sounds.

I can start a new game, which results in the following errors on a loop:

Assertion failure at F3DAudioCheckCalculateParams (src/F3DAudio.c:379), triggered 1 time: 'pEmitter->InnerRadiusAngle <= 6.283185307f / 4.0f && "Value" "pEmitter->InnerRadiusAngle" "Is too big"'

Assertion failure at F3DAudioCalculate (src/F3DAudio.c:1491), triggered 1 time: '0 && "LPF_DIRECT not implemented!"

Assertion failure at F3DAudioCalculate (src/F3DAudio.c:1497), triggered 1 time: '0 && "LPF_REVERB not implemented!"

Assertion failure at F3DAudioCalculate (src/F3DAudio.c:1502), triggered 1 time: '0 && "REVERB not implemented!"

Assertion failure at FAudioVoice_SetOutputFilterParameters (src/FAudio.c:947), triggered 1 time: '0 && "Output filters are not supported!"'

If i hit "ignore permanently" I can continue playing the game without sound.

I would like to have this github issue to help track the progress of these features for myself and anyone else interested in using FAudio for skyrim. I have some C knowledge, and i would like to be pointed in the right direction for additional debugging info or paths forward for implementing these features.

[flibitijibibo]

The first assertion is #32, the very last is a per-output filter, and the rest are indeed F3DAudio, but this isn't just Skyrim, it's basically all the more obscure X3DAudio features.

Of the three issues, here are the most important:

1. WMA support. Look at Andrew's work and see if you can't get FFmpeg working with this format.
2. Output filters. Filters are already implemented but we have to have a filter for each send:

Filter type, filter function:

• https://github.com/FNA-XNA/FAudio/blob/master/src/FAudio_internal.h#L243
• https://github.com/FNA-XNA/FAudio/blob/master/src/FAudio_internal.c#L292

Voice filter:

• https://github.com/FNA-XNA/FAudio/blob/master/src/FAudio_internal.h#L291
• https://github.com/FNA-XNA/FAudio/blob/master/src/FAudio.c#L896

Output send data, SetOutputFilterParameters:

• https://github.com/FNA-XNA/FAudio/blob/master/src/FAudio_internal.h#L279
• https://github.com/FNA-XNA/FAudio/blob/master/src/FAudio.c#L940

3. X3DAudio. This is just a plain C library even for the MS version so for now I'm just saying use their version, but if you're extremely good at 3D math (we're talking top 0.1% worldwide) then you may be able to figure this out. For the matrix calculation it was a LOT of data plotting and guesswork only to get a similar-but-not-equal algorithm.

[kMaiSmith]

i see a lot of references to the term "voice" in the FAudio interface. Is a voice a sound or something more specific

[flibitijibibo]

You’re going to want to read the XAudio2 spec in full, grab it from the DXSDK:

https://www.microsoft.com/en-us/download/details.aspx?id=6812

[kMaiSmith]

So i've done a bit of digging and debugging and i have found Skyrim uses FAUDIO_FORMAT_EXTENSIBLE for all sounds that aren't music or character voices. The weird part is that those sounds aren't playing at all.

I added some debug assertions in each of the PCM decoder functions and it appears that those decoders never actually get called. So i added some debug assertions all the way up what looks like the decoder call tree (FAudio_INTERNAL_DecodeBuffers, FAudio_INTERNAL_MixSource, FAudio_INTERNAL_UpdateEngine) and each of those functions only get called once at the beginning of the initial loading screen. Where is the mechanism that is supposed to call the decoders when the buffers are updated and/or voices are added?

[flibitijibibo]

Are you sure you're not hitting 'always ignore' when you hit those functions...? Also the extensible format may still refer to WMA, so you'll want to dump the full format structure if you're not sure what's being used.

DecodeBuffers calls voice->src.decode, which is assigned when calling CreateSourceVoice.

[kMaiSmith]

I have not been hitting "always ignore." I think what I'm trying to find out is what calls FAudio_INTERNAL_DecodeBuffers. it is only referenced by FAudio_INTERNAL_MixSource, which is only referenced by FAudio_INTERNAL_UpdateEngine which is only referenced by FAudio_INTERNAL_MixCallback (in FAudio_platform_sdl2.h) which is only referenced in FAudio_PlatformInit which looks like it should only be called once when you initialize an FAudio.

I had put a debug assertion in each of those functions listed and they were all called when FAudio was initialized the first time. I'm clearly missing something about the FAudio/XAudio2 lifecycle here. Could the debug assertions i've put in being suppressed by being in a thread owned by the application referencing XAudio2? Is there a better way you'd recommend debugging this?

[flibitijibibo]

Could always printf, or whatever works best for however you get logs.

Keep in mind that the mix callback isn't where you want to focus, it's CreateSourceVoice. The decoder gets set based on the format, which it sounds like we don't handle, so do that first before even looking at the mixer. If that's not handled, it may just be the thread exiting from some kind of exception handler.

Consider making a Linux-native program that plays Skyrim's data as sample input and nothing else. Debugging this with all the layers between you and the mixer is probably too difficult.

[flibitijibibo]

Along with #32 I also have #42 and #43 filed as their own issues. This should cover everything that Skyrim cares about, so I'll close this in favor of using those threads instead.

[fls2018]

With Proton 3.16-6 beta and a compiled Faudio is working without audio crashing however I'm experiencing an echo/reverb issue in buildings most noticeable at dragons reach. Also volumes & panning gets a bit out of whack at times.