Some SoundSourceProxyTests fail on Win11 with VS2022

[JoergAtGithub]

Bug Description

I build the same versions of Main (9965d55) and PR #11074 on two Windows laptops. On the older one with Win7 and VS2019 all tests passed.
But on the newer one with Win11 and VS2022 four tests failed:

SoundSourceProxyTest.seekForwardBackward 
SoundSourceProxyTest.seekBoundaries
SoundSourceProxyTest.readBeyondEnd
SoundSourceProxyTest.regressionTestCachingReaderChunkJumpForward

The test SoundSourceProxyTest.seekForwardBackward generated ~2GByte of similar log messages, here is shortened version of the log file:
LastTest_soundsourceproxytestsfailed_newbuilenv.log

Version

Main

OS

Windows11 + VS2022

[uklotzde]

SoundSourceMediaFoundation is hardly working due to lack of Windows developers/testers and might fail when Microsoft decides to either change the API or behavior.

The API itself is not really suitable for sample-accurate decoding.

[uklotzde]

https://www.reddit.com/r/DJs/comments/z9f7yf/comment/iygulke/

https://community.native-instruments.com/discussion/7760/3-7-0-336-rc-uploaded

Known Issues that we’re still working on:

• Audio glitches on M4A files introduced by the latest Windows 11 update.

[daschuer]

Does the issue move with the binary, or is it an issue of the windows version running on?

Since this issue makes the cue points and waveforms unreliable we may consider to retire MediaFoundation and move to FFMPEG for aac/mp4/m4a decoding.

Unfortunately there is a high probability that the cues and waveforms will be shifted for all such files after the switch.

To be aware of that I have prepared:
#4773
waiting for merge.

A next step would an automatic adjustment of these offsets.

[JoergAtGithub]

I just updated to the latest Windows11 update - these tests still fail.

[daschuer]

Without rebuilding?

[JoergAtGithub]

I now copied all .exe and .dll files from the build directory of my Windows7 laptop to the build directory of my Windows11 laptopand ran CTest.
On Windows7 all tests with these executables were pass, on Windows11 the 4 SoundSourceProxyTests failed.

Conclusion: It's an unfixed OS bug in Windows11 and not a Mixxx bug.

[daschuer]

Great, thank you. for investigations.
Does anyone know how to report a bug upstream? Even if there is a chance that it is fixed some day, we will have users suffering this issue.

[JoergAtGithub]

These tests still fail, using latest Win11.

[daschuer]

@JoergAtGithub: Does Windows 11 find the Sound Start Cue created in Windows < 11? Just play one or more m4a file you had in your Windows before the OS Update.

and look for First sound has been moved!

Than please Build Mixxx with ffmpeg

mixxx/src/sources/soundsourceffmpeg.cpp

Line 470 in a547664

return SoundSourceProviderPriority::Lowest;

= Highest

And verify it again.

[daschuer]

The Test failure is reproducible on the GitHub Workflows using Windows-2022:
https://github.com/daschuer/mixxx/actions/runs/5855464923/job/15873332041

[JoergAtGithub]

The Test failure is reproducible on the GitHub Workflows using Windows-2022

That means that also Windows Server 2022 is affected: https://github.com/actions/runner-images/blob/main/images/win/Windows2022-Readme.md

[daschuer]

As expected the windows-2022 build with FFmpeg succeeds https://github.com/daschuer/mixxx/actions/runs/5865154501/job/15901524981

The crucial question is if the MediaFoundation Windows 10 decoded files match FFmpeg decoded files perectly, or if we need to consider a cue point shift.

@JoergAtGithub: did you found time to check for the "First sound has been moved!" log entry with m4a files? Interesting would bee if you see that in the FFmpeg build linked above.

I think we should establish a unit test for it.

[daschuer]

#11887 Discovered that moving to ffmpeg is not a trivial solution, because there is a timing offset between the FFmpeg and Media Foindation that varies across tracks.

@JoergAtGithub I am now clueless how to continue.

Since I am not on windows I have hard time to investigate that issue further.

It looks like Media Foundation 10.0.20348.1 introduces an offset after seeking. The question is if this offset is constant so that we can compensate that within Media Foundation.

Can you test that with a track with a perfect beat grid before the update and test it on Win11?
Has the track always the same offset after the first seek?

Is the issue notable?

Is it an option to release Mixxx 2.4 with this issue included? Shall we move to FFmpeg and force a reanalysis for m4a tracks?

Ideas?

[daschuer]

To continue here we need a Windows user with a Windows 11 machine that has a significant amount (> 10) m4a tracks analysed with WIN10 or before. Who can help here?

[JoergAtGithub]

I could to the analysis task, since I've both, a Win7 and Win11 laptop with latest Mixxx builds. But I don't have any m4a tracks. Does anybody know, where I could download suitable test files?

[daschuer]

Thank you. You can just convert a few. I think Audacity can do it. It uses Ffmpeg.

[daschuer]

It still fails with Windows 2022:
https://github.com/daschuer/mixxx/actions/runs/7285956895/job/19853789720

[daschuer]

The Preview Windows 10 is also failing :-(

10.0.26016.1000 fail
10.0.22621.2506 fail
10.0.22000.1880 fail
10.0.20348.1fail
10.0.20348.2031fail
10.0.20348.2655 fail
10.0.19041.3636 fail
10.0.19041.746 fail

10.0.17763.2989 OK

[daschuer]

Tested with: #12289 (comment)

[daschuer]

Do we have a working Windows version in the meantime?

[JoergAtGithub]

No, this test is not fixed in any Windows 11 version known to me.

[EchoCoder1729]

How are we planning to fox this? Any leads so that I can take this up and maybe fix this test? I am ready to dive into the Windows 11 Sound Driver API calls if necessary. But if this test is going to be breaking across windows versions, then doesn't make much sense to work on it though. Just to clarify, each platform (Linux/ Windows) uses their own test suites or do they run the same tests?

[JoergAtGithub]

The soundsource code tested here is platform specific. In general we have only one test suite.

[daschuer]

I have just pushed a new CI build to check the current situation with Windows 2022
https://github.com/daschuer/mixxx/actions/runs/10733009450

It is Windows Server 2022, Version 21H2: 10.0.20348 Build 2655
https://github.com/actions/runner-images/blob/main/images/windows/Windows2022-Readme.md

[daschuer]

No :-(
https://github.com/daschuer/mixxx/actions/runs/10733009450/job/29765629719

[EchoCoder1729]

Do we have any environment variable to specify that it is a windows machine and disable the specific test in case windows? Is it that we need to modify the test or any changes would break the test in previous versions like windows 7?

[daschuer]

Windows is really broken here. So the failure of the test is correct. You may disable/skip the test if you like.