Integration tests fail to wait for code action application (or silent failure)

[sharwell]

https://dev.azure.com/dnceng/public/_build/results?buildId=1432244

Xunit.Sdk.ContainsException: Assert.Contains() Failure
Not found: ClassLibrary3
In value:  String[] ["ClassLibrary1", "ClassLibrary2"]
   at Xunit.Assert.Contains[T](T expected, IEnumerable`1 collection, IEqualityComparer`1 comparer) in C:\projects\xunit\src\xunit.assert\Asserts\CollectionAsserts.cs:line 114

Build	Kind	Start Time
1425665	Rolling	2021-17-10
1426510	Rolling	2021-18-10
1428386	Rolling	2021-18-10
1428256	Rolling	2021-18-10
1430603	Rolling	2021-19-10
1430869	Rolling	2021-19-10
1432244	Rolling	2021-20-10
1434178	PR 57029	2021-21-10
1434386	PR 57300	2021-21-10
1434538	PR 55469	2021-21-10
1434564	PR 57305	2021-21-10
1434504	PR 57084	2021-21-10
1434626	PR 57306	2021-21-10
1434813	PR 57254	2021-21-10
1434988	PR 57284	2021-21-10
1435049	PR 57310	2021-21-10
1435242	PR 57312	2021-21-10
1435296	PR 57303	2021-21-10
1435190	PR 57290	2021-21-10
1435421	PR 57228	2021-21-10
1435484	PR 56276	2021-22-10
1436064	PR 57029	2021-22-10
1436110	PR 56197	2021-22-10
1436306	PR 54559	2021-22-10
1436419	PR 45105	2021-22-10
1436477	Rolling	2021-22-10
1436429	Rolling	2021-22-10
1436565	PR 57266	2021-22-10
1436574	Rolling	2021-22-10
1436611	Rolling	2021-22-10
1440184	Rolling	2021-26-10
1440339	PR 56957	2021-26-10
1440379	PR 57380	2021-26-10

[jasonmalinowski]

Any chance this is fallout from the async lightbulb issues? @CyrusNajmabadi?

[CyrusNajmabadi]

@jasonmalinowski discussing this with @AmadeusW now. He's observed this on their end as well. There are times when roslyn returns results, but they don't display. It's one of the reasons we're disabling async-lightbulbs for 17.0.

I hope we can track this down and fix early in the 17.1 timeframe.

[jasonmalinowski]

@CyrusNajmabadi Alright, good to know there's a chance it's related.

[CyrusNajmabadi]

@sharwell @jasonmalinowski are we still seeing this with the latest bits from platform?

[sharwell]

@CyrusNajmabadi I never saw clear evidence that this issue was caused by the platform. It seemed more related to Windows Defender triggering file change notifications at unexpected times, and is mitigated in the new integration test library by a79d0e0 (once merged).

[CyrusNajmabadi]

Ok. I'm unassigning me then :)

[jasonmalinowski]

@CyrusNajmabadi @sharwell Should we just close this then? Or if there's still pending work, what is it?