Latest NET 9 runtime causes EventCounterPipelineUnitTests.TestCounterEventPipeline test to fail

[mikem8361]

The Microsoft.Diagnostics.Monitoring.EventPipe.UnitTests.EventCounterPipelineUnitTests.TestCounterEventPipeline test in the diagnostics repo is failing on the latest NET 9.0 runtime on almost all platforms with:

Assert.Equal() Failure: Collections differ\n ↓ (pos 0)\nExpected: string[] ["cpu-usage", "working-set"]\nActual: OrderedEnumerable<string, string> [null, "cpu-usage", "working-set"]\n ↑ (pos 0)

Stack trace
at Microsoft.Diagnostics.Monitoring.EventPipe.UnitTests.EventCounterPipelineUnitTests.TestCounterEventPipeline(TestConfiguration config) in /Users/runner/work/1/s/src/tests/Microsoft.Diagnostics.Monitoring.EventPipe/EventCounterPipelineUnitTests.cs:line 129 --- End of stack trace from previous location ---

This failure is causing all the DARC updates to fail so this test is disabled.

https://dev.azure.com/dnceng-public/public/_build/results?buildId=772212&view=results

#4845

[noahfalk]

@dotnet/dotnet-monitor - I investigated and the test failure is a combination of three things:

1. Change EventSource startup initialization runtime#106014 in .NET runtime means that MetricsEventSource is now accessible for all apps at startup. This enables tools such as dotnet-counters and dotnet-monitor to observe the new runtime metrics in any app, such as a trivial HelloWorld-style console app being used in this test. This is the change that caused the test to start failing but it is by design with no plan to revert it.

2. When the CountersPipeline is run, the value for Settings.MaxTimeSeries is never initialized so it remains at zero. The test enables MetricsEventSource with this zero setting. The presence of the runtime metrics + zero limit for TimeSeries causes MetricsEventSource to emit an error event indicating the time series limit was hit.

3. The test code which validates the received payloads didn't do anything to filter out the error payload so it shows up in the list as the null counter causing the list not to match what the test expects.

The unexpected payload printed in VS:

logger.Metrics.ToArray()[2]
{Microsoft.Diagnostics.Monitoring.EventPipe.ErrorPayload}
CounterMetadata: {Microsoft.Diagnostics.Monitoring.EventPipe.CounterMetadata}
CounterType: Metric
DisplayName: ""
ErrorMessage: "Warning: Time series tracking limit (0) reached. Not all data is being shown. The limit can be changed but will use more memory in the target process."
EventType: TimeSeriesLimitError
Interval: 0
IsMeter: true
Series: 0x00000000
Timestamp: {8/12/2024 12:01:24 PM}
Unit: ""
Value: 0
ValueTags: null

I assume the test should either not enable MetricsEventSource, not set the TimeSeries limit to zero, or apply a filter to the final results to remove the expected error message.

[tommcdon]

Fixed by #4871