Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[IAST] skip dotnet 2.1 tests that can't work #6467

Merged
merged 1 commit into from
Dec 19, 2024

Conversation

robertpi
Copy link
Member

Summary of changes

Previous fix didn't work.

@robertpi robertpi requested a review from a team as a code owner December 19, 2024 13:43
@github-actions github-actions bot added the area:tests unit tests, integration tests label Dec 19, 2024
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Dec 19, 2024

Datadog Report

Branch report: robert/asm/skip-dotnet-20-tests-that-cant-work
Commit report: 9c4ba35
Test service: dd-trace-dotnet

❌ 3 Failed (0 Known Flaky), 461155 Passed, 2850 Skipped, 20h 21m 19.52s Total Time
❄️ 2 New Flaky

❌ Failed Tests (3)

  • StringConcatAspectBenchmark - Benchmarks.Trace.Iast.StringAspectsBenchmark - Details

  • StringConcatAspectBenchmark - Benchmarks.Trace.Iast.StringAspectsBenchmark - Details

  • StringConcatAspectBenchmark - Benchmarks.Trace.Iast.StringAspectsBenchmark - Details

New Flaky Tests (2)

  • AdditionalArguments - Datadog.Trace.Tools.dd_dotnet.ArtifactTests.RunCommandTests - Last Failure

    Expand for error
     
     
     With StandardOutput:
     Waiting - PID: 7884 - Main thread: 10024 - Profiler attached: True Args: traces 5 --dd-env test Sending 5 spans 
     With ErrorOutput:
      Unhandled Exception: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.DllNotFoundException: Unable to load DLL 'datadog_profiling_ffi': The specified module could not be found. (Exception from HRESULT: 0x8007007E)    at Datadog.Trace.LibDatadog.TraceExporterNative.ddog_trace_exporter_config_new(IntPtr& outHandle)    at Datadog.Trace.LibDatadog.TraceExporterConfiguration..ctor()    at Datadog.Trace.TracerManagerFactory.GetApi(TracerSettings settings, IDogStatsd statsd, Action\`1 updateSampleRates, IApiRequestFactory apiRequestFactory, Boolean partialFlushEnabled)    at Datadog.Trace.TracerManagerFactory.GetAgentWriter(TracerSettings settings, IDogStatsd statsd, Action\`1 updateSampleRates, IDiscoveryService discoveryService)    at Datadog.Trace.TracerManagerFactory.CreateTracerManager(TracerSettings settings, IAgentWriter agentWriter, ITraceSampler sampler, IScopeManager scopeManager, IDogStatsd statsd, RuntimeMetricsWriter runtimeMetrics, DirectLogSubmissionManager logSubmissionManager, ITelemetryController telemetry, IDiscoveryService discoveryService, DataStreamsManager dataStreamsManager, IRemoteConfigurationManager remoteConfigurationManager, IDynamicConfigurationManager dynamicConfigurationManager, ITracerFlareManager tracerFlareManager)    at Datadog.Trace.TracerManagerFactory.CreateTracerManager(TracerSettings settings, TracerManager previous)    at Datadog.Trace.TracerManager.CreateInitializedTracer(TracerSettings settings, TrayInitializer.EnsureInitializedCore[T](T& target, Boolean& initialized, Object& syncLock, Func\`1 valueFactory)    at Datadog.Trace.TracerManager.get_Instance()    at Datadog.Trace.Tracer..ctor(TracerManager tracerManager)    at Datadog.Trace.Tracer.get_Instance()    --- End of inner exception stack trace ---    at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)    at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)    at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)    at Samples.SampleHelpers.CreateScope(String operationName)    at Samples.Console_.Program.<AsyncMain>d__2.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Samples.Console_.Program.Main(String[] args) ".
    
  • AdditionalArguments - Datadog.Trace.Tools.dd_dotnet.ArtifactTests.RunCommandTests - Last Failure

    Expand for error
     
     
     With StandardOutput:
     Waiting - PID: 6648 - Main thread: 8168 - Profiler attached: True Args: traces 5 --dd-env test Sending 5 spans 
     With ErrorOutput:
      Unhandled Exception: System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.DllNotFoundException: Unable to load DLL 'datadog_profiling_ffi': The specified module could not be found. (Exception from HRESULT: 0x8007007E)    at Datadog.Trace.LibDatadog.TraceExporterNative.ddog_trace_exporter_config_new(IntPtr& outHandle)    at Datadog.Trace.LibDatadog.TraceExporterConfiguration..ctor()    at Datadog.Trace.TracerManagerFactory.GetApi(TracerSettings settings, IDogStatsd statsd, Action\`1 updateSampleRates, IApiRequestFactory apiRequestFactory, Boolean partialFlushEnabled)    at Datadog.Trace.TracerManagerFactory.GetAgentWriter(TracerSettings settings, IDogStatsd statsd, Action\`1 updateSampleRates, IDiscoveryService discoveryService)    at Datadog.Trace.TracerManagerFactory.CreateTracerManager(TracerSettings settings, IAgentWriter agentWriter, ITraceSampler sampler, IScopeManager scopeManager, IDogStatsd statsd, RuntimeMetricsWriter runtimeMetrics, DirectLogSubmissionManager logSubmissionManager, ITelemetryController telemetry, IDiscoveryService discoveryService, DataStreamsManager dataStreamsManager, IRemoteConfigurationManager remoteConfigurationManager, IDynamicConfigurationManager dynamicConfigurationManager, ITracerFlareManager tracerFlareManager)    at Datadog.Trace.TracerManagerFactory.CreateTracerManager(TracerSettings settings, TracerManager previous)    at Datadog.Trace.TracerManager.CreateInitializedTracer(TracerSettings settings, TracerManagerFactory factory)    at System.Threading.LazyInitializer.EnsureInitializedCore[T](T& target, Boolean& initialized, Object& syncLock, F   --- End of inner exception stack trace ---    at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)    at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)    at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)    at Samples.SampleHelpers.CreateScope(String operationName)    at Samples.Console_.Program.<AsyncMain>d__2.MoveNext() --- End of stack trace from previous location where exception was thrown ---    at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)    at Samples.Console_.Program.Main(String[] args) ".
    

@robertpi robertpi force-pushed the robert/asm/skip-dotnet-20-tests-that-cant-work branch from de72d03 to 9c4ba35 Compare December 19, 2024 14:14
@andrewlock
Copy link
Member

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6467) - mean (68ms)  : 65, 71
     .   : milestone, 68,
    master - mean (69ms)  : 65, 73
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (6467) - mean (977ms)  : 954, 1000
     .   : milestone, 977,
    master - mean (981ms)  : 957, 1005
     .   : milestone, 981,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6467) - mean (107ms)  : 104, 110
     .   : milestone, 107,
    master - mean (108ms)  : 105, 110
     .   : milestone, 108,

    section CallTarget+Inlining+NGEN
    This PR (6467) - mean (680ms)  : 666, 693
     .   : milestone, 680,
    master - mean (679ms)  : 666, 693
     .   : milestone, 679,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6467) - mean (91ms)  : 89, 93
     .   : milestone, 91,
    master - mean (91ms)  : 89, 93
     .   : milestone, 91,

    section CallTarget+Inlining+NGEN
    This PR (6467) - mean (635ms)  : 620, 650
     .   : milestone, 635,
    master - mean (637ms)  : 622, 651
     .   : milestone, 637,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6467) - mean (194ms)  : 189, 198
     .   : milestone, 194,
    master - mean (194ms)  : 189, 199
     .   : milestone, 194,

    section CallTarget+Inlining+NGEN
    This PR (6467) - mean (1,102ms)  : 1067, 1137
     .   : milestone, 1102,
    master - mean (1,100ms)  : 1064, 1136
     .   : milestone, 1100,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6467) - mean (279ms)  : 274, 283
     .   : milestone, 279,
    master - mean (278ms)  : 274, 282
     .   : milestone, 278,

    section CallTarget+Inlining+NGEN
    This PR (6467) - mean (872ms)  : 842, 903
     .   : milestone, 872,
    master - mean (870ms)  : 836, 905
     .   : milestone, 870,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6467) - mean (267ms)  : 262, 271
     .   : milestone, 267,
    master - mean (266ms)  : 262, 269
     .   : milestone, 266,

    section CallTarget+Inlining+NGEN
    This PR (6467) - mean (852ms)  : 819, 886
     .   : milestone, 852,
    master - mean (853ms)  : 825, 881
     .   : milestone, 853,

Loading

@link04 link04 merged commit ff3f016 into master Dec 19, 2024
59 of 64 checks passed
@link04 link04 deleted the robert/asm/skip-dotnet-20-tests-that-cant-work branch December 19, 2024 18:57
@github-actions github-actions bot added this to the vNext-v3 milestone Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:tests unit tests, integration tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants