Test failure: tracing_eventpipetrace._eventpipetrace_eventpipetrace_/_eventpipetrace_eventpipetrace_cmd

[ghost]

Opened on behalf of @Sunny-pu

The test tracing_eventpipetrace._eventpipetrace_eventpipetrace_/_eventpipetrace_eventpipetrace_cmd has failed.

Return code:      1
Raw output file:      C:\dotnetbuild\work\c3d2bb58-cd4b-483e-8169-07a5ef0dc304\Work\39854026-d90f-47cd-ab05-25e4708908cf\Unzip\Reports\tracing.eventpipetrace\eventpipetrace\eventpipetrace.output.txt
Raw output:
BEGIN EXECUTION\r
        1 file(s) copied.\r
" C:\dotnetbuild\work\c3d2bb58-cd4b-483e-8169-07a5ef0dc304\Payload\crossgen.exe" /Platform_Assemblies_Paths C:\dotnetbuild\work\c3d2bb58-cd4b-483e-8169-07a5ef0dc304\Payload;C:\dotnetbuild\work\c3d2bb58-cd4b-483e-8169-07a5ef0dc304\Work\39854026-d90f-47cd-ab05-25e4708908cf\Unzip\eventpipetrace\IL;C:\dotnetbuild\work\c3d2bb58-cd4b-483e-8169-07a5ef0dc304\Work\39854026-d90f-47cd-ab05-25e4708908cf\Unzip\eventpipetrace /in eventpipetrace.org /out eventpipetrace.exe\r
Microsoft (R) CoreCLR Native Image Generator - Version 4.5.22220.0\r
Copyright (c) Microsoft Corporation.  All rights reserved.\r
\r
Native image eventpipetrace.exe generated successfully.\r
 "C:\dotnetbuild\work\c3d2bb58-cd4b-483e-8169-07a5ef0dc304\Payload\corerun.exe" eventpipetrace.exe \r
\tStart: Enable tracing.\r
\tEnd: Enable tracing.
\r
\tStart: Generating CLR events\r
\tEnd: Generating CLR Events
\r
\tStart: Disable tracing.\r
\tEnd: Disable tracing.
\r
\tStart: Processing events from file.\r
\tEnd: Processing events from file.
\r
\tProcessed 1 GCAllocationTick events\r
\tProcessed 1 GCTriggered events\r
Expected: 100\r
Actual: 0\r
END EXECUTION - FAILED\r
FAILED\r
Test Harness Exitcode is : 1\r

To run the test:
> set CORE_ROOT=C:\dotnetbuild\work\c3d2bb58-cd4b-483e-8169-07a5ef0dc304\Payload
> C:\dotnetbuild\work\c3d2bb58-cd4b-483e-8169-07a5ef0dc304\Work\39854026-d90f-47cd-ab05-25e4708908cf\Unzip\eventpipetrace\eventpipetrace.cmd
\r
Expected: True\r
Actual:   False

    Stack Trace:

    null

Build : Master - 20180119.01 (Core Tests)
Failing configurations:

• Windows.7.Amd64
  • x64
    Details:
    https://mc.dot.net/#/product/netcore/master/source/official~2Fcoreclr~2Fmaster~2F/type/test~2Ffunctional~2Fcli~2F/build/20180119.01/workItem/tracing.eventpipetrace.XUnitWrapper/analysis/xunit/tracing_eventpipetrace._eventpipetrace_eventpipetrace_~2F_eventpipetrace_eventpipetrace_cmd

[tannergooding]

This is failing the JIT stress jobs. Looks to have been enabled yesterday: dotnet/coreclr#15917.

cc. @brianrob, @nategraf

[nategraf]

The test triggers GCs and looks to see if the GC Triggered event comes through. It looks like although it's supposed to have triggered 10 GCs and receive 10 events, it is only receiving 1 GC Triggered event.

@brianrob Do you know of any reason a GC may not occur when induced, or does this mean the event was dropped? Is there any way to tell if the event was dropped due to a full circular buffer?

[BruceForstall]

IMO, this test should be disabled, and then re-enabled only when it passes the failing stress jobs.

[sandreenko]

The test doesn't work without stress is well, it is just pri1, so we do not see how it fails.

[nategraf]

I'll get an issue and a PR to disable it together.

@sandreenko could you clarify your comment? Does it always fail for you? It ran and worked locally, and on the CI run for the PR which enabled it. Also, the test is pri0

[sandreenko]

@nategraf sorry, I was talking about eventsourcetrace. #9428 , that fails, but for some reasons is not disabled.

[nategraf]

Does this not disable it? https://github.com/dotnet/coreclr/blob/master/tests/issues.targets#L1729

[sandreenko]

I think it should, but this condition doesn't affect x86_checked_windows_nt_jitstress1_prtest for some reasons.

[nategraf]

I think I just realized what it is. The * should be a **

[brianrob]

@nategraf, thanks for fixing this!

[4creators]

@nategraf Unfortunately this failures are still happening specifically for Windows stress jobs:

https://ci.dot.net/job/dotnet_coreclr/job/master/job/jitstress/job/x64_checked_windows_nt_jitincompletehwintrinsic_prtest/26/
https://ci.dot.net/job/dotnet_coreclr/job/master/job/jitstress/job/x64_checked_windows_nt_jitx86hwintrinsicnoavx2_prtest/25/
https://ci.dot.net/job/dotnet_coreclr/job/master/job/jitstress/job/x64_checked_windows_nt_jitx86hwintrinsicnoavx_prtest/24/

https://ci.dot.net/job/dotnet_coreclr/job/master/job/jitstress/job/x86_checked_windows_nt_jitx86hwintrinsicnoavx_prtest/22/
https://ci.dot.net/job/dotnet_coreclr/job/master/job/jitstress/job/x86_checked_windows_nt_jitincompletehwintrinsic_prtest/23/
https://ci.dot.net/job/dotnet_coreclr/job/master/job/jitstress/job/x86_checked_windows_nt_jitx86hwintrinsicnosimd_prtest/24/

[nategraf]

Thanks @4creators. I'll figure out why the patch didn't fix the Windows stress runs and address it

[4creators]

@nategraf Thanks for help

[nategraf]

I see the JIT Stress runs are working now. Unless there is another place where this failure is showing I am going to close this issue

[tannergooding]

@nategraf. This is failing for Ubuntu jobs: https://ci.dot.net/job/dotnet_coreclr/job/master/job/jitstress/job/x64_checked_ubuntu_jitx86hwintrinsicnoavx_tst_prtest/16/testReport/

[tannergooding]

PR: dotnet/coreclr#15961

[4creators]

Now it is failing in Ubuntu and OSX, we cannot test CentOS since it does not listen to start commands.
PR dotnet/coreclr#15585

[4creators]

There are reproducible errors for: tracing/eventsourcetrace/eventsourcetrace/eventsourcetrace.sh when running jitstress jobs for HW intrinsics:

https://ci.dot.net/job/dotnet_coreclr/job/master/job/jitstress/job/x64_checked_ubuntu_jitx86hwintrinsicnoavx_tst_prtest/17/

https://ci.dot.net/job/dotnet_coreclr/job/master/job/jitstress/job/x64_checked_osx10.12_jitincompletehwintrinsic_tst_prtest/12/
https://ci.dot.net/job/dotnet_coreclr/job/master/job/jitstress/job/x64_checked_osx10.12_jitx86hwintrinsicnosimd_tst_prtest/11/
https://ci.dot.net/job/dotnet_coreclr/job/master/job/jitstress/job/x64_checked_osx10.12_jitx86hwintrinsicnoavx2_flow_prtest/11/

Tests can be triggered with commands as in comment dotnet/coreclr#15585 (comment)