Test failure: profiler/eventpipe/eventpipe/eventpipe.sh

[BruceForstall]

OSX, x64, Checked, JitStress=1

https://dev.azure.com/dnceng/public/_build/results?buildId=737405&view=ms.vss-test-web.build-test-results-tab&runId=22835648&resultId=100002&paneView=dotnet-dnceng.dnceng-build-release-tasks.helix-test-information-tab

    profiler/eventpipe/eventpipe/eventpipe.sh [FAIL]
      libc++abi.dylib: terminating with uncaught exception of type PAL_SEHException
      Unhandled exception. System.Exception: Profiler tests are expected to contain the text 'PROFILER TEST PASSES' in the console output of the profilee app to indicate a passing test. Usually it is printed from the Shutdown() method of the profiler implementation. This text was not found in the output above.
         at Profiler.Tests.ProfilerTestRunner.LogTestFailure(String error) in /Users/runner/work/1/s/src/tests/profiler/common/ProfilerTestRunner.cs:line 136
         at Profiler.Tests.ProfilerTestRunner.Run(String profileePath, String testName, Guid profilerClsid, String profileeArguments, ProfileeOptions profileeOptions) in /Users/runner/work/1/s/src/tests/profiler/common/ProfilerTestRunner.cs:line 90
         at EventPipeTests.EventPipe.Main(String[] args) in /Users/runner/work/1/s/src/tests/profiler/eventpipe/eventpipe.cs:line 31
      task_for_pid(38717) FAILED 5 (os/kern) failure
      /private/tmp/helix/working/B57F094D/w/A40A08EC/e/profiler/eventpipe/eventpipe/eventpipe.sh: line 333: 38717 Abort trap: 6           $LAUNCHER $ExePath "${CLRTestExecutionArguments[@]}"
      
      Return code:      1

@janvorli looks like a PAL / OS failure?

[Dotnet-GitSync-Bot]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[sywhang]

cc @davmason

[ghost]

Tagging subscribers to this area: @tommcdon
Notify danmosemsft if you want to be subscribed.

[v-haren]

failed again in job: runtime-coreclr jitstressregs 20200913.1

failed test: profiler/eventpipe/eventpipe/eventpipe.sh

CoreCLR OSX x64 Checked jitstressregs8 @ OSX.1013.Amd64.Open

Error message

libc++abi.dylib: terminating with uncaught exception of type PAL_SEHException

Unhandled exception. System.Exception: Profiler tests are expected to contain the text 'PROFILER TEST PASSES' in the console output of the profilee app to indicate a passing test. Usually it is printed from the Shutdown() method of the profiler implementation. This text was not found in the output above.
 at Profiler.Tests.ProfilerTestRunner.LogTestFailure(String error) in /__w/1/s/src/tests/profiler/common/ProfilerTestRunner.cs:line 138
   at Profiler.Tests.ProfilerTestRunner.Run(String profileePath, String testName, Guid profilerClsid, String profileeArguments, ProfileeOptions profileeOptions) in /__w/1/s/src/tests/profiler/common/ProfilerTestRunner.cs:line 92
   at EventPipeTests.EventPipe.Main(String[] args) in /__w/1/s/src/tests/profiler/eventpipe/eventpipe.cs:line 31
MachO coredumps are not supported. To enable ELF coredumps on MacOS, set the COMPlus_DbgEnableElfDumpOnMacOS environment variable to 1.
/private/tmp/helix/working/B84609E1/w/ABDD095B/e/profiler/eventpipe/eventpipe/eventpipe.sh: line 363: 77146 Abort trap: 6           (core dumped) $LAUNCHER $ExePath "${CLRTestExecutionArguments[@]}"

Return code:      1
Raw output file:      /private/tmp/helix/working/B84609E1/w/ABDD095B/e/profiler/eventpipe/Reports/profiler.eventpipe/eventpipe/eventpipe.output.txt
Raw output:
BEGIN EXECUTION
/tmp/helix/working/B84609E1/p/corerun eventpipe.dll ''
Profiler path: /private/tmp/helix/working/B84609E1/w/ABDD095B/e/profiler/eventpipe/eventpipe/libProfiler.dylib
Profiler.dll!DllGetClassObject
Profiler.dll!Profiler::Initialize
This method being jitted should trigger events firing...
TriggerMethod found! Sending event...

Test failed: Profiler tests are expected to contain the text 'PROFILER TEST PASSES' in the console output of the profilee app to indicate a passing test. Usually it is printed from the Shutdown() method of the profiler implementation. This text was not found in the output above.
Expected: 100
Actual: 134
END EXECUTION - FAILED
Test Harness Exitcode is : 1
To run the test:
> set CORE_ROOT=/tmp/helix/working/B84609E1/p
> /private/tmp/helix/working/B84609E1/w/ABDD095B/e/profiler/eventpipe/eventpipe/eventpipe.sh
Expected: True
Actual:   False


Stack trace
   at profiler_eventpipe._eventpipe_eventpipe_._eventpipe_eventpipe_sh() in /Users/runner/work/1/s/artifacts/tests/coreclr/OSX.x64.Checked/TestWrappers/profiler.eventpipe/profiler.eventpipe.XUnitWrapper.cs:line 140

[ViktorHofer]

Failed again in https://dev.azure.com/dnceng/public/_build/results?buildId=905607&view=ms.vss-test-web.build-test-results-tab&runId=28901910&resultId=100002&paneView=debug. Unsure if this is the same cause?

Dumps are logged as well.

[davmason]

It doesn't look like the same failure. Looking at the dumps, I don't even see a problem. It just looks like it's executing normally.

00 (Inline Function) --------`--------     libcoreclr!CMiniMdBase::getIX+0x20 [/__w/1/s\src/coreclr/src/md/inc/metamodel.h @ 479] 
01 (Inline Function) --------`--------     libcoreclr!CMiniMdTemplate<CMiniMd>::getResolutionScopeOfTypeRef+0x2c [/__w/1/s\src/coreclr/src/md/inc/metamodel.h @ 1725] 
02 0000007f`e50c2500 0000007f`a4edd1f4     libcoreclr!MDInternalRO::GetResolutionScopeOfTypeRef+0x84 [/__w/1/s\src/coreclr/src/md/runtime/mdinternalro.cpp @ 1131] 
03 0000007f`e50c2530 0000007f`a4edc710     libcoreclr!PrettyPrintClass+0x1d8 [/__w/1/s\src/coreclr/src/vm/../inc/formattype.cpp @ 799] 
04 0000007f`e50c29a0 0000007f`a4eda758     libcoreclr!PrettyPrintType+0xa1c [/__w/1/s\src/coreclr/src/vm/../inc/formattype.cpp @ 546] 
05 (Inline Function) --------`--------     libcoreclr!PrettyPrintTypeOrDef+0x10 [/__w/1/s\src/coreclr/src/vm/../inc/formattype.cpp @ 444] 
06 0000007f`e50c2eb0 0000007f`a4ed99c8     libcoreclr!PrettyPrintSignature+0xaf0 [/__w/1/s\src/coreclr/src/vm/../inc/formattype.cpp @ 361] 
07 0000007f`e50c35a0 0000007f`a4f8307c     libcoreclr!PrettyPrintSig+0x170 [/__w/1/s\src/coreclr/src/vm/../inc/formattype.cpp @ 161] 
08 0000007f`e50c3670 0000007f`a4f83544     libcoreclr!FormatSigHelper+0x254 [/__w/1/s\src/coreclr/src/vm/sigformat.cpp @ 15732480] 
09 0000007f`e50c37a0 0000007f`a50c2a20     libcoreclr!FormatSig+0x3c [/__w/1/s\src/coreclr/src/vm/sigformat.cpp @ 15732480] 
0a 0000007f`e50c39e0 0000007f`a50b65ec     libcoreclr!MethodTableBuilder::SetupMethodTable2+0xa5c [/__w/1/s\src/coreclr/src/vm/methodtablebuilder.cpp @ 10286] 
0b 0000007f`e50c3b50 0000007f`a50cff34     libcoreclr!MethodTableBuilder::BuildMethodTableThrowing+0xf4c [/__w/1/s\src/coreclr/src/vm/methodtablebuilder.cpp @ 1794] 
0c 0000007f`e50c4230 0000007f`a4e7bf1c     libcoreclr!ClassLoader::CreateTypeHandleForTypeDefThrowing+0xc44 [/__w/1/s\src/coreclr/src/vm/methodtablebuilder.cpp @ 12220] 
0d 0000007f`e50c65b0 0000007f`a4e7b9b8     libcoreclr!ClassLoader::CreateTypeHandleForTypeKey+0x32c [/__w/1/s\src/coreclr/src/vm/clsload.cpp @ 3197] 
0e 0000007f`e50c66d0 0000007f`a4e7d428     libcoreclr!ClassLoader::DoIncrementalLoad+0x130 [/__w/1/s\src/coreclr/src/vm/clsload.cpp @ 3126] 
0f 0000007f`e50c67d0 0000007f`a4e77ee8     libcoreclr!ClassLoader::LoadTypeHandleForTypeKey_Body+0x6ec [/__w/1/s\src/coreclr/src/vm/clsload.cpp @ 3895] 
10 0000007f`e50c6940 0000007f`a4e79750     libcoreclr!ClassLoader::LoadTypeHandleForTypeKey+0x14c [/__w/1/s\src/coreclr/src/vm/clsload.cpp @ 3613] 
11 0000007f`e50c69e0 0000007f`a4e7ab20     libcoreclr!ClassLoader::LoadTypeDefThrowing+0x250 [/__w/1/s\src/coreclr/src/vm/clsload.cpp @ 2501] 
12 0000007f`e50c6b20 0000007f`a4f84df0     libcoreclr!ClassLoader::LoadTypeDefOrRefThrowing+0x31c [/__w/1/s\src/coreclr/src/vm/clsload.cpp @ 2673] 
13 0000007f`e50c6c00 0000007f`a4f85adc     libcoreclr!SigPointer::GetTypeHandleThrowing+0x700 [/__w/1/s\src/coreclr/src/vm/siginfo.cpp @ 15732480] 
14 0000007f`e50c6cf0 0000007f`a4ed7d64     libcoreclr!SigPointer::GetTypeHandleThrowing+0x13ec [/__w/1/s\src/coreclr/src/vm/siginfo.cpp @ 15732480] 
15 (Inline Function) --------`--------     libcoreclr!MetaSig::GetLastTypeHandleThrowing+0x24 [/__w/1/s\src/coreclr/src/vm/siginfo.hpp @ 15732480] 
16 0000007f`e50c6df0 0000007f`a4f50c90     libcoreclr!FieldDesc::GetFieldTypeHandleThrowing+0x6c [/__w/1/s\src/coreclr/src/vm/field.cpp @ 191] 
17 0000007f`e50c6ea0 0000007f`a4f509a4     libcoreclr!MethodTable::DoFullyLoad+0x9a4 [/__w/1/s\src/coreclr/src/vm/methodtable.cpp @ 5418] 
18 0000007f`e50c6fe0 0000007f`a4e7ddf0     libcoreclr!MethodTable::DoFullyLoad+0x6b8 [/__w/1/s\src/coreclr/src/vm/methodtable.cpp @ 15732480] 
19 0000007f`e50c7120 0000007f`a4e77f40     libcoreclr!PushFinalLevels+0x64 [/__w/1/s\src/coreclr/src/vm/clsload.cpp @ 3532] 
1a 0000007f`e50c71d0 0000007f`a4e79750     libcoreclr!ClassLoader::LoadTypeHandleForTypeKey+0x1a4 [/__w/1/s\src/coreclr/src/vm/clsload.cpp @ 3623] 
1b 0000007f`e50c7270 0000007f`a4e7ab20     libcoreclr!ClassLoader::LoadTypeDefThrowing+0x250 [/__w/1/s\src/coreclr/src/vm/clsload.cpp @ 2501] 
1c 0000007f`e50c73b0 0000007f`a4f84df0     libcoreclr!ClassLoader::LoadTypeDefOrRefThrowing+0x31c [/__w/1/s\src/coreclr/src/vm/clsload.cpp @ 2673] 
1d 0000007f`e50c7490 0000007f`a4efd4b0     libcoreclr!SigPointer::GetTypeHandleThrowing+0x700 [/__w/1/s\src/coreclr/src/vm/siginfo.cpp @ 15732480] 
1e 0000007f`e50c7580 0000007f`a4f0d8a0     libcoreclr!CEEInfo::ConvToJitSig+0x2f0 [/__w/1/s\src/coreclr/src/vm/jitinterface.cpp @ 548] 
1f 0000007f`e50c7620 0000007f`a4f0cddc     libcoreclr!CEEInfo::getMethodSigInternal+0x16c [/__w/1/s\src/coreclr/src/vm/jitinterface.cpp @ 8632] 
20 0000007f`e50c76c0 0000007f`a1357d44     libcoreclr!CEEInfo::getCallInfo+0x195c [/__w/1/s\src/coreclr/src/vm/jitinterface.cpp @ 5734] 
21 (Inline Function) --------`--------     libclrjit!Compiler::eeGetCallInfo+0x40 [/__w/1/s\src/coreclr/src/jit/importer.cpp @ 15732480] 
22 0000007f`e50c7a60 0000007f`a136389c     libclrjit!Compiler::impImportBlockCode+0x669c [/__w/1/s\src/coreclr/src/jit/importer.cpp @ 15732480] 
23 (Inline Function) --------`--------     libclrjit!<unnamed-class>::operator()+0x24 [/__w/1/s\src/coreclr/src/jit/importer.cpp @ 17597] 
24 0000007f`e50c7fd0 0000007f`a136639c     libclrjit!Compiler::impImportBlock+0x1a0 [/__w/1/s\src/coreclr/src/jit/importer.cpp @ 17602] 
25 0000007f`e50c80a0 0000007f`a12ce948     libclrjit!Compiler::impImport+0x568 [/__w/1/s\src/coreclr/src/jit/importer.cpp @ 18692] 
26 0000007f`e50c8100 0000007f`a142eed0     libclrjit!Compiler::fgImport+0x18 [/__w/1/s\src/coreclr/src/jit/flowgraph.cpp @ 15732480] 
27 0000007f`e50c8130 0000007f`a129f7e4     libclrjit!Phase::Run+0x84 [/__w/1/s\src/coreclr/src/jit/phase.cpp @ 62] 
28 (Inline Function) --------`--------     libclrjit!DoPhase+0x3c [/__w/1/s\src/coreclr/src/jit/phase.h @ 118] 
29 0000007f`e50c8180 0000007f`a12a3210     libclrjit!Compiler::compCompile+0xb0 [/__w/1/s\src/coreclr/src/jit/compiler.cpp @ 4271] 
2a 0000007f`e50c8470 0000007f`a12a1d4c     libclrjit!Compiler::compCompileHelper+0x8ec [/__w/1/s\src/coreclr/src/jit/compiler.cpp @ 6187] 
2b (Inline Function) --------`--------     libclrjit!<unnamed-class>::operator()+0x18 [/__w/1/s\src/coreclr/src/jit/compiler.hpp @ 3957] 
2c 0000007f`e50c8540 0000007f`a12a3eb4     libclrjit!Compiler::compCompile+0x68c [/__w/1/s\src/coreclr/src/jit/compiler.cpp @ 5501] 
2d (Inline Function) --------`--------     libclrjit!<unnamed-class>::operator()+0x160 [/__w/1/s\src/coreclr/src/jit/compiler.cpp @ 6835] 
2e (Inline Function) --------`--------     libclrjit!<unnamed-class>::operator()+0x164 [/__w/1/s\src/coreclr/src/jit/compiler.cpp @ 6855] 
2f 0000007f`e50c85b0 0000007f`a12aea28     libclrjit!jitNativeCode+0x268 [/__w/1/s\src/coreclr/src/jit/compiler.cpp @ 6857] 
30 0000007f`e50c8be0 0000007f`a4f26098     libclrjit!CILJit::compileMethod+0x124 [/__w/1/s\src/coreclr/src/jit/ee_il_dll.cpp @ 15732480] 
31 0000007f`e50c8c90 0000007f`a4f26278     libcoreclr!invokeCompileMethodHelper+0x104 [/__w/1/s\src/coreclr/src/vm/jitinterface.cpp @ 12384] 
32 (Inline Function) --------`--------     libcoreclr!invokeCompileMethod+0x28 [/__w/1/s\src/coreclr/src/vm/jitinterface.cpp @ 15732480] 
33 (Inline Function) --------`--------     libcoreclr!<unnamed-class>::operator()+0x28 [/__w/1/s\src/coreclr/src/vm/jitinterface.cpp @ 12503] 
34 0000007f`e50c8ce0 0000007f`a4f271cc     libcoreclr!CallCompileMethodWithSEHWrapper+0x94 [/__w/1/s\src/coreclr/src/vm/jitinterface.cpp @ 12533] 
35 0000007f`e50c8d70 0000007f`a4f754ac     libcoreclr!UnsafeJitFunction+0x848 [/__w/1/s\src/coreclr/src/vm/jitinterface.cpp @ 15732480] 
36 0000007f`e50c9900 0000007f`a4f74ed4     libcoreclr!MethodDesc::JitCompileCodeLocked+0x294 [/__w/1/s\src/coreclr/src/vm/prestub.cpp @ 1019] 
37 0000007f`e50c9a40 0000007f`a4f74424     libcoreclr!MethodDesc::JitCompileCodeLockedEventWrapper+0x33c [/__w/1/s\src/coreclr/src/vm/prestub.cpp @ 887] 
38 0000007f`e50c9b30 0000007f`a4f73968     libcoreclr!MethodDesc::JitCompileCode+0x394 [/__w/1/s\src/coreclr/src/vm/prestub.cpp @ 15732480] 
39 0000007f`e50c9c00 0000007f`a4e85bf8     libcoreclr!MethodDesc::PrepareILBasedCode+0x100 [/__w/1/s\src/coreclr/src/vm/prestub.cpp @ 408] 
3a 0000007f`e50c9ca0 0000007f`a4f78468     libcoreclr!CodeVersionManager::PublishVersionableCodeIfNecessary+0x2b0 [/__w/1/s\src/coreclr/src/vm/codeversion.cpp @ 1704] 
3b 0000007f`e50c9e10 0000007f`a4f77aa8     libcoreclr!MethodDesc::DoPrestub+0x438 [/__w/1/s\src/coreclr/src/vm/prestub.cpp @ 2170] 
3c 0000007f`e50c9f10 0000007f`a522ce08     libcoreclr!PreStubWorker+0x89c [/__w/1/s\src/coreclr/src/vm/prestub.cpp @ 15732480] 
3d 0000007f`e50ca040 0000007f`2beb5170     libcoreclr!ThePreStub+0x50
3e 0000007f`e50ca170 0000007f`a522d94c     0x0000007f`2beb5170
3f 0000007f`e50ca180 0000007f`a5000c40     libcoreclr!CallDescrWorkerInternal+0x84
40 0000007f`e50ca1d0 0000007f`a5001590     libcoreclr!CallDescrWorkerWithHandler+0xfc [/__w/1/s\src/coreclr/src/vm/callhelpers.cpp @ 73] 
41 0000007f`e50ca220 0000007f`a4e3f664     libcoreclr!MethodDescCallSite::CallTargetWorker+0x5e0 [/__w/1/s\src/coreclr/src/vm/callhelpers.cpp @ 547] 
42 (Inline Function) --------`--------     libcoreclr!MethodDescCallSite::Call_RetArgSlot+0x60 [/__w/1/s\src/coreclr/src/vm/callhelpers.h @ 458] 
43 (Inline Function) --------`--------     libcoreclr!RunMainInternal+0x230 [/__w/1/s\src/coreclr/src/vm/assembly.cpp @ 1468] 
44 (Inline Function) --------`--------     libcoreclr!<unnamed-class>::operator()+0x230 [/__w/1/s\src/coreclr/src/vm/assembly.cpp @ 1536] 
45 (Inline Function) --------`--------     libcoreclr!<unnamed-class>::operator()+0x2fc [/__w/1/s\src/coreclr/src/vm/assembly.cpp @ 1538] 
46 0000007f`e50ca4a0 0000007f`a4e3fa38     libcoreclr!RunMain+0x444 [/__w/1/s\src/coreclr/src/vm/assembly.cpp @ 1538] 
47 0000007f`e50ca6f0 0000007f`a4e9abf4     libcoreclr!Assembly::ExecuteMainMethod+0x228 [/__w/1/s\src/coreclr/src/vm/assembly.cpp @ 15732480] 
48 0000007f`e50ca9a0 0000007f`a4e2048c     libcoreclr!CorHost2::ExecuteAssembly+0x320 [/__w/1/s\src/coreclr/src/vm/corhost.cpp @ 385] 
49 0000007f`e50caad0 00000055`8a7716c4     libcoreclr!coreclr_execute_assembly+0x160 [/__w/1/s\src/coreclr/src/dlls/mscoree/unixinterface.cpp @ 15732480] 
4a 0000007f`e50cab80 00000055`8a771da8     corerun!ExecuteManagedAssembly+0x548 [/__w/1/s\src/coreclr/src/hosts/unixcorerun/corerun.cpp @ 15732480] 
4b 0000007f`e50cad20 0000007f`a57476e0     corerun!main+0x1a0 [/__w/1/s\src/coreclr/src/hosts/unixcorerun/corerun.cpp @ 15732480] 
4c 0000007f`e50cbe70 00000000`00000000     libc_2_27+0x206e0

[benaadams]

Seen in #45699 runtime (CoreCLR Pri0 Runtime Tests Run Linux x64 checked

https://dev.azure.com/dnceng/public/_build/results?buildId=911975&view=ms.vss-test-web.build-test-results-tab&runId=29049726&resultId=101698&paneView=debug

[tommcdon]

@davmason is this issue fixed and unrelated issues are being added to this one?

[davmason]

@tommcdon the same test is failing again, it is likely a different underlying issue