Test failure: CoreMangLib/system/span/RefStructWithSpan/RefStructWithSpan.cmd #103624

v-wenyuxu · 2024-06-18T08:27:47Z

Failed in: runtime-coreclr outerloop 20240617.4

Failed tests:

coreclr windows arm64 Checked no_tiered_compilation @ Windows.11.Arm64.Open
    - CoreMangLib/system/span/RefStructWithSpan/RefStructWithSpan.cmd

Error message:

 
cmdLine:C:hwB88F09C1wC2F00A46eCoreMangLibCoreMangLib../system/span/RefStructWithSpan/RefStructWithSpan.cmd Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 6/18/2024 2:47:58 AM, end: 6/18/2024 2:57:58 AM)

Return code:      -100
Raw output file:      C:hwB88F09C1wC2F00A46uploadssystemspanRefStructWithSpanoutput.txt
Raw output:
BEGIN EXECUTION
 "C:hwB88F09C1pcorerun.exe" -p "System.Reflection.Metadata.MetadataUpdater.IsSupported=false" -p "System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true"  RefStructWithSpan.dll 

cmdLine:C:hwB88F09C1wC2F00A46eCoreMangLibCoreMangLib../system/span/RefStructWithSpan/RefStructWithSpan.cmd Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 6/18/2024 2:47:58 AM, end: 6/18/2024 2:57:58 AM)
Test Harness Exitcode is : -100
To run the test:
Set up CORE_ROOT and run.
> C:hwB88F09C1wC2F00A46eCoreMangLibCoreMangLib../system/span/RefStructWithSpan/RefStructWithSpan.cmd

Stack trace:

   at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/Microsoft.DotNet.XUnitAssert/src/BooleanAsserts.cs:line 146
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath, String testPathPrefix)
   at Program.<<Main>$>g__TestExecutor46|0_47(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)

The text was updated successfully, but these errors were encountered:

dotnet-policy-service · 2024-06-18T16:01:28Z

Tagging subscribers to this area: @dotnet/area-system-runtime
See info in area-owners.md if you want to be subscribed.

jkotas · 2024-06-18T17:51:23Z

RefStructWithSpan test was introduced by dotnet/coreclr#16231

v-wenyuxu · 2024-06-20T08:24:04Z

Failed in: runtime-coreclr outerloop 20240619.4

Failed tests:

coreclr windows x86 Checked @ Windows.10.Amd64.Open
    - CoreMangLib/system/span/RefStructWithSpan/RefStructWithSpan.cmd

Error message:

 
cmdLine:C:hwA7080977wA8580954eCoreMangLibCoreMangLib../system/span/RefStructWithSpan/RefStructWithSpan.cmd Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 6/20/2024 2:40:26 AM, end: 6/20/2024 2:50:26 AM)

Return code:      -100
Raw output file:      C:hwA7080977wA8580954uploadssystemspanRefStructWithSpanoutput.txt
Raw output:
BEGIN EXECUTION
 "C:hwA7080977pcorerun.exe" -p "System.Reflection.Metadata.MetadataUpdater.IsSupported=false" -p "System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true"  RefStructWithSpan.dll 

cmdLine:C:hwA7080977wA8580954eCoreMangLibCoreMangLib../system/span/RefStructWithSpan/RefStructWithSpan.cmd Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 6/20/2024 2:40:26 AM, end: 6/20/2024 2:50:26 AM)
Test Harness Exitcode is : -100
To run the test:
Set up CORE_ROOT and run.
> C:hwA7080977wA8580954eCoreMangLibCoreMangLib../system/span/RefStructWithSpan/RefStructWithSpan.cmd

Stack trace:

   at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/Microsoft.DotNet.XUnitAssert/src/BooleanAsserts.cs:line 146
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath, String testPathPrefix)
   at Program.<<Main>$>g__TestExecutor46|0_47(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)

v-wenyuxu · 2024-06-21T02:02:59Z

Failed in: runtime-coreclr outerloop 20240620.2

Failed tests:

coreclr windows x64 Checked @ Windows.10.Amd64.Open
    - JIT/Regression/JitBlue/GitHub_27924/GitHub_27924/GitHub_27924.cmd

Error message:

 
cmdLine:C:hwAE210920wB9380A20eJITRegressionRegression_3../JitBlue/GitHub_27924/GitHub_27924/GitHub_27924.cmd Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 6/20/2024 7:40:51 PM, end: 6/20/2024 7:50:51 PM)

Return code:      -100
Raw output file:      C:hwAE210920wB9380A20uploadsJitBlueGitHub_27924GitHub_27924output.txt
Raw output:
BEGIN EXECUTION
 "C:hwAE210920pcorerun.exe" -p "System.Reflection.Metadata.MetadataUpdater.IsSupported=false" -p "System.Runtime.Serialization.EnableUnsafeBinaryFormatterSerialization=true"  GitHub_27924.dll 

cmdLine:C:hwAE210920wB9380A20eJITRegressionRegression_3../JitBlue/GitHub_27924/GitHub_27924/GitHub_27924.cmd Timed Out (timeout in milliseconds: 600000 from variable __TestTimeout, start: 6/20/2024 7:40:51 PM, end: 6/20/2024 7:50:51 PM)
Test Harness Exitcode is : -100
To run the test:
Set up CORE_ROOT and run.
> C:hwAE210920wB9380A20eJITRegressionRegression_3../JitBlue/GitHub_27924/GitHub_27924/GitHub_27924.cmd

Stack trace:

   at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/Microsoft.DotNet.XUnitAssert/src/BooleanAsserts.cs:line 146
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String assemblyPath, String testPathPrefix)
   at Program.<<Main>$>g__TestExecutor344|0_345(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)

EgorBo · 2024-06-21T16:48:52Z

In this pipeline (x86) multiple test failed with this symptom - timeout. https://dev.azure.com/dnceng-public/public/_build/results?buildId=715523&view=ms.vss-test-web.build-test-results-tab&runId=17909268&resultId=122250&paneView=debug

I was able to repro it locally but unfortunately didn't collect a dump

EgorBo · 2024-06-21T17:14:06Z

Stack-trace:

[0x0]   ntdll!NtWaitForSingleObject+0x14   0xd20177f0f8   0x7ffc2535952e   
[0x1]   KERNELBASE!WaitForSingleObjectEx+0x8e   0xd20177f100   0x7ffb1a92a0d3   
[0x2]   coreclr!WKS::GCHeap::WaitUntilGCComplete+0x83   0xd20177f1a0   0x7ffb1a6f8539   
[0x3]   coreclr!Thread::RareDisablePreemptiveGC+0x469   0xd20177f1e0   0x7ffb1a08e0d9   
[0x4]   coreclr!Thread::DisablePreemptiveGC+0xd9   0xd20177f300   0x7ffb1a08f98b   
[0x5]   coreclr!GCHolderBase::EnterInternalCoop+0x15b   0xd20177f340   0x7ffb1a076e46   
[0x6]   coreclr!GCCoop::GCCoop+0x56   0xd20177f380   0x7ffb1a3d724d   
[0x7]   coreclr!Thread::DetachThread+0x23d   0xd20177f3d0   0x7ffb1ab08035   
[0x8]   coreclr!TlsDestructionMonitor::~TlsDestructionMonitor+0x155   0xd20177f4c0   0x7ffb1acc5266

EgorBo · 2024-06-21T18:08:57Z

Minimal repro (Win-x64): https://gist.github.com/EgorBo/295ab4df2f337306d3d0afd5e158ac93

run it and wait for 30 sec or so - if it stuck - it hits the bug (100% reproducible for me)

I run it with R2R=0, TC=0 but likely reproes with default args too

EgorBo · 2024-06-21T19:47:00Z

My bisect script points to #103055 cc @jkoritzinsky @jkotas

Fixes dotnet#103624

dotnet-policy-service · 2024-06-26T09:41:54Z

Tagging subscribers to this area: @mangod9
See info in area-owners.md if you want to be subscribed.

JulieLeeMSFT · 2024-06-26T20:19:10Z

It also failed in runtime-coreclr gcstress-extra / 20240625.2.

coreclr windows arm64 Checked heapverify1 @ Windows.11.Arm64.Open

CoreMangLib/system/span/RefStructWithSpan/RefStructWithSpan.cmd

AndyAyersMS · 2024-06-28T14:52:23Z

Ah this looks like what I saw in #104057 too.

Switching to cooperative mode is not safe during process shutdown on Windows. Process shutdown can terminate a thread in the middle of the GC. The shutdown thread deadlocks if it tries to switch to cooperative mode and wait for the GC to finish in this situation. Use RtlDllShutdownInProgress Windows API to detect process shutdown to skip cleanup that has to be done in cooperative mode. The existing g_fProcessDetach flag is set too late - using this flag to skip cooperative mode switch would lead to shutdown deadlocks, and the existing g_fEEShutDown flag is set too early - using this flag to skip cooperative mode switch would lead to shutdown crashes. Fixes dotnet#103624

v-wenyuxu added arch-arm64 os-windows blocking-outerloop Blocking the 'runtime-coreclr outerloop' and 'runtime-libraries-coreclr outerloop' runs labels Jun 18, 2024

dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Jun 18, 2024

dotnet-policy-service bot added the untriaged New issue has not been triaged by the area owner label Jun 18, 2024

jeffschwMSFT added the area-System.Runtime label Jun 18, 2024

vcsjones removed the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Jun 18, 2024

jkotas added a commit to jkotas/runtime that referenced this issue Jun 24, 2024

Disable of RuntimeThreadLocals during shutdown

b648fd9

Fixes dotnet#103624

jkotas mentioned this issue Jun 24, 2024

Use RtlDllShutdownInProgress to detect process shutdown on Windows #103877

Merged

tannergooding added needs-further-triage Issue has been initially triaged, but needs deeper consideration or reconsideration and removed untriaged New issue has not been triaged by the area owner labels Jun 24, 2024

jakobbotsch added area-VM-coreclr and removed area-System.Runtime needs-further-triage Issue has been initially triaged, but needs deeper consideration or reconsideration labels Jun 26, 2024

jakobbotsch mentioned this issue Jun 26, 2024

JIT: Check for call interference when placing PUTARG nodes #103301

Merged

JulieLeeMSFT assigned jkotas Jun 26, 2024

JulieLeeMSFT added this to the 9.0.0 milestone Jun 26, 2024

jakobbotsch mentioned this issue Jun 28, 2024

Stack allocate unescaped boxes #103361

Merged

jkotas closed this as completed in #103877 Jul 2, 2024

jkotas closed this as completed in 5044e93 Jul 2, 2024

jakobbotsch mentioned this issue Jul 2, 2024

Test failure: Regressions/coreclr/GitHub_89834/test89834/test89834.cmd #104057

Closed

marcpopMSFT mentioned this issue Jul 9, 2024

Revert "Disable templating tests on windows to unblock PR builds" dotnet/sdk#42051

Merged

github-actions bot locked and limited conversation to collaborators Aug 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test failure: CoreMangLib/system/span/RefStructWithSpan/RefStructWithSpan.cmd #103624

Test failure: CoreMangLib/system/span/RefStructWithSpan/RefStructWithSpan.cmd #103624

v-wenyuxu commented Jun 18, 2024

dotnet-policy-service bot commented Jun 18, 2024

jkotas commented Jun 18, 2024

v-wenyuxu commented Jun 20, 2024

v-wenyuxu commented Jun 21, 2024

EgorBo commented Jun 21, 2024 •

edited

Loading

EgorBo commented Jun 21, 2024 •

edited

Loading

EgorBo commented Jun 21, 2024 •

edited

Loading

EgorBo commented Jun 21, 2024

dotnet-policy-service bot commented Jun 26, 2024

JulieLeeMSFT commented Jun 26, 2024

AndyAyersMS commented Jun 28, 2024

Test failure: CoreMangLib/system/span/RefStructWithSpan/RefStructWithSpan.cmd #103624

Test failure: CoreMangLib/system/span/RefStructWithSpan/RefStructWithSpan.cmd #103624

Comments

v-wenyuxu commented Jun 18, 2024

dotnet-policy-service bot commented Jun 18, 2024

jkotas commented Jun 18, 2024

v-wenyuxu commented Jun 20, 2024

v-wenyuxu commented Jun 21, 2024

EgorBo commented Jun 21, 2024 • edited Loading

EgorBo commented Jun 21, 2024 • edited Loading

EgorBo commented Jun 21, 2024 • edited Loading

EgorBo commented Jun 21, 2024

dotnet-policy-service bot commented Jun 26, 2024

JulieLeeMSFT commented Jun 26, 2024

AndyAyersMS commented Jun 28, 2024

EgorBo commented Jun 21, 2024 •

edited

Loading

EgorBo commented Jun 21, 2024 •

edited

Loading

EgorBo commented Jun 21, 2024 •

edited

Loading