Test failure: JIT/HardwareIntrinsics/Arm/AdvSimd/AdvSimd_r/AdvSimd_r.sh #33948

v-haren · 2020-03-23T07:55:24Z

test failed in job: runtime-coreclr gcstress-extra 20200321.1

Error message

cmdLine:/root/helix/work/workitem/JIT/HardwareIntrinsics/Arm/AdvSimd/AdvSimd_r/AdvSimd_r.sh Timed Out
Return code: -100
Raw output file: /root/helix/work/workitem/JIT/HardwareIntrinsics/Reports/JIT.HardwareIntrinsics/Arm/AdvSimd/AdvSimd_r/AdvSimd_r.output.txt
Raw output:
BEGIN EXECUTION
/root/helix/work/correlation/corerun AdvSimd_r.dll ''
Supported ISAs:
AdvSimd: True
Aes: True
ArmBase: True
Crc32: True
Sha1: True
Sha256: True
Beginning test case Abs.Vector64.Int16 at 3/21/2020 11:21:34 PM
Random seed: 20010415; set environment variable CORECLR_SEED to this value to repro
Beginning scenario: RunBasicScenario_UnsafeRead
Beginning scenario: RunBasicScenario_Load
Beginning scenario: RunReflectionScenario_UnsafeRead
Beginning scenario: RunReflectionScenario_Load
Beginning scenario: RunClsVarScenario
Beginning scenario: RunClsVarScenario_Load
Beginning scenario: RunLclVarScenario_UnsafeRead
Beginning scenario: RunLclVarScenario_Load
Beginning scenario: RunClassLclFldScenario
Beginning scenario: RunClassLclFldScenario_Load
Beginning scenario: RunClassFldScenario
Beginning scenario: RunClassFldScenario_Load
Beginning scenario: RunStructLclFldScenario
Beginning scenario: RunStructLclFldScenario_Load
Beginning scenario: RunStructFldScenario
Beginning scenario: RunStructFldScenario_Load
Ending test case at 3/21/2020 11:22:05 PM
Beginning test case Abs.Vector64.Int32 at 3/21/2020 11:22:05 PM
Random seed: 20010415; set environment variable CORECLR_SEED to this value to repro
Beginning scenario: RunBasicScenario_UnsafeRead
Beginning scenario: RunBasicScenario_Load
Beginning scenario: RunReflectionScenario_UnsafeRead
Beginning scenario: RunReflectionScenario_Load
Beginning scenario: RunClsVarScenario
Beginning scenario: RunClsVarScenario_Load
Beginning scenario: RunLclVarScenario_UnsafeRead
Beginning scenario: RunLclVarScenario_Load
Beginning scenario: RunClassLclFldScenario
Beginning scenario: RunClassLclFldScenario_Load
Beginning scenario: RunClassFldScenario
Beginning scenario: RunClassFldScenario_Load
Beginning scenario: RunStructLclFldScenario
Beginning scenario: RunStructLclFldScenario_Load
Beginning scenario: RunStructFldScenario
Beginning scenario: RunStructFldScenario_Load
Ending test case at 3/21/2020 11:22:12 PM
Beginning test case Abs.Vector64.SByte at 3/21/2020 11:22:12 PM
Random seed: 20010415; set environment variable CORECLR_SEED to this value to repro
Beginning scenario: RunBasicScenario_UnsafeRead
Beginning scenario: RunBasicScenario_Load
Beginning scenario: RunReflectionScenario_UnsafeRead
Beginning scenario: RunReflectionScenario_Load
Beginning scenario: RunClsVarScenario
Beginning scenario: RunClsVarScenario_Load
Beginning scenario: RunLclVarScenario_UnsafeRead
Beginning scenario: RunLclVarScenario_Load
Beginning scenario: RunClassLclFldScenario
Beginning scenario: RunClassLclFldScenario_Load
Beginning scenario: RunClassFldScenario
Beginning scenario: RunClassFldScenario_Load
Beginning scenario: RunStructLclFldScenario
Beginning scenario: RunStructLclFldScenario_Load
Beginning scenario: RunStructFldScenario
Beginning scenario: RunStructFldScenario_Load
Ending test case at 3/21/2020 11:22:19 PM
Beginning test case Abs.Vector64.Single at 3/21/2020 11:22:19 PM
Random seed: 20010415; set environment variable CORECLR_SEED to this value to repro
Beginning scenario: RunBasicScenario_UnsafeRead
Beginning scenario: RunBasicScenario_Load
Beginning scenario: RunReflectionScenario_UnsafeRead
Beginning scenario: RunReflectionScenario_Load
Beginning scenario: RunClsVarScenario
Beginning scenario: RunClsVarScenario_Load
Beginning scenario: RunLclVarScenario_UnsafeRead
Beginning scenario: RunLclVarScenario_Load
Beginning scenario: RunClassLclFldScenario
Beginning scenario: RunClassLclFldScenario_Load
Beginning scenario: RunClassFldScenario
Beginning scenario: RunClassFldScenario_Load
Begin


Stack trace
   at JIT_HardwareIntrinsics._Arm_AdvSimd_AdvSimd_r_AdvSimd_r_._Arm_AdvSimd_AdvSimd_r_AdvSimd_r_sh() in /__w/1/s/artifacts/tests/coreclr/Linux.arm64.Checked/TestWrappers/JIT.HardwareIntrinsics/JIT.HardwareIntrinsics.XUnitWrapper.cs:line 334

category:correctness
theme:testing
skill-level:expert
cost:medium

The text was updated successfully, but these errors were encountered:

tannergooding · 2020-03-23T14:56:30Z

CC. @echesakovMSFT, @CarolEidt

v-haren · 2020-04-13T05:29:12Z

failed again in job: runtime-coreclr gcstress-extra 20200412.1

Error message

cmdLine:/root/helix/work/workitem/JIT/HardwareIntrinsics/Arm/AdvSimd.Arm64/AdvSimd.Arm64_r/AdvSimd.Arm64_r.sh Timed Out
Return code: -100\nRaw output file: /root/helix/work/workitem/JIT/HardwareIntrinsics/Reports/JIT.HardwareIntrinsics/Arm/AdvSimd.Arm64/AdvSimd.Arm64_r/AdvSimd.Arm64_r.output.txt
Raw output:\nBEGIN EXECUTION\n/root/helix/work/correlation/corerun AdvSimd.Arm64_r.dll ''
Supported ISAs:
AdvSimd: True
Aes: True
ArmBase: True
Crc32: True
Sha1: True
Sha256: True
Beginning test case Abs.Vector128.Double at 4/12/2020 11:41:31 PM
Random seed: 20010415; set environment variable CORECLR_SEED to this value to repro
Beginning scenario: RunBasicScenario_UnsafeRead
Beginning scenario: RunBasicScenario_Load
Beginning scenario: RunReflectionScenario_UnsafeRead
Beginning scenario: RunReflectionScenario_Load
Beginning scenario: RunClsVarScenario
Beginning scenario: RunClsVarScenario_Load
Beginning scenario: RunLclVarScenario_UnsafeRead
Beginning scenario: RunLclVarScenario_Load
Beginning scenario: RunClassLclFldScenario
Beginning scenario: RunClassLclFldScenario_Load
Beginning scenario: RunClassFldScenario
Beginning scenario: RunClassFldScenario_Load
Beginning scenario: RunStructLclFldScenario
Beginning scenario: RunStructLclFldScenario_Load
Beginning scenario: RunStructFldScenario
Beginning scenario: RunStructFldScenario_Load
Ending test case at 4/12/2020 11:42:05 PM
Beginning test case Abs.Vector128.Int64 at 4/12/2020 11:42:05 PM
Random seed: 20010415; set environment variable CORECLR_SEED to this value to repro
Beginning scenario: RunBasicScenario_UnsafeRead
Beginning scenario: RunBasicScenario_Load
Beginning scenario: RunReflectionScenario_UnsafeRead
Beginning scenario: RunReflectionScenario_Load
Beginning scenario: RunClsVarScenario
Beginning scenario: RunClsVarScenario_Load
Beginning scenario: RunLclVarScenario_UnsafeRead
Beginning scenario: RunLclVarScenario_Load
Beginning scenario: RunClassLclFldScenario
Beginning scenario: RunClassLclFldScenario_Load
Beginning scenario: RunClassFldScenario
Beginning scenario: RunClassFldScenario_Load
Beginning scenario: RunStructLclFldScenario
Beginning scenario: RunStructLclFldScenario_Load
Beginning scenario: RunStructFldScenario
Beginning scenario: RunStructFldScenario_Load
Ending test case at 4/12/2020 11:42:13 PM
Beginning test case AbsScalar.Vector64.Int64 at 4/12/2020 11:42:13 PM
Random seed: 20010415; set environment variable CORECLR_SEED to this value to repro
Beginning scenario: RunBasicScenario_UnsafeRead
Beginning scenario: RunBasicScenario_Load
Beginning scenario: RunReflectionScenario_UnsafeRead
Beginning scenario: RunReflectionScenario_Load
Beginning scenario: RunClsVarScenario
Beginning scenario: RunClsVarScenario_Load
Beginning scenario: RunLclVarScenario_UnsafeRead
Beginning scenario: RunLclVarScenario_Load
Beginning scenario: RunClassLclFldScenario
Beginning scenario: RunClassLclFldScenario_Load
Beginning scenario: RunClassFldScenario
Beginning scenario: RunClassFldScenario_Load
Beginning scenario: RunStructLclFldScenario
Beginning scenario: RunStructLclFldScenario_Load
Beginning scenario: RunStructFldScenario
Beginning scenario: RunStructFldScenario_Load
Ending test case at 4/12/2020 11:42:21 PM
Beginning test case AbsoluteCompareGreaterThan.Vector128.Double at 4/12/2020 11:42:21 PM
Random seed: 20010415; set environment variable CORECLR_SEED to this value to repro
Beginning scenario: RunBasicScenario_UnsafeRead
Beginning scenario: RunBasicScenario_Load
Beginning scenario: RunReflectionScenario_UnsafeRead
Beginning scenario: RunReflectionScenario_Load
Beginning scenario: RunClsVarScenario
Beginning scenario: RunClsVarScenario_Load
Beginning scenario: RunLclVarScenario_UnsafeRead
Beginning scenario: RunLclVarScenario_Load
Beginning scenario: RunClassLclFldScenario
Beginning scenario: RunClassLclFldScenario_Load
Beginning scenari


Stack trace
   at JIT_HardwareIntrinsics._Arm_AdvSimd_Arm64_AdvSimd_Arm64_r_AdvSimd_Arm64_r_._Arm_AdvSimd_Arm64_AdvSimd_Arm64_r_AdvSimd_Arm64_r_sh() in /__w/1/s/artifacts/tests/coreclr/Linux.arm64.Checked/TestWrappers/JIT.HardwareIntrinsics/JIT.HardwareIntrinsics.XUnitWrapper.cs:line 140

BruceForstall · 2020-06-28T23:03:19Z

@tannergooding Your change #37859 broke the AdvSimd into 4 parts. However, it looks like that's not sufficient.

A recent gcstress-extra job:

https://dev.azure.com/dnceng/public/_build/results?buildId=708029&view=ms.vss-test-web.build-test-results-tab&runId=21934032&paneView=debug&resultId=110183

still timed out

JIT/HardwareIntrinsics/Arm/AdvSimd/AdvSimd_Part0_r/AdvSimd_Part0_r.sh
JIT/HardwareIntrinsics/Arm/AdvSimd/AdvSimd_Part3_r/AdvSimd_Part3_r.sh

in CoreCLR Linux arm64 Checked gcstress0xc_zapdisable_heapverify1:

export COMPlus_GCStress=0xC
export COMPlus_HeapVerify=1
export COMPlus_ReadyToRun=0
export COMPlus_ZapDisable=1

This, of course, is a very slow kind of run.

echesakov · 2020-06-29T17:23:15Z

@BruceForstall @tannergooding I will fix this - I am going to split the AdvSimd tests by intrinsic method name - the current way of splitting (as done in #37859) has another big problem - you can't merge two PRs adding hardware intrinsics tests without rebasing one of them on top on another - they will almost certainly conflict in AdvSimd_Part*_r.proj file.

tannergooding · 2020-06-29T19:21:45Z

There is a downside to splitting by as fine grained as by method name in that it will increase build time and the number of build artifacts that exist. It will also increase runtime costs as the JIT will have to restart and recompile the same methods over and over which might cause other timeouts to be hit.

echesakov · 2020-06-29T19:32:53Z

There is a downside to splitting by as fine grained as by method name in that it will increase build time and the number of build artifacts that exist. It will also increase runtime costs as the JIT will have to restart and recompile the same methods over and over which might cause other timeouts to be hit.

I will do measurements and come back with numbers.

echesakov · 2020-07-01T20:32:34Z

I have draft PR #38599 that does splitting by method name

@tannergooding Regarding your concerns

There is a downside to splitting by as fine grained as by method name in that it will increase build time and the number of build artifacts that exist. It will also increase runtime costs as the JIT will have to restart and recompile the same methods over and over which might cause other timeouts to be hit.

It shouldn't increase either build time or the number of build artifacts since I build only one executable AdvSimd.dll and create many .cmd files that run this executable with arguments that are space separated test names. Judging by Kusto telemetry I don't see the dramatic increase in the tests running time either.

However, in the PR the HWIntrinsic tests have failed on win-x86 with timeout - so I will need to dig further to see what is going on.

Tanner, do you have any concerns with that approach assuming I can figure out what fails on win-x86?

tannergooding · 2020-07-01T20:50:00Z

Provided it isn't significantly increasing the build or run time. I don't have a problem.

BruceForstall · 2020-07-26T18:11:21Z

I still see Part0 and Part3 time out, as above:

https://dev.azure.com/dnceng/public/_build/results?buildId=745918&view=ms.vss-test-web.build-test-results-tab&runId=23118794&paneView=debug&resultId=110176

tannergooding · 2020-07-27T15:18:47Z

@echesakovMSFT, it looks like your draft PR didn't get merged, was there usability concerns?

Should we just lower the number of tests per project for now so they can run without timing out?

echesakov · 2020-07-27T17:19:59Z

it looks like your draft PR didn't get merged, was there usability concerns?

Should we just lower the number of tests per project for now so they can run without timing out?

@tannergooding I found some issues with the approach - I will work on the issue this week.

v-haren added GCStress area-Interop-coreclr labels Mar 23, 2020

Dotnet-GitSync-Bot added the untriaged New issue has not been triaged by the area owner label Mar 23, 2020

v-haren changed the title ~~Test faiure: JIT/HardwareIntrinsics/Arm/AdvSimd/AdvSimd_r/AdvSimd_r.sh~~ Test failure: JIT/HardwareIntrinsics/Arm/AdvSimd/AdvSimd_r/AdvSimd_r.sh Mar 23, 2020

BruceForstall added this to the 5.0 milestone Mar 23, 2020

BruceForstall added area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI arch-arm64 os-linux Linux OS (any supported distro) and removed area-Interop-coreclr untriaged New issue has not been triaged by the area owner labels Mar 23, 2020

BruceForstall assigned echesakov Mar 23, 2020

BruceForstall mentioned this issue Jun 1, 2020

Break up AdvSimd test #37282

Closed

AndyAyersMS mentioned this issue Jun 5, 2020

GC stress fix for reverse pinvoke stubs #37432

Merged

echesakov mentioned this issue Aug 6, 2020

Split and group HardwareIntrinsics Arm tests by method name #40177

Closed

echesakov mentioned this issue Aug 6, 2020

Fix #33948 by decreasing number of tests per group #40491

Merged

echesakov closed this as completed in #40491 Aug 10, 2020

echesakov added a commit that referenced this issue Aug 10, 2020

Fix #33948 by decreasing number of tests per group (#40491)

4988c25

ghost locked as resolved and limited conversation to collaborators Dec 10, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test failure: JIT/HardwareIntrinsics/Arm/AdvSimd/AdvSimd_r/AdvSimd_r.sh #33948

Test failure: JIT/HardwareIntrinsics/Arm/AdvSimd/AdvSimd_r/AdvSimd_r.sh #33948

v-haren commented Mar 23, 2020 •

edited by BruceForstall

Loading

tannergooding commented Mar 23, 2020

v-haren commented Apr 13, 2020

BruceForstall commented Jun 28, 2020

echesakov commented Jun 29, 2020

tannergooding commented Jun 29, 2020

echesakov commented Jun 29, 2020 •

edited

Loading

echesakov commented Jul 1, 2020

tannergooding commented Jul 1, 2020

BruceForstall commented Jul 26, 2020

tannergooding commented Jul 27, 2020

echesakov commented Jul 27, 2020

Test failure: JIT/HardwareIntrinsics/Arm/AdvSimd/AdvSimd_r/AdvSimd_r.sh #33948

Test failure: JIT/HardwareIntrinsics/Arm/AdvSimd/AdvSimd_r/AdvSimd_r.sh #33948

Comments

v-haren commented Mar 23, 2020 • edited by BruceForstall Loading

tannergooding commented Mar 23, 2020

v-haren commented Apr 13, 2020

BruceForstall commented Jun 28, 2020

echesakov commented Jun 29, 2020

tannergooding commented Jun 29, 2020

echesakov commented Jun 29, 2020 • edited Loading

echesakov commented Jul 1, 2020

tannergooding commented Jul 1, 2020

BruceForstall commented Jul 26, 2020

tannergooding commented Jul 27, 2020

echesakov commented Jul 27, 2020

v-haren commented Mar 23, 2020 •

edited by BruceForstall

Loading

echesakov commented Jun 29, 2020 •

edited

Loading