Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test failure ilasm/PortablePdb/IlasmPortablePdbTests/IlasmPortablePdbTests.sh #49632

Closed
VincentBu opened this issue Mar 15, 2021 · 5 comments
Closed
Assignees
Labels
arch-x64 area-ILTools-coreclr GCStress os-linux Linux OS (any supported distro) Priority:1 Work that is critical for the release, but we could probably ship without
Milestone

Comments

@VincentBu
Copy link
Contributor

Run: runtime-coreclr gcstress-extra 20210313.1

Failed test:

CoreCLR Linux x64 Checked gcstress0xc_jitstress2 @ Ubuntu.1804.Amd64.Open
 -ilasm/PortablePdb/IlasmPortablePdbTests/IlasmPortablePdbTests.sh

Error message:

/home/helixbot/work/AA000949/w/B1AD0A0B/e/ilasm/PortablePdb/IlasmPortablePdbTests/IlasmPortablePdbTests.sh: line 374: 24063 Segmentation fault (core dumped) $LAUNCHER $ExePath "${CLRTestExecutionArguments[@]}"

Return code: 1
Raw output file: /home/helixbot/work/AA000949/w/B1AD0A0B/e/ilasm/PortablePdb/Reports/ilasm.PortablePdb/IlasmPortablePdbTests/IlasmPortablePdbTests.output.txt
Raw output:
BEGIN EXECUTION
/home/helixbot/work/AA000949/p/corerun IlasmPortablePdbTests.dll ''
Discovering...
Gathering state for process 24063 corerun
Writing minidump with heap to file /home/helixbot/dotnetbuild/dumps/coredump.24063.dmp
Written 95232000 bytes (23250 pages) to core file
Dump successfully written
Expected: 100
Actual: 139
END EXECUTION - FAILED
Test Harness Exitcode is : 1
To run the test:
> set CORE_ROOT=/home/helixbot/work/AA000949/p
> /home/helixbot/work/AA000949/w/B1AD0A0B/e/ilasm/PortablePdb/IlasmPortablePdbTests/IlasmPortablePdbTests.sh
Expected: True
Actual: False

Stack trace
   at ilasm_PortablePdb._IlasmPortablePdbTests_IlasmPortablePdbTests_._IlasmPortablePdbTests_IlasmPortablePdbTests_sh()

@VincentBu VincentBu added os-linux Linux OS (any supported distro) GCStress arch-x64 labels Mar 15, 2021
@dotnet-issue-labeler dotnet-issue-labeler bot added area-ILTools-coreclr untriaged New issue has not been triaged by the area owner labels Mar 15, 2021
@JulieLeeMSFT JulieLeeMSFT added needs-further-triage Issue has been initially triaged, but needs deeper consideration or reconsideration and removed untriaged New issue has not been triaged by the area owner labels Mar 17, 2021
@VincentBu
Copy link
Contributor Author

Failed again in runtime-coreclr gcstress-extra 20210327.1

Failed test:

CoreCLR Linux x64 Checked gcstress0xc_zapdisable_jitstress2 @ Ubuntu.1804.Amd64.Open
 -ilasm/PortablePdb/IlasmPortablePdbTests/IlasmPortablePdbTests.sh

Error message:

/home/helixbot/work/9AE4088E/w/B5C70A32/e/ilasm/PortablePdb/IlasmPortablePdbTests/IlasmPortablePdbTests.sh: line 375: 6820 Segmentation fault (core dumped) $LAUNCHER $ExePath "${CLRTestExecutionArguments[@]}"

Return code: 1
Raw output file: /home/helixbot/work/9AE4088E/w/B5C70A32/e/ilasm/PortablePdb/Reports/ilasm.PortablePdb/IlasmPortablePdbTests/IlasmPortablePdbTests.output.txt
Raw output:
BEGIN EXECUTION
/home/helixbot/work/9AE4088E/p/corerun IlasmPortablePdbTests.dll ''
Discovering...
Gathering state for process 6820 corerun
Writing minidump with heap to file /home/helixbot/dotnetbuild/dumps/coredump.6820.dmp
Written 101179392 bytes (24702 pages) to core file
Dump successfully written
Expected: 100
Actual: 139
END EXECUTION - FAILED
Test Harness Exitcode is : 1
To run the test:
> set CORE_ROOT=/home/helixbot/work/9AE4088E/p
> /home/helixbot/work/9AE4088E/w/B5C70A32/e/ilasm/PortablePdb/IlasmPortablePdbTests/IlasmPortablePdbTests.sh
Expected: True
Actual: False

Stack trace
   at ilasm_PortablePdb._IlasmPortablePdbTests_IlasmPortablePdbTests_._IlasmPortablePdbTests_IlasmPortablePdbTests_sh()

@kunalspathak kunalspathak added the Priority:1 Work that is critical for the release, but we could probably ship without label Apr 1, 2021
@kunalspathak kunalspathak added this to the 6.0.0 milestone Apr 1, 2021
@JulieLeeMSFT JulieLeeMSFT added preview and removed needs-further-triage Issue has been initially triaged, but needs deeper consideration or reconsideration labels Apr 1, 2021
@kunalspathak
Copy link
Member

Historical failures of this test:

Queued OS Arch Column1
2021-04-01 09:53:50.6200000 windows.10.arm64v8.open arm64 runtime-coreclr crossgen2 outerloop Checked-no_tiered_compilation
2021-04-01 09:53:49.1820000 windows.10.arm64v8.open arm64 runtime-coreclr crossgen2 outerloop Checked
2021-04-01 09:52:42.7110000 windows.10.amd64.open.rt x64 runtime-coreclr crossgen2 outerloop Checked
2021-04-01 09:52:41.8360000 windows.10.amd64.open.rt x64 runtime-coreclr crossgen2 outerloop Checked-no_tiered_compilation
2021-04-01 09:49:28.0340000 osx.1013.amd64.open x64 runtime-coreclr crossgen2 outerloop Checked
2021-04-01 09:47:40.3000000 osx.1013.amd64.open x64 runtime-coreclr crossgen2 outerloop Checked-no_tiered_compilation
2021-04-01 07:58:14.8520000 ubuntu.1804.armarch.open arm64 runtime-coreclr crossgen2 Checked-no_tiered_compilation
2021-04-01 07:58:13.7270000 ubuntu.1804.armarch.open arm64 runtime-coreclr crossgen2 Checked
2021-04-01 07:56:44.7140000 osx.1013.amd64.open x64 runtime-coreclr crossgen2 Checked-no_tiered_compilation
2021-04-01 07:56:44.0100000 osx.1013.amd64.open x64 runtime-coreclr crossgen2 Checked
2021-04-01 07:56:34.5090000 ubuntu.1804.amd64.open.rt x64 runtime-coreclr crossgen2 Checked-no_tiered_compilation
2021-04-01 07:56:33.8950000 ubuntu.1804.amd64.open.rt x64 runtime-coreclr crossgen2 Checked
2021-04-01 07:51:19.1240000 windows.10.amd64.open.rt x64 runtime-coreclr crossgen2 Checked
2021-04-01 07:51:17.9790000 windows.10.amd64.open.rt x64 runtime-coreclr crossgen2 Checked-no_tiered_compilation
2021-04-01 07:50:51.7180000 windows.10.arm64v8.open arm64 runtime-coreclr crossgen2 Checked
2021-04-01 07:50:51.0230000 windows.10.arm64v8.open arm64 runtime-coreclr crossgen2 Checked-no_tiered_compilation
2021-03-29 08:47:20.8920000 ubuntu.1804.amd64.open.rt x64 runtime-coreclr crossgen2-composite gcstress Checked-gcstress0xc_zapdisable_jitstress2
2021-03-29 08:38:59.3830000 windows.10.amd64.open.rt x64 runtime-coreclr crossgen2-composite gcstress Checked-gcstress0xc_zapdisable_jitstress2
2021-03-29 00:31:39.5610000 windows.10.amd64.open.rt x64 runtime-coreclr gcstress-extra Checked-gcstress0xc_jitstress2
2021-03-29 00:28:41.1420000 ubuntu.1804.amd64.open.rt x64 runtime-coreclr gcstress-extra Checked-gcstress0xc_jitstress2
2021-03-28 23:28:25.6420000 ubuntu.1804.amd64.open.rt x64 runtime-coreclr gcstress-extra Checked-gcstress0xc_zapdisable_jitstress2
2021-03-28 23:27:13.6930000 windows.10.amd64.open.rt x64 runtime-coreclr gcstress-extra Checked-gcstress0xc_zapdisable_jitstress2
2021-03-28 09:00:36.8930000 ubuntu.1804.amd64.open.rt x64 runtime-coreclr crossgen2-composite gcstress Checked-gcstress0xc_zapdisable_jitstress2
2021-03-28 08:48:19.7690000 windows.10.amd64.open.rt x64 runtime-coreclr crossgen2-composite gcstress Checked-gcstress0xc_zapdisable_jitstress2
2021-03-28 01:14:28.7640000 ubuntu.1804.amd64.open.rt x64 runtime-coreclr gcstress-extra Checked-gcstress0xc_jitstress2
2021-03-28 00:29:38.3330000 windows.10.amd64.open.rt x64 runtime-coreclr gcstress-extra Checked-gcstress0xc_jitstress2
2021-03-27 23:58:26.7340000 ubuntu.1804.amd64.open.rt x64 runtime-coreclr gcstress-extra Checked-gcstress0xc_zapdisable_jitstress2
2021-03-27 23:53:44.7360000 windows.10.amd64.open.rt x64 runtime-coreclr gcstress-extra Checked-gcstress0xc_zapdisable_jitstress2
2021-03-22 08:57:24.1440000 ubuntu.1804.amd64.open.rt x64 runtime-coreclr crossgen2-composite gcstress Checked-gcstress0xc_zapdisable_jitstress2
2021-03-22 08:53:36.8620000 windows.10.amd64.open.rt x64 runtime-coreclr crossgen2-composite gcstress Checked-gcstress0xc_zapdisable_jitstress2
2021-03-22 00:55:25.1780000 windows.10.amd64.open.rt x64 runtime-coreclr gcstress-extra Checked-gcstress0xc_jitstress2
2021-03-22 00:24:24.9570000 ubuntu.1804.amd64.open.rt x64 runtime-coreclr gcstress-extra Checked-gcstress0xc_jitstress2
2021-03-21 23:57:35.2640000 windows.10.amd64.open.rt x64 runtime-coreclr gcstress-extra Checked-gcstress0xc_zapdisable_jitstress2
2021-03-21 23:51:31.8750000 ubuntu.1804.amd64.open.rt x64 runtime-coreclr gcstress-extra Checked-gcstress0xc_zapdisable_jitstress2
2021-03-21 08:42:38.1290000 ubuntu.1804.amd64.open.rt x64 runtime-coreclr crossgen2-composite gcstress Checked-gcstress0xc_zapdisable_jitstress2
2021-03-21 02:19:49.0470000 ubuntu.1804.amd64.open.rt x64 runtime-coreclr gcstress-extra Checked-gcstress0xc_jitstress2
2021-03-20 23:46:18.9170000 ubuntu.1804.amd64.open.rt x64 runtime-coreclr gcstress-extra Checked-gcstress0xc_zapdisable_jitstress2
2021-03-15 08:50:16.3910000 ubuntu.1804.amd64.open.rt x64 runtime-coreclr crossgen2-composite gcstress Checked-gcstress0xc_zapdisable_jitstress2
2021-03-15 01:00:07.9560000 ubuntu.1804.amd64.open.rt x64 runtime-coreclr gcstress-extra Checked-gcstress0xc_jitstress2
2021-03-14 23:46:18.9650000 ubuntu.1804.amd64.open.rt x64 runtime-coreclr gcstress-extra Checked-gcstress0xc_zapdisable_jitstress2
2021-03-14 08:30:52.3440000 ubuntu.1804.amd64.open.rt x64 runtime-coreclr crossgen2-composite gcstress Checked-gcstress0xc_zapdisable_jitstress2
2021-03-14 00:57:37.6660000 ubuntu.1804.amd64.open.rt x64 runtime-coreclr gcstress-extra Checked-gcstress0xc_jitstress2
2021-03-14 00:16:04.3680000 ubuntu.1804.amd64.open.rt x64 runtime-coreclr gcstress-extra Checked-gcstress0xc_zapdisable_jitstress2

@kunalspathak
Copy link
Member

This test started failing with the composite errors:

 Could Not Find C:\h\w\AD4508E7\w\B61909FD\e\ilasm\PortablePdb\IlasmPortablePdbTests\composite-r2r.dll.rsp
      Unhandled exception. System.AggregateException: One or more errors occurred. (Code generation failed for method '[IlasmPortablePdbTests]IlasmPortablePdbTests.IlasmPortablePdbTesterCommon.GetExpectedForTestLocalScopes(string)')
       ---> ILCompiler.CodeGenerationFailedException: Code generation failed for method '[IlasmPortablePdbTests]IlasmPortablePdbTests.IlasmPortablePdbTesterCommon.GetExpectedForTestLocalScopes(string)'
       ---> System.NullReferenceException: Object reference not set to an instance of an object.
         at ILCompiler.ReadyToRunCompilationModuleGroupBase.VersionsWithType(TypeDesc typeDesc)
         at ILCompiler.DependencyAnalysis.ReadyToRun.ModuleTokenResolver.AddModuleTokenForType(TypeDesc type, ModuleToken token)
         at Internal.JitInterface.CorInfoImpl.resolveToken(CORINFO_RESOLVED_TOKEN& pResolvedToken)
         at Internal.JitInterface.CorInfoImpl._resolveToken(IntPtr thisHandle, IntPtr* ppException, CORINFO_RESOLVED_TOKEN* pResolvedToken)
         --- End of inner exception stack trace ---
         at Internal.JitInterface.CorInfoImpl.CompileMethodInternal(IMethodNode methodCodeNodeNeedingCode, MethodIL methodIL)
         at Internal.JitInterface.CorInfoImpl.CompileMethod(MethodWithGCInfo methodCodeNodeNeedingCode, Logger logger)
         at ILCompiler.ReadyToRunCodegenCompilation.<ComputeDependencyNodeDependencies>b__36_0(DependencyNodeCore`1 dependency)
         at System.Threading.Tasks.Parallel.<>c__DisplayClass33_0`2.<ForEachWorker>b__0(Int32 i)
         at System.Threading.Tasks.Parallel.<>c__DisplayClass19_0`1.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)
      --- End of stack trace from previous location ---
         at System.Threading.Tasks.Parallel.<>c__DisplayClass19_0`1.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)
         at System.Threading.Tasks.TaskReplicator.Replica`1.ExecuteAction(Boolean& yieldedBeforeCompletion)
         at System.Threading.Tasks.TaskReplicator.Replica.Execute()
         --- End of inner exception stack trace ---
         at System.Threading.Tasks.TaskReplicator.Run[TState](ReplicatableUserAction`1 action, ParallelOptions options, Boolean stopOnFirstFailure)
         at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Func`4 bodyWithLocal, Func`1 localInit, Action`1 localFinally)
      --- End of stack trace from previous location ---
         at System.Threading.Tasks.Parallel.ThrowSingleCancellationExceptionOrOtherException(ICollection exceptions, CancellationToken cancelToken, Exception otherException)
         at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Func`4 bodyWithLocal, Func`1 localInit, Action`1 localFinally)
         at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IList`1 list, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
         at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable`1 source, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
         at System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable`1 source, ParallelOptions parallelOptions, Action`1 body)
         at ILCompiler.ReadyToRunCodegenCompilation.ComputeDependencyNodeDependencies(List`1 obj)
         at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.ComputeMarkedNodes()
         at ILCompiler.ReadyToRunCodegenCompilation.Compile(String outputFile)
         at ILCompiler.Program.Run(String[] args)
         at ILCompiler.Program.Main(String[] args)

@mangod9 - I am assigning it to you. It looks similar to #49985

@kunalspathak kunalspathak assigned mangod9 and unassigned kunalspathak Apr 8, 2021
@trylek
Copy link
Member

trylek commented Apr 11, 2021

@kunalspathak - I believe that the compilation failure you quoted was #50520 and that got fixed by #50539. This corresponds to the fact that the Crossgen2 compilation issue hasn't reproed since JanK merged the fix in. For the actual GC stress-related crash in the test that Vincent originally created this issue for, I guess that probably got independently fixed as you're not tracking any reproes since 4/1 and in March it was failing several times a day.

@trylek trylek closed this as completed Apr 11, 2021
@kunalspathak
Copy link
Member

It failed again because of timeout - runtime-coreclr r2r-extra Checked-gcstress0xf

@ghost ghost locked as resolved and limited conversation to collaborators May 12, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
arch-x64 area-ILTools-coreclr GCStress os-linux Linux OS (any supported distro) Priority:1 Work that is critical for the release, but we could probably ship without
Projects
Archived in project
Development

No branches or pull requests

5 participants