JIT assert on arm32 when running DllImportGenerator unit tests: remainingSize == 4

[safern]

I just hit this on CI on a coreclr checked alpine 3.14 arm and Linux arm runs

net7.0-Linux-Debug-arm-CoreCLR_checked-(Alpine.314.Arm32.Open)

Console log:

/root/helix/work/workitem/e /root/helix/work/workitem/e
  Discovering: DllImportGenerator.Unit.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  DllImportGenerator.Unit.Tests (found 114 test cases)
  Starting:    DllImportGenerator.Unit.Tests (parallel test collections = on, max threads = 4)
    DllImportGenerator.UnitTests.IncrementalGenerationTests.AppendingUnrelatedSource_DoesNotRegenerateSource [SKIP]
      The GeneratorDriver treats all SyntaxTree replace operations on a Compilation as an Add/Remove operation instead of a Modify operation, so all cached results based on that input are thrown out. As a result, we cannot validate that unrelated changes within the same SyntaxTree do not cause regeneration.
    DllImportGenerator.UnitTests.IncrementalGenerationTests.ChangingMarshallingAttributes_SameStrategy_DoesNotRegenerate [SKIP]
      The GeneratorDriver treats all SyntaxTree replace operations on a Compilation as an Add/Remove operation instead of a Modify operation, so all cached results based on that input are thrown out. As a result, we cannot validate that unrelated changes within the same SyntaxTree do not cause regeneration.

Assert failure(PID 23 [0x00000017], Thread: 52 [0x0034]): Assertion failed 'remainingSize == 4' in 'Microsoft.CodeAnalysis.CSharp.Formatting.NewLineUserSettingFormattingRule:WithOptions(Microsoft.CodeAnalysis.Diagnostics.AnalyzerConfigOptions):Microsoft.CodeAnalysis.Formatting.Rules.AbstractFormattingRule:this' during 'Generate code' (IL size 31)

    File: /__w/1/s/src/coreclr/jit/codegenarmarch.cpp Line: 1062
    Image: /root/helix/work/correlation/dotnet

./RunTests.sh: line 162:    23 Aborted                 (core dumped) "$RUNTIME_PATH/dotnet" exec --runtimeconfig DllImportGenerator.Unit.Tests.runtimeconfig.json --depsfile DllImportGenerator.Unit.Tests.deps.json xunit.console.dll DllImportGenerator.Unit.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing $RSP_FILE
/root/helix/work/workitem/e
----- end Wed Oct 20 21:35:54 UTC 2021 ----- exit code 134 ----------------------------------------------------------

Core Dump: https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-60652-merge-c3b62da787c7471eb4/DllImportGenerator.Unit.Tests/1/core.1001.23?sv=2019-07-07&se=2021-11-09T21%3A34%3A31Z&sr=c&sp=rl&sig=AjcPkewA%2BElB7eAZKfTMCMRY7ZJgIJfGFW3iDA0wq%2FM%3D
How to debug dump: https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-60652-merge-c3b62da787c7471eb4/DllImportGenerator.Unit.Tests/1/how-to-debug-dump.md?sv=2019-07-07&se=2021-11-09T21%3A34%3A31Z&sr=c&sp=rl&sig=AjcPkewA%2BElB7eAZKfTMCMRY7ZJgIJfGFW3iDA0wq%2FM%3D

cc: @jkoritzinsky @elinor-fung

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ghost]

Tagging subscribers to this area: @JulieLeeMSFT
See info in area-owners.md if you want to be subscribed.

Issue Details

---

I just hit this on CI on a coreclr checked alpine 3.14 arm and Linux arm runs

net7.0-Linux-Debug-arm-CoreCLR_checked-(Alpine.314.Arm32.Open)

Console log:

/root/helix/work/workitem/e /root/helix/work/workitem/e
  Discovering: DllImportGenerator.Unit.Tests (method display = ClassAndMethod, method display options = None)
  Discovered:  DllImportGenerator.Unit.Tests (found 114 test cases)
  Starting:    DllImportGenerator.Unit.Tests (parallel test collections = on, max threads = 4)
    DllImportGenerator.UnitTests.IncrementalGenerationTests.AppendingUnrelatedSource_DoesNotRegenerateSource [SKIP]
      The GeneratorDriver treats all SyntaxTree replace operations on a Compilation as an Add/Remove operation instead of a Modify operation, so all cached results based on that input are thrown out. As a result, we cannot validate that unrelated changes within the same SyntaxTree do not cause regeneration.
    DllImportGenerator.UnitTests.IncrementalGenerationTests.ChangingMarshallingAttributes_SameStrategy_DoesNotRegenerate [SKIP]
      The GeneratorDriver treats all SyntaxTree replace operations on a Compilation as an Add/Remove operation instead of a Modify operation, so all cached results based on that input are thrown out. As a result, we cannot validate that unrelated changes within the same SyntaxTree do not cause regeneration.

Assert failure(PID 23 [0x00000017], Thread: 52 [0x0034]): Assertion failed 'remainingSize == 4' in 'Microsoft.CodeAnalysis.CSharp.Formatting.NewLineUserSettingFormattingRule:WithOptions(Microsoft.CodeAnalysis.Diagnostics.AnalyzerConfigOptions):Microsoft.CodeAnalysis.Formatting.Rules.AbstractFormattingRule:this' during 'Generate code' (IL size 31)

    File: /__w/1/s/src/coreclr/jit/codegenarmarch.cpp Line: 1062
    Image: /root/helix/work/correlation/dotnet

./RunTests.sh: line 162:    23 Aborted                 (core dumped) "$RUNTIME_PATH/dotnet" exec --runtimeconfig DllImportGenerator.Unit.Tests.runtimeconfig.json --depsfile DllImportGenerator.Unit.Tests.deps.json xunit.console.dll DllImportGenerator.Unit.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing $RSP_FILE
/root/helix/work/workitem/e
----- end Wed Oct 20 21:35:54 UTC 2021 ----- exit code 134 ----------------------------------------------------------

Core Dump: https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-60652-merge-c3b62da787c7471eb4/DllImportGenerator.Unit.Tests/1/core.1001.23?sv=2019-07-07&se=2021-11-09T21%3A34%3A31Z&sr=c&sp=rl&sig=AjcPkewA%2BElB7eAZKfTMCMRY7ZJgIJfGFW3iDA0wq%2FM%3D
How to debug dump: https://helixre8s23ayyeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-60652-merge-c3b62da787c7471eb4/DllImportGenerator.Unit.Tests/1/how-to-debug-dump.md?sv=2019-07-07&se=2021-11-09T21%3A34%3A31Z&sr=c&sp=rl&sig=AjcPkewA%2BElB7eAZKfTMCMRY7ZJgIJfGFW3iDA0wq%2FM%3D

cc: @jkoritzinsky @elinor-fung

Author:	safern
Assignees:	-
Labels:	area-CodeGen-coreclr, blocking-clean-ci, untriaged
Milestone:	-

[echesakov]

Un-assigning myself
cc @BruceForstall

[ericstj]

Looks like this is still being hit. I just happened to look at a PR build and noticed it: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-69895-merge-c971a8c218514b10a9/System.Text.RegularExpressions.Unit.Tests/1/console.5b33749d.log?helixlogtype=result

Assert failure(PID 24 [0x00000018], Thread: 66 [0x0042]): Assertion failed 'remainingSize == 4' in 'Microsoft.CodeAnalysis.CSharp.Formatting.NewLineUserSettingFormattingRule:WithOptions(Microsoft.CodeAnalysis.Diagnostics.AnalyzerConfigOptions):Microsoft.CodeAnalysis.Formatting.Rules.AbstractFormattingRule:this' during 'Generate code' (IL size 31; hash 0x3116fdc4; Tier0)

    File: /__w/1/s/src/coreclr/jit/codegenarmarch.cpp Line: 1119
    Image: /root/helix/work/correlation/dotnet

From https://dev.azure.com/dnceng/public/_build/results?buildId=1793005&view=logs&j=62876419-b96e-5483-94d3-b47175c1afe7&t=69034341-3a68-59aa-c247-b3e7d2608abb for #69895

[buyaa-n]

Failed again at https://github.com/dotnet/runtime/pull/69506/checks?check_run_id=6633313299
Should we reopen this issue @jkoritzinsky?
Log: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-69506-merge-07e620e4268e4042bf/System.Text.RegularExpressions.Unit.Tests/1/console.c89e4a26.log?helixlogtype=result

Assert failure(PID 23 [0x00000017], Thread: 71 [0x0047]): Assertion failed 'remainingSize == 4' in 'Microsoft.CodeAnalysis.CSharp.Formatting.NewLineUserSettingFormattingRule:WithOptions(Microsoft.CodeAnalysis.Diagnostics.AnalyzerConfigOptions):Microsoft.CodeAnalysis.Formatting.Rules.AbstractFormattingRule:this' during 'Generate code' (IL size 31; hash 0x3116fdc4; Tier0)

    File: /__w/1/s/src/coreclr/jit/codegenarmarch.cpp Line: 1119
    Image: /root/helix/work/correlation/dotnet

./RunTests.sh: line 168:    23 Aborted                 (core dumped) "$RUNTIME_PATH/dotnet" exec --runtimeconfig System.Text.RegularExpressions.Unit.Tests.runtimeconfig.json --depsfile System.Text.RegularExpressions.Unit.Tests.deps.json xunit.console.dll System.Text.RegularExpressions.Unit.Tests.dll -xml testResults.xml -nologo -nocolor -notrait category=IgnoreForCI -notrait category=OuterLoop -notrait category=failing $RSP_FILE
/root/helix/work/workitem/e
----- end Fri May 27 23:59:38 UTC 2022 ----- exit code 134 ----------------------------------------------------------
exit code 134 means SIGABRT Abort. Managed or native assert, or runtime check such as heap corruption, caused call to abort(). Core dumped.

[jakobbotsch]

@buyaa-n the failure looks 4 days old. #69905 was merged 3 days ago.

[buyaa-n]

Yes it is 4 days old, but the PR #69506 and it is not merged yet

[jakobbotsch]

Yes it is 4 days old, but the PR #69506 and it is not merged yet

The failed test run you linked to is based on the commit cacf4f77c58ca21a2daa3a1b1ef7842f175e7c9c.
I think you need to use /azp run runtime or close/reopen the PR to trigger a build with a new merge commit that should have the fix.