Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fatal error. Internal CLR error. (0x80131506) #65890

Closed
MaximLipnin opened this issue Feb 25, 2022 · 10 comments · Fixed by #65910
Closed

Fatal error. Internal CLR error. (0x80131506) #65890

MaximLipnin opened this issue Feb 25, 2022 · 10 comments · Fixed by #65910
Assignees
@VSadov
Labels
arch-x86 area-VM-coreclr blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' os-windows runtime-coreclr specific to the CoreCLR runtime
Milestone
7.0.0

Comments

@MaximLipnin
Copy link
Contributor

MaximLipnin commented Feb 25, 2022 •
edited
Loading

Configuration: net7.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open

Build
Console log

Fatal error. Internal CLR error. (0x80131506)
   at System.GC.AllocateNewArray(IntPtr, Int32, GC_ALLOC_FLAGS)
   at System.Buffers.TlsOverPerCoreLockedStacksArrayPool`1[[System.Byte, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Rent(Int32)
   at System.IO.Stream+<<CopyToAsync>g__Core|27_0>d.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.IO.Stream+<<CopyToAsync>g__Core|27_0>d, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](<<CopyToAsync>g__Core|27_0>d ByRef)
...
   at System.Threading.Tasks.Task+<>c.<.cctor>b__272_0(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread)
   at System.Threading.Tasks.Task.ExecuteEntryUnsafe(System.Threading.Thread)
   at System.Threading.Tasks.ThreadPoolTaskScheduler+<>c.<.cctor>b__10_0(System.Object)
   at System.Threading.Thread.StartCallback()
----- end Fri 02/25/2022  8:56:40.95 ----- exit code -2146233082 ---------------------------------------------------------
@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label Feb 25, 2022
@MaximLipnin MaximLipnin added arch-x86 os-windows runtime-coreclr specific to the CoreCLR runtime labels Feb 25, 2022
@ghost
Copy link

ghost commented Feb 25, 2022

Tagging subscribers to this area: @dotnet/gc
See info in area-owners.md if you want to be subscribed.

Issue Details

Configuration: net7.0-windows-Release-x86-CoreCLR_release-Windows.7.Amd64.Open

Build
Console log

Fatal error. Internal CLR error. (0x80131506)
   at System.GC.AllocateNewArray(IntPtr, Int32, GC_ALLOC_FLAGS)
   at System.Buffers.TlsOverPerCoreLockedStacksArrayPool`1[[System.Byte, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Rent(Int32)
   at System.IO.Stream+<<CopyToAsync>g__Core|27_0>d.MoveNext()
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[[System.IO.Stream+<<CopyToAsync>g__Core|27_0>d, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]](<<CopyToAsync>g__Core|27_0>d ByRef)
...
   at System.Threading.Tasks.Task+<>c.<.cctor>b__272_0(System.Object)
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread)
   at System.Threading.Tasks.Task.ExecuteEntryUnsafe(System.Threading.Thread)
   at System.Threading.Tasks.ThreadPoolTaskScheduler+<>c.<.cctor>b__10_0(System.Object)
   at System.Threading.Thread.StartCallback()
----- end Fri 02/25/2022  8:56:40.95 ----- exit code -2146233082 ---------------------------------------------------------
Author: MaximLipnin
Assignees: -
Labels:

arch-x86, os-windows, area-GC-coreclr, untriaged, runtime-coreclr
Milestone: -

@jkotas
Copy link
Member

jkotas commented Feb 25, 2022

Stacktrace:

 # ChildEBP RetAddr      
00 0791d464 71f07b8e     coreclr!EEPolicy::HandleFatalError+0x72 [D:\a\_work\1\s\src\coreclr\vm\eepolicy.cpp @ 776] 
01 0791d5fc 77d034a1     coreclr!CPFH_VerifyThreadIsInValidState+0x1210d9 [D:\a\_work\1\s\src\coreclr\vm\i386\excepx86.cpp @ 345] 
02 0791d620 77d03473     ntdll!ExecuteHandler2+0x26 [d:\w7rtm\minkernel\ntos\rtl\i386\xcptmisc.asm @ 233] 
03 0791d6d0 77d03414     ntdll!ExecuteHandler+0x24 [d:\w7rtm\minkernel\ntos\rtl\i386\xcptmisc.asm @ 192] 
04 0791d6d0 77cb0133     ntdll!RtlDispatchException+0x127 [d:\w7rtm\minkernel\ntos\rtl\i386\exdsptch.c @ 560] 
05 0791db9c 7781f4a5     ntdll!KiUserExceptionDispatcher+0xf [d:\w7rtm\minkernel\ntos\rtl\i386\userdisp.asm @ 503] 
06 0791db9c 71e3b671     kernel32!SetXStateFeaturesMask+0x8 [d:\w7rtm\base\win32\client\xsave.c @ 296] 
07 0791dbc4 71df1278     coreclr!Thread::RedirectThreadAtHandledJITCase+0x45 [D:\a\_work\1\s\src\coreclr\vm\threadsuspend.cpp @ 2917] 
08 (Inline) --------     coreclr!Thread::CheckForAndDoRedirect+0xc [D:\a\_work\1\s\src\coreclr\vm\threadsuspend.cpp @ 2998] 
09 (Inline) --------     coreclr!Thread::CheckForAndDoRedirectForGC+0xc [D:\a\_work\1\s\src\coreclr\vm\threadsuspend.cpp @ 3130] 
0a 0791dc40 71e3cd4c     coreclr!ThreadSuspend::SuspendRuntime+0x3aa [D:\a\_work\1\s\src\coreclr\vm\threadsuspend.cpp @ 3525] 
0b 0791dce8 71e3cbe3     coreclr!ThreadSuspend::SuspendEE+0xf9 [D:\a\_work\1\s\src\coreclr\vm\threadsuspend.cpp @ 5753] 
0c 0791dd10 71de6ba8     coreclr!GCToEEInterface::SuspendEE+0x26 [D:\a\_work\1\s\src\coreclr\vm\gcenv.ee.cpp @ 28] 
0d 0791dd10 71e69ff9     coreclr!WKS::GCHeap::GarbageCollectGeneration+0xa8 [D:\a\_work\1\s\src\coreclr\gc\gc.cpp @ 45766]

@jkotas
Copy link
Member

jkotas commented Feb 25, 2022

@VSadov more x86 thread suspension issues.

@VSadov
Copy link
Member

VSadov commented Feb 25, 2022

Hmm. Windows7. It could be something now or could be a regression caused by the workaround for Win11.

@jkotas jkotas mentioned this issue Feb 25, 2022
Merged
@VSadov
Copy link
Member

VSadov commented Feb 25, 2022 •
edited
Loading

The redirect context is NULL. It is supposed to be preallocated before redirecting a thread. Perhaps there is some kind of OOM condition involved.
Anyways, the condition should not be fatal. It should cause the redirect attempt to fail and we would retry.

After a recent change for GC stress the first use of the context is SetXStateFeaturesMask and it looks like, unlike EEGetThreadContext which would report a failure, SetXStateFeaturesMask crashes with NRE.

Should just check pCtx for NULL before using it.

@VSadov VSadov mentioned this issue Feb 25, 2022
Merged
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Feb 25, 2022
@jkotas jkotas mentioned this issue Feb 26, 2022
Closed
@stephentoub stephentoub added the blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' label Feb 26, 2022
@jkotas jkotas mentioned this issue Feb 28, 2022
Merged
@runfoapp runfoapp bot mentioned this issue Feb 28, 2022
Closed
@mangod9 mangod9 removed the untriaged New issue has not been triaged by the area owner label Feb 28, 2022
@mangod9 mangod9 added this to the 7.0.0 milestone Feb 28, 2022
@agocke
Copy link
Member

agocke commented Feb 28, 2022 •
edited
Loading

Confirmed that this was caused by #65825. @VSadov do you have a fix ready? Otherwise, I think we should revert the change.

@VSadov
Copy link
Member

VSadov commented Feb 28, 2022

#65910 should fix the issue

@VSadov
Copy link
Member

VSadov commented Feb 28, 2022 •
edited
Loading

"Fix" in the sense that it could be a preexisting problem that sometimes we cannot allocate or initialize the context.
Prior to #65825 such situation would just cause a quiet retry, #65825 made it fatal.

It is still not clear why allocation fails. It could happen in a case of memory shortage, but frequency seems a bit too high for that.

So - it looks like we have exposed a preexisting issue and there is a fix for that.
The root cause is unclear though.

@VSadov
Copy link
Member

VSadov commented Feb 28, 2022 •
edited
Loading

Also - is this Windows7/x86 only?

I do not think I have seen reports with other combinations and cannot repro the issue on my machines (they are not Win7)

@elinor-fung elinor-fung mentioned this issue Mar 1, 2022
Merged
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Mar 1, 2022
@eerhardt eerhardt mentioned this issue Mar 1, 2022
Merged
2 tasks
@mdh1418 mdh1418 mentioned this issue Mar 2, 2022
Merged
@jkotas jkotas mentioned this issue Mar 3, 2022
Closed
@ghost ghost locked as resolved and limited conversation to collaborators Mar 31, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@VSadov VSadov

Labels
arch-x86 area-VM-coreclr blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' os-windows runtime-coreclr specific to the CoreCLR runtime
Projects
None yet
Milestone
7.0.0
Development

Successfully merging a pull request may close this issue.

null-check the redirect context before using. VSadov/runtime
7 participants
@agocke @EgorBo @stephentoub @MaximLipnin @jkotas @VSadov @mangod9