Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix locking around m_LoaderAllocatorReferences iteration #68336

Merged

Conversation

janvorli
Copy link
Member

@janvorli janvorli commented Apr 21, 2022

The iteration that marks loader allocators in LoaderAllocator::Mark()
was not being called under an appropriate lock, which lead to a rare and
hard to reproduce race condition and a crash in the
LoaderAllocator::Mark(). The issue happened when the
m_LoaderAllocatorReferences.Begin() returned an iterator at the time the
m_LoaderAllocatorReferences were empty and another thread has written
something into those references before the
m_LoaderAllocatorReferences.End() was called. The iter was then
evaluated as not equal to the End and the iter was getting dereferenced.
Since the iterator had m_table set to NULL, the dereferencing caused
crash.

The issue shows up as something like this:

Fatal error. Internal CLR error. (0x80131506)
   at System.Reflection.LoaderAllocatorScout.<Destroy>g____PInvoke__|1_0(IntPtr)
   at System.Reflection.LoaderAllocatorScout.Finalize()

The iteration that marks loader allocators in LoaderAllocator::Mark()
was not being called under an appropriate lock, which lead to a rare and
hard to reproduce race condition and a crash in the
LoaderAllocator::Mark(). The issue happened when the
m_LoaderAllocatorReferences.Begin() returned an iterator at the time the
m_LoaderAllocatorReferences were empty and another thread has written
something into those references before the
m_LoaderAllocatorReferences.End() was called. The iter was then
evaluated as not equal to the End and the iter was getting dereferenced.
Since the iterator had m_table set to NULL, the dereferencing caused
crash.
@janvorli janvorli added this to the 6.0.x milestone Apr 21, 2022
@janvorli janvorli self-assigned this Apr 21, 2022
@@ -384,6 +384,7 @@ LoaderAllocator * LoaderAllocator::GCLoaderAllocators_RemoveAssemblies(AppDomain
AppDomain::AssemblyIterator i;
// Iterate through every loader allocator, marking as we go
{
CrstHolder chLoaderAllocatorReferencesLock(pAppDomain->GetLoaderAllocatorReferencesLock());
CrstHolder chAssemblyListLock(pAppDomain->GetAssemblyListLock());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The second iteration below is going to acquire same locks now. We can acquire the locks just once for the whole duration of both iterations.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, I'll update the PR that way.

@teo-tsirpanis
Copy link
Contributor

(setting the milestone to 7.0.0; the PR targets main, 6.0.x is for backports)

@teo-tsirpanis teo-tsirpanis modified the milestones: 6.0.x, 7.0.0 Apr 21, 2022
@jkotas jkotas merged commit 233c40e into dotnet:main Apr 21, 2022
@janvorli janvorli deleted the fix-loaderallocatorreferences-iteration-lock branch April 28, 2022 13:18
janvorli added a commit to janvorli/runtime that referenced this pull request Apr 28, 2022
This change ports two unloadability fixes for issues found by a
customer.
* dotnet#68550 that adds reference between native
`LoaderAllocator`s of two collectible `AssemblyLoadContexts` when one of
them is used to resolve assembly using the other one. This reference
ensures that the one that provides the resolved `Assembly` isn't
collected before the one that uses it.
* dotnet#68336 that adds a missing lock around `m_LoaderAllocatorReferences`
iteration during assembly load context destruction.
carlossanlop pushed a commit that referenced this pull request May 5, 2022
* [Release/6.0] Port unloadability fixes

This change ports two unloadability fixes for issues found by a
customer.
* #68550 that adds reference between native
`LoaderAllocator`s of two collectible `AssemblyLoadContexts` when one of
them is used to resolve assembly using the other one. This reference
ensures that the one that provides the resolved `Assembly` isn't
collected before the one that uses it.
* #68336 that adds a missing lock around `m_LoaderAllocatorReferences`
iteration during assembly load context destruction.

* Remove breaking change part

The change in main has added an exception in case a collectible
`Assembly` is resolved into a non-collectible `AssemblyLoadContext`.
Although that is incorrect, there are cases when it would work. So the
added exception is a breaking change that I think we likely don't want
to get into 6.0
@ghost ghost locked as resolved and limited conversation to collaborators May 28, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants