Support filtering ObjectAllocated callback for pinned object heap allocation only #53134

cshung · 2021-05-23T00:46:45Z

Problem

In ICorProfiler, we can have the runtime to notify the profiler when an object is allocated on the large object heap only. We should have a similar option for the pinned object heap.

The remainder of the issue is meant for explaining to @Yauk how to contribute to this issue.

Background

If we wanted to capture the object allocations in an ICorProfiler based profiler, we can enable it when the profiler initialize as follow:

pCorProfilerInfo->SetEventMask2(COR_PRF_ENABLE_OBJECT_ALLOCATED, COR_PRF_HIGH_MONITOR_LARGEOBJECT_ALLOCATED)

And then the profiler's implementation for ObjectAllocated will be invoked when a large object is allocated.

A prototype that illustrates how that can be done is available in my private branch here.

How to run the prototype

Build for the first time

The build instruction should be clear about how to install the prerequisites, build the product and run the tests. Here is just a summary of the commands to run the prototype.

For building the CoreCLR runtime and the libraries:

build.cmd -s clr
build.cmd -c release -s libs /p:RuntimeConfiguration=Debug

To build ALL the tests:

cd src\tests
build.cmd
cd ..\..

To run all the gcallocate test case:

set CORE_ROOT=C:\Dev\runtime\artifacts\tests\coreclr\Windows.x64.Debug\Tests\Core_Root
cd C:\Dev\runtime\artifacts\tests\coreclr\windows.x64.Debug\profiler\gc\gcallocate
gcallocate.cmd

If all is well, we should hit an assert here because the ObjectAllocated callback is called as we expect it to.

https://github.com/cshung/runtime/blob/a1752998e2284d20a393d429390bfe7ab0ff2d5c/src/tests/profiler/native/gcallocateprofiler/gcallocateprofiler.cpp#L29

To make changes

I understand the building process takes a lot of time, and it would be bad if we have to wait that long for any changes, fortunately, we don't have to.

Rebuilding the CoreCLR, note that we need to update the test layouts (aka CORE_ROOT):

build.cmd -s clr
cd src\coreclr
build.cmd generatelayoutonly
cd ..\..

Changing the profiler is a bit more involved, but not all that hard, we need to setup the build system once.

cd C:\Dev\runtime\src\tests\profiler\native
mkdir build
cd build
cmake ..

And then we can just build, note that we need to make sure the test is running the built profiler.

cd C:\Dev\runtime\src\tests\profiler\native\build
msbuild ALL_BUILD.vcxproj
copy /y C:\Dev\runtime\src\tests\profiler\native\build\Debug\Profiler.dll C:\Dev\runtime\artifacts\tests\coreclr\windows.x64.Debug\profiler\gc\gcallocate\

Last but not least, we might want to change the managed test code, in that case we need to remove the dependency to the CMakeLists.txt here

https://github.com/cshung/runtime/blob/a1752998e2284d20a393d429390bfe7ab0ff2d5c/src/tests/profiler/gc/gcallocate.csproj#L21

and then run this:

cd C:\Dev\runtime\src\tests\profiler\gc
c:\dev\runtime\dotnet.cmd build gcallocate.csproj

How to get started

Here is where the COR_PRF_HIGH_MONITOR_LARGEOBJECT_ALLOCATED event mask is consumed.

runtime/src/coreclr/inc/profilepriv.inl

Lines 270 to 283 in 01b7e73

    
           inline BOOL CORProfilerTrackLargeAllocations() 
        
           { 
        
               CONTRACTL 
        
               { 
        
                   NOTHROW; 
        
                   GC_NOTRIGGER; 
        
                   CANNOT_TAKE_LOCK; 
        
               } 
        
               CONTRACTL_END; 
        
               return 
        
                       (CORProfilerPresent() && 
        
                       ((&g_profControlBlock)->dwEventMaskHigh & COR_PRF_HIGH_MONITOR_LARGEOBJECT_ALLOCATED)); 
        
           }

and therefore it is checked here during allocation to fire callbacks

runtime/src/coreclr/vm/gchelpers.cpp

Lines 326 to 334 in df6da13

    
           if (TrackAllocations() || 
        
               (TrackLargeAllocations() && flags & GC_ALLOC_LARGE_OBJECT_HEAP)) 
        
           { 
        
               OBJECTREF objref = ObjectToOBJECTREF((Object*)orObject); 
        
               GCPROTECT_BEGIN(objref); 
        
               ProfilerObjectAllocatedCallback(objref, (ClassID) orObject->GetTypeHandle().AsPtr()); 
        
               GCPROTECT_END(); 
        
               orObject = (TObj*) OBJECTREFToObject(objref); 
        
           }

There we should have a different case for GC_ALLOC_PINNED_OBJECT_HEAP

If all goes well, we should be able to add a GC.AllocateArray in the gcallocate test case and have the assertion fired for it (together with some runtime internal pinned object heap allocation)

Testing

Last but not least, we should have some testing for this capability, we should be able to reuse this test case for it, of course, we should not be doing assert(false), change it to something like counting the number of events fired for the object allocated event should be sufficient. (Do not hard code counts as things changes, check > 0 should be good enough)

@davmason

The text was updated successfully, but these errors were encountered:

ghost · 2021-05-23T00:46:47Z

Tagging subscribers to this area: @tommcdon
See info in area-owners.md if you want to be subscribed.

Issue Details

Problem

In ICorProfiler, we can have the runtime to notify the profiler when an object is allocated on the large object heap only. We should have a similar option for the pinned object heap.

The remainder of the issue is meant for explaining to our external contributors how to contribute to this issue.

Background

If we wanted to capture the object allocations in an ICorProfiler based profiler, we can enable it when the profiler initialize as follow:

pCorProfilerInfo->SetEventMask2(COR_PRF_ENABLE_OBJECT_ALLOCATED, COR_PRF_HIGH_MONITOR_LARGEOBJECT_ALLOCATED)

And then the profiler's implementation for ObjectAllocated will be invoked when a large object is allocated.

A prototype that illustrates how that can be done is available in my private branch here.

How to run the prototype

Build for the first time

The build instruction should be clear about how to install the prerequisites, build the product and run the tests. Here is just a summary of the commands to run the prototype.

For building the CoreCLR runtime and the libraries:

build.cmd -s clr
build.cmd -c release -s libs /p:RuntimeConfiguration=Debug

To build ALL the tests:

cd src\coreclr
build.cmd
cd ..\..

To run all the gcallocate test case:

set CORE_ROOT=C:\Dev\runtime\artifacts\tests\coreclr\Windows.x64.Debug\Tests\Core_Root
cd C:\Dev\runtime\artifacts\tests\coreclr\windows.x64.Debug\profiler\gc\gcallocate
gcallocate.cmd

If all is well, we should hit an assert here because the ObjectAllocated callback is called as we expect it to.

https://github.com/cshung/runtime/blob/a1752998e2284d20a393d429390bfe7ab0ff2d5c/src/tests/profiler/native/gcallocateprofiler/gcallocateprofiler.cpp#L29

To make changes

I understand the building process takes a lot of time, and it would be bad if we have to wait that long for any changes, fortunately, we don't have to.

Rebuilding the CoreCLR, note that we need to update the test layouts (aka CORE_ROOT):

build.cmd -s clr
cd src\coreclr
build.cmd generatelayoutonly
cd ..\..

Changing the profiler is a bit more involved, but not all that hard, we need to setup the build system once.

C:\Dev\runtime\src\tests\profiler\native
mkdir build
cd build
cmake ..

And then we can just build, note that we need to make sure the test is running the built profiler.

C:\Dev\runtime\src\tests\profiler\native\build
msbuild ALL_BUILD.vcxproj
copy /y C:\Dev\runtime\src\tests\profiler\native\build\Debug\Profiler.dll C:\Dev\runtime\artifacts\tests\coreclr\windows.x64.Debug\profiler\gc\gcallocate\

Last but not least, we might want to change the managed test code, in that case we need to remove the dependency to the CMakeLists.txt here

https://github.com/cshung/runtime/blob/a1752998e2284d20a393d429390bfe7ab0ff2d5c/src/tests/profiler/gc/gcallocate.csproj#L21

and then run this:

cd C:\Dev\runtime\src\tests\profiler\gc
c:\dev\runtime\dotnet.cmd build gcallocate.csproj

How to get started

Here is where the COR_PRF_HIGH_MONITOR_LARGEOBJECT_ALLOCATED event mask is consumed.

runtime/src/coreclr/inc/profilepriv.inl

Lines 270 to 283 in 01b7e73

    
           inline BOOL CORProfilerTrackLargeAllocations() 
        
           { 
        
               CONTRACTL 
        
               { 
        
                   NOTHROW; 
        
                   GC_NOTRIGGER; 
        
                   CANNOT_TAKE_LOCK; 
        
               } 
        
               CONTRACTL_END; 
        
               return 
        
                       (CORProfilerPresent() && 
        
                       ((&g_profControlBlock)->dwEventMaskHigh & COR_PRF_HIGH_MONITOR_LARGEOBJECT_ALLOCATED)); 
        
           }

and therefore it is checked here during allocation to fire callbacks

runtime/src/coreclr/vm/gchelpers.cpp

Lines 326 to 334 in df6da13

    
           if (TrackAllocations() || 
        
               (TrackLargeAllocations() && flags & GC_ALLOC_LARGE_OBJECT_HEAP)) 
        
           { 
        
               OBJECTREF objref = ObjectToOBJECTREF((Object*)orObject); 
        
               GCPROTECT_BEGIN(objref); 
        
               ProfilerObjectAllocatedCallback(objref, (ClassID) orObject->GetTypeHandle().AsPtr()); 
        
               GCPROTECT_END(); 
        
               orObject = (TObj*) OBJECTREFToObject(objref); 
        
           }

There we should have a different case for GC_ALLOC_PINNED_OBJECT_HEAP

If all goes well, we should be able to add a GC.AllocateArray in the gcallocate test case and have the assertion fired for it (together with some runtime internal pinned object heap allocation)

Testing

Last but not least, we should have some testing for this capability, we should be able to reuse this test case for it, of course, we should not be doing assert(false), change it to something like counting the number of events fired for the object allocated event should be sufficient. (Do not hard code counts as things changes, check > 0 should be good enough)

@davmason

Author:	cshung
Assignees:	-
Labels:	`area-Diagnostics-coreclr`
Milestone:	-

Yauk · 2021-06-08T22:25:52Z

Thanks for the detailed info, @cshung. This helps a lot and I am now working on this issue.

cshung added the area-Diagnostics-coreclr label May 23, 2021

dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label May 23, 2021

tommcdon added the enhancement Product code improvement that does NOT require public API changes/additions label May 23, 2021

tommcdon added this to the 6.0.0 milestone May 23, 2021

tommcdon removed the untriaged New issue has not been triaged by the area owner label May 23, 2021

tommcdon modified the milestones: 6.0.0, 7.0.0 Jun 16, 2021

cshung mentioned this issue Jul 10, 2021

Support filtering ObjectAllocated callback for pinned object heap allocation only #55448

Merged

cshung closed this as completed Jul 10, 2021

cshung mentioned this issue Jul 19, 2021

[Regions] pCorProfilerInfo->GetObjectGeneration may fail when running under USE_REGIONS #55965

Closed

ghost locked as resolved and limited conversation to collaborators Aug 9, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support filtering ObjectAllocated callback for pinned object heap allocation only #53134

Support filtering ObjectAllocated callback for pinned object heap allocation only #53134

cshung commented May 23, 2021 •

edited

Loading

ghost commented May 23, 2021

Problem

Background

How to run the prototype

Build for the first time

To make changes

How to get started

Testing

Yauk commented Jun 8, 2021

Support filtering ObjectAllocated callback for pinned object heap allocation only #53134

Support filtering ObjectAllocated callback for pinned object heap allocation only #53134

Comments

cshung commented May 23, 2021 • edited Loading

Problem

Background

How to run the prototype

Build for the first time

To make changes

How to get started

Testing

ghost commented May 23, 2021

Problem

Background

How to run the prototype

Build for the first time

To make changes

How to get started

Testing

Yauk commented Jun 8, 2021

cshung commented May 23, 2021 •

edited

Loading