Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Valgrind reports Native AOT shared library memory leaks #81008

Closed
BahJiy opened this issue Jan 23, 2023 · 6 comments
Closed

Valgrind reports Native AOT shared library memory leaks #81008

BahJiy opened this issue Jan 23, 2023 · 6 comments

Comments

@BahJiy
Copy link

BahJiy commented Jan 23, 2023

Description

When calling a C# function from C/C++ via the NativeAOT shared library, valgrind reports multiple memory leaks.

At first, I though it was become I was linking to the shared library like a normal C library and that was causing the leak. So I changed it and used dlsym as suggested in the sample project, but that didn't work either.

Reproduction Steps

Class.cs

using System.Runtime.InteropServices;
namespace AOT;
public static class Interface {
    [UnmanagedCallersOnly( EntryPoint = "add" )]
    public static int Add( int a, int b ) {
        return a + b;
    }
}

AOT.csproj

<Project Sdk="Microsoft.NET.Sdk">
    <PropertyGroup>
        <TargetFramework>net7.0</TargetFramework>
        <ImplicitUsings>enable</ImplicitUsings>
        <Nullable>enable</Nullable>
        <AssemblyName>libaot</AssemblyName>    

        <PublishAot>true</PublishAot>
        <NativeLib>shared</NativeLib>
    </PropertyGroup>
</Project>

main_extern.cpp

#include <iostream>

extern "C" {
    int add( int a, int b );
}

void run( ) {
    auto result = add( 10, 20 );
    std::cout << result << std::endl;
}

int main( ) {
    run( );
}

main_dlload.cpp

#include <iostream>
#include <dlfcn.h>

#define symload dlsym
typedef int(*addFunc)(int, int);

void run( ) {
    void* handle = dlopen( "./publish/libaot.so", RTLD_LAZY );
    addFunc add = (addFunc)symload( handle, "add" );

    auto result = add( 10, 20 );

    std::cout << result << std::endl;
    dlclose( handle );
}

int main( ) {
    run( );
}
  1. Build AOT C# project and publish it: dotnet publish -o publish -r linux-x64
  2. Build the C++ programs: g++ main_dlload.cpp -o main_dlload g++ main_extern.cpp publish/libaot.so -o main_extern
  3. Run valgrind against either one: valgrind ./main_dlload or valgrind ./main_extern

Expected behavior

I expected valgrind to report no memory leaks.

Actual behavior

Valgrind reported reachable leaks and possibly lost leaks:

(Note: I edited the output to remove personal info)

==36422== Memcheck, a memory error detector
==36422== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==36422== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==36422== Command: ./main_dlload
==36422==
--36422-- WARNING: unhandled amd64-linux syscall: 334
--36422-- You may be able to write your own handler.
--36422-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
--36422-- Nevertheless we consider this a bug.  Please report
--36422-- it at http://valgrind.org/support/bug_reports.html.
==36422== Warning: set address range perms: large range [0x56f2000, 0x256f3000) (noaccess)
30
==36422==
==36422== HEAP SUMMARY:
==36422==     in use at exit: 1,014,509 bytes in 59 blocks
==36422==   total heap usage: 123 allocs, 64 frees, 1,114,813 bytes allocated
==36422==
==36422== 304 bytes in 1 blocks are possibly lost in loss record 46 of 59
==36422==    at 0x484D208: calloc (vg_replace_malloc.c:1328)
==36422==    by 0x40147D9: calloc (rtld-malloc.h:44)
==36422==    by 0x40147D9: allocate_dtv (dl-tls.c:375)
==36422==    by 0x40147D9: _dl_allocate_tls (dl-tls.c:634)
==36422==    by 0x4B2A834: allocate_stack (allocatestack.c:430)
==36422==    by 0x4B2A834: pthread_create@@GLIBC_2.34 (pthread_create.c:647)
==36422==    by 0x52B50AC: PalStartBackgroundWork (src/coreclr/nativeaot/Runtime/unix/PalRedhawkUnix.cpp:629)
==36422==    by 0x52B50AC: PalStartFinalizerThread (src/coreclr/nativeaot/Runtime/unix/PalRedhawkUnix.cpp:648)
==36422==    by 0x5264C47: RedhawkGCInterface::InitializeSubsystems() (src/coreclr/nativeaot/Runtime/gcrhenv.cpp:195)
==36422==    by 0x5269E16: InitDLL (src/coreclr/nativeaot/Runtime/startup.cpp:140)
==36422==    by 0x5269E16: RhInitialize (src/coreclr/nativeaot/Runtime/startup.cpp:460)
==36422==    by 0x52620D4: ??? (src/coreclr/nativeaot/Bootstrap/main.cpp:152)
==36422==    by 0x526BB04: EnsureRuntimeInitialized (src/coreclr/nativeaot/Runtime/thread.cpp:1193)
==36422==    by 0x526BB04: Thread::ReversePInvokeAttachOrTrapThread(ReversePInvokeFrame*) (src/coreclr/nativeaot/Runtime/thread.cpp:1155)
==36422==    by 0x5443197: libaot_AOT_Interface__Add (in /home/***/tmp/publish/libaot.so)
==36422==    by 0x109256: run() (in /home/***/tmp/main_dlload)
==36422==    by 0x10929B: main (in /home/***/tmp/main_dlload)
==36422==
==36422== LEAK SUMMARY:
==36422==    definitely lost: 0 bytes in 0 blocks
==36422==    indirectly lost: 0 bytes in 0 blocks
==36422==      possibly lost: 304 bytes in 1 blocks
==36422==    still reachable: 1,014,205 bytes in 58 blocks
==36422==         suppressed: 0 bytes in 0 blocks
==36422== Reachable blocks (those to which a pointer was found) are not shown.
==36422== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==36422==
==36422== For lists of detected and suppressed errors, rerun with: -s
==36422== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
==36511== Memcheck, a memory error detector
==36511== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==36511== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==36511== Command: ./main_extern
==36511==
--36511-- WARNING: unhandled amd64-linux syscall: 334
--36511-- You may be able to write your own handler.
--36511-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
--36511-- Nevertheless we consider this a bug.  Please report
--36511-- it at http://valgrind.org/support/bug_reports.html.
==36511== Warning: set address range perms: large range [0x56f3000, 0x256f4000) (noaccess)
30
==36511==
==36511== HEAP SUMMARY:
==36511==     in use at exit: 1,012,031 bytes in 53 blocks
==36511==   total heap usage: 114 allocs, 61 frees, 1,109,919 bytes allocated
==36511==
==36511== 304 bytes in 1 blocks are possibly lost in loss record 42 of 53
==36511==    at 0x484D208: calloc (vg_replace_malloc.c:1328)
==36511==    by 0x40147D9: calloc (rtld-malloc.h:44)
==36511==    by 0x40147D9: allocate_dtv (dl-tls.c:375)
==36511==    by 0x40147D9: _dl_allocate_tls (dl-tls.c:634)
==36511==    by 0x5053834: allocate_stack (allocatestack.c:430)
==36511==    by 0x5053834: pthread_create@@GLIBC_2.34 (pthread_create.c:647)
==36511==    by 0x49460AC: PalStartBackgroundWork (src/coreclr/nativeaot/Runtime/unix/PalRedhawkUnix.cpp:629)
==36511==    by 0x49460AC: PalStartFinalizerThread (src/coreclr/nativeaot/Runtime/unix/PalRedhawkUnix.cpp:648)
==36511==    by 0x48F5C47: RedhawkGCInterface::InitializeSubsystems() (src/coreclr/nativeaot/Runtime/gcrhenv.cpp:195)
==36511==    by 0x48FAE16: InitDLL (src/coreclr/nativeaot/Runtime/startup.cpp:140)
==36511==    by 0x48FAE16: RhInitialize (src/coreclr/nativeaot/Runtime/startup.cpp:460)
==36511==    by 0x48F30D4: ??? (src/coreclr/nativeaot/Bootstrap/main.cpp:152)
==36511==    by 0x48FCB04: EnsureRuntimeInitialized (src/coreclr/nativeaot/Runtime/thread.cpp:1193)
==36511==    by 0x48FCB04: Thread::ReversePInvokeAttachOrTrapThread(ReversePInvokeFrame*) (src/coreclr/nativeaot/Runtime/thread.cpp:1155)
==36511==    by 0x4AD4197: libaot_AOT_Interface__Add (in /home/***/tmp/publish/libaot.so)
==36511==    by 0x1091E3: run() (in /home/***/tmp/main_extern)
==36511==    by 0x10921C: main (in /home/***/tmp/main_extern)
==36511==
==36511== LEAK SUMMARY:
==36511==    definitely lost: 0 bytes in 0 blocks
==36511==    indirectly lost: 0 bytes in 0 blocks
==36511==      possibly lost: 304 bytes in 1 blocks
==36511==    still reachable: 1,011,727 bytes in 52 blocks
==36511==         suppressed: 0 bytes in 0 blocks
==36511== Reachable blocks (those to which a pointer was found) are not shown.
==36511== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==36511==
==36511== For lists of detected and suppressed errors, rerun with: -s
==36511== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

Regression?

No response

Known Workarounds

No response

Configuration

.NET SDK:
Version: 7.0.102
Commit: 4bbdd14480

Runtime Environment:
OS Name: ubuntu
OS Version: 22.04
OS Platform: Linux
RID: ubuntu.22.04-x64
Base Path: /usr/share/dotnet/sdk/7.0.102/

Host:
Version: 7.0.2
Architecture: x64
Commit: d037e07

.NET SDKs installed:
6.0.405 [/usr/share/dotnet/sdk]
7.0.102 [/usr/share/dotnet/sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 7.0.2 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 6.0.13 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.2 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
None

Environment variables:
DOTNET_ROOT [/usr/share/dotnet/]

global.json file:
Not found

Other information

On another attempt, I exposed the GC.Collect( ) function to C++ hoping that forcing the garbage collector to cleanup before disposing of the library handle would remove the leaks, but that did nothing.

    [UnmanagedCallersOnly( EntryPoint = "gc_collect" )]
    public static void GCCollect( ) {
        GC.Collect( );
    }
@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@ghost ghost added the untriaged New issue has not been triaged by the area owner label Jan 23, 2023
@ghost
Copy link

ghost commented Jan 23, 2023

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

Description

When calling a C# function from C/C++ via the NativeAOT shared library, valgrind reports multiple memory leaks.

At first, I though it was become I was linking to the shared library like a normal C library and that was causing the leak. So I changed it and used dlsym as suggested in the sample project, but that didn't work either.

Reproduction Steps

Class.cs

using System.Runtime.InteropServices;
namespace AOT;
public static class Interface {
    [UnmanagedCallersOnly( EntryPoint = "add" )]
    public static int Add( int a, int b ) {
        return a + b;
    }
}

AOT.csproj

<Project Sdk="Microsoft.NET.Sdk">
    <PropertyGroup>
        <TargetFramework>net7.0</TargetFramework>
        <ImplicitUsings>enable</ImplicitUsings>
        <Nullable>enable</Nullable>
        <AssemblyName>libaot</AssemblyName>    

        <PublishAot>true</PublishAot>
        <NativeLib>shared</NativeLib>
    </PropertyGroup>
</Project>

main_extern.cpp

#include <iostream>

extern "C" {
    int add( int a, int b );
}

void run( ) {
    auto result = add( 10, 20 );
    std::cout << result << std::endl;
}

int main( ) {
    run( );
}

main_dlload.cpp

#include <iostream>
#include <dlfcn.h>

#define symload dlsym
typedef int(*addFunc)(int, int);

void run( ) {
    void* handle = dlopen( "./publish/libaot.so", RTLD_LAZY );
    addFunc add = (addFunc)symload( handle, "add" );

    auto result = add( 10, 20 );

    std::cout << result << std::endl;
    dlclose( handle );
}

int main( ) {
    run( );
}
  1. Build AOT C# project and publish it: dotnet publish -o publish -r linux-x64
  2. Build the C++ programs: g++ main_dlload.cpp -o main_dlload g++ main_extern.cpp publish/libaot.so -o main_extern
  3. Run valgrind against either one: valgrind ./main_dlload or valgrind ./main_extern

Expected behavior

I expected valgrind to report no memory leaks.

Actual behavior

Valgrind reported reachable leaks and possibly lost leaks:

(Note: I edited the output to remove personal info)

==36422== Memcheck, a memory error detector
==36422== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==36422== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==36422== Command: ./main_dlload
==36422==
--36422-- WARNING: unhandled amd64-linux syscall: 334
--36422-- You may be able to write your own handler.
--36422-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
--36422-- Nevertheless we consider this a bug.  Please report
--36422-- it at http://valgrind.org/support/bug_reports.html.
==36422== Warning: set address range perms: large range [0x56f2000, 0x256f3000) (noaccess)
30
==36422==
==36422== HEAP SUMMARY:
==36422==     in use at exit: 1,014,509 bytes in 59 blocks
==36422==   total heap usage: 123 allocs, 64 frees, 1,114,813 bytes allocated
==36422==
==36422== 304 bytes in 1 blocks are possibly lost in loss record 46 of 59
==36422==    at 0x484D208: calloc (vg_replace_malloc.c:1328)
==36422==    by 0x40147D9: calloc (rtld-malloc.h:44)
==36422==    by 0x40147D9: allocate_dtv (dl-tls.c:375)
==36422==    by 0x40147D9: _dl_allocate_tls (dl-tls.c:634)
==36422==    by 0x4B2A834: allocate_stack (allocatestack.c:430)
==36422==    by 0x4B2A834: pthread_create@@GLIBC_2.34 (pthread_create.c:647)
==36422==    by 0x52B50AC: PalStartBackgroundWork (src/coreclr/nativeaot/Runtime/unix/PalRedhawkUnix.cpp:629)
==36422==    by 0x52B50AC: PalStartFinalizerThread (src/coreclr/nativeaot/Runtime/unix/PalRedhawkUnix.cpp:648)
==36422==    by 0x5264C47: RedhawkGCInterface::InitializeSubsystems() (src/coreclr/nativeaot/Runtime/gcrhenv.cpp:195)
==36422==    by 0x5269E16: InitDLL (src/coreclr/nativeaot/Runtime/startup.cpp:140)
==36422==    by 0x5269E16: RhInitialize (src/coreclr/nativeaot/Runtime/startup.cpp:460)
==36422==    by 0x52620D4: ??? (src/coreclr/nativeaot/Bootstrap/main.cpp:152)
==36422==    by 0x526BB04: EnsureRuntimeInitialized (src/coreclr/nativeaot/Runtime/thread.cpp:1193)
==36422==    by 0x526BB04: Thread::ReversePInvokeAttachOrTrapThread(ReversePInvokeFrame*) (src/coreclr/nativeaot/Runtime/thread.cpp:1155)
==36422==    by 0x5443197: libaot_AOT_Interface__Add (in /home/***/tmp/publish/libaot.so)
==36422==    by 0x109256: run() (in /home/***/tmp/main_dlload)
==36422==    by 0x10929B: main (in /home/***/tmp/main_dlload)
==36422==
==36422== LEAK SUMMARY:
==36422==    definitely lost: 0 bytes in 0 blocks
==36422==    indirectly lost: 0 bytes in 0 blocks
==36422==      possibly lost: 304 bytes in 1 blocks
==36422==    still reachable: 1,014,205 bytes in 58 blocks
==36422==         suppressed: 0 bytes in 0 blocks
==36422== Reachable blocks (those to which a pointer was found) are not shown.
==36422== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==36422==
==36422== For lists of detected and suppressed errors, rerun with: -s
==36422== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
==36511== Memcheck, a memory error detector
==36511== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==36511== Using Valgrind-3.18.1 and LibVEX; rerun with -h for copyright info
==36511== Command: ./main_extern
==36511==
--36511-- WARNING: unhandled amd64-linux syscall: 334
--36511-- You may be able to write your own handler.
--36511-- Read the file README_MISSING_SYSCALL_OR_IOCTL.
--36511-- Nevertheless we consider this a bug.  Please report
--36511-- it at http://valgrind.org/support/bug_reports.html.
==36511== Warning: set address range perms: large range [0x56f3000, 0x256f4000) (noaccess)
30
==36511==
==36511== HEAP SUMMARY:
==36511==     in use at exit: 1,012,031 bytes in 53 blocks
==36511==   total heap usage: 114 allocs, 61 frees, 1,109,919 bytes allocated
==36511==
==36511== 304 bytes in 1 blocks are possibly lost in loss record 42 of 53
==36511==    at 0x484D208: calloc (vg_replace_malloc.c:1328)
==36511==    by 0x40147D9: calloc (rtld-malloc.h:44)
==36511==    by 0x40147D9: allocate_dtv (dl-tls.c:375)
==36511==    by 0x40147D9: _dl_allocate_tls (dl-tls.c:634)
==36511==    by 0x5053834: allocate_stack (allocatestack.c:430)
==36511==    by 0x5053834: pthread_create@@GLIBC_2.34 (pthread_create.c:647)
==36511==    by 0x49460AC: PalStartBackgroundWork (src/coreclr/nativeaot/Runtime/unix/PalRedhawkUnix.cpp:629)
==36511==    by 0x49460AC: PalStartFinalizerThread (src/coreclr/nativeaot/Runtime/unix/PalRedhawkUnix.cpp:648)
==36511==    by 0x48F5C47: RedhawkGCInterface::InitializeSubsystems() (src/coreclr/nativeaot/Runtime/gcrhenv.cpp:195)
==36511==    by 0x48FAE16: InitDLL (src/coreclr/nativeaot/Runtime/startup.cpp:140)
==36511==    by 0x48FAE16: RhInitialize (src/coreclr/nativeaot/Runtime/startup.cpp:460)
==36511==    by 0x48F30D4: ??? (src/coreclr/nativeaot/Bootstrap/main.cpp:152)
==36511==    by 0x48FCB04: EnsureRuntimeInitialized (src/coreclr/nativeaot/Runtime/thread.cpp:1193)
==36511==    by 0x48FCB04: Thread::ReversePInvokeAttachOrTrapThread(ReversePInvokeFrame*) (src/coreclr/nativeaot/Runtime/thread.cpp:1155)
==36511==    by 0x4AD4197: libaot_AOT_Interface__Add (in /home/***/tmp/publish/libaot.so)
==36511==    by 0x1091E3: run() (in /home/***/tmp/main_extern)
==36511==    by 0x10921C: main (in /home/***/tmp/main_extern)
==36511==
==36511== LEAK SUMMARY:
==36511==    definitely lost: 0 bytes in 0 blocks
==36511==    indirectly lost: 0 bytes in 0 blocks
==36511==      possibly lost: 304 bytes in 1 blocks
==36511==    still reachable: 1,011,727 bytes in 52 blocks
==36511==         suppressed: 0 bytes in 0 blocks
==36511== Reachable blocks (those to which a pointer was found) are not shown.
==36511== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==36511==
==36511== For lists of detected and suppressed errors, rerun with: -s
==36511== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

Regression?

No response

Known Workarounds

No response

Configuration

.NET SDK:
Version: 7.0.102
Commit: 4bbdd14480

Runtime Environment:
OS Name: ubuntu
OS Version: 22.04
OS Platform: Linux
RID: ubuntu.22.04-x64
Base Path: /usr/share/dotnet/sdk/7.0.102/

Host:
Version: 7.0.2
Architecture: x64
Commit: d037e07

.NET SDKs installed:
6.0.405 [/usr/share/dotnet/sdk]
7.0.102 [/usr/share/dotnet/sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 7.0.2 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 6.0.13 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.2 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
None

Environment variables:
DOTNET_ROOT [/usr/share/dotnet/]

global.json file:
Not found

Other information

On another attempt, I exposed the GC.Collect( ) function to C++ hoping that forcing the garbage collector to cleanup before disposing of the library handle would remove the leaks, but that did nothing.

    [UnmanagedCallersOnly( EntryPoint = "gc_collect" )]
    public static void GCCollect( ) {
        GC.Collect( );
    }
Author: BahJiy
Assignees: -
Labels:

untriaged, area-NativeAOT-coreclr

Milestone: -

@MichalStrehovsky
Copy link
Member

I'm not sure how to interpret this - do you see more and more memory getting leaked as you call the UnmanagedCallersOnly method, or as you call it the first time, a leak happens and the memory is never freed?

NativeAOT is not unloadable (#64629) - once the library is loaded into the process, it's expected to stay there forever. So it won't try to do cleanup because it's designed for the ultimate cleanup - process termination. The "leaked" amount should remain more or less constant.

@BahJiy
Copy link
Author

BahJiy commented Jan 24, 2023

Ok. Did not know it was unloadable. No extra leaks occur on other calls.

Then would it be safe to ignore theses in valgrind via a suppression file?

{
   ignore_unversioned_libs
   Memcheck:Leak
   ...
   obj:*/libaot.so
}

@MichalStrehovsky
Copy link
Member

Then would it be safe to ignore theses in valgrind via a suppression file?

Yes, I think so

@agocke agocke added this to AppModel Mar 6, 2023
@agocke agocke added this to the Future milestone Mar 13, 2023
@agocke agocke removed the untriaged New issue has not been triaged by the area owner label Mar 13, 2023
@MichalStrehovsky
Copy link
Member

As long as we don't support unloading, this is expected, no need to track it with an issue.

@MichalStrehovsky MichalStrehovsky closed this as not planned Won't fix, can't repro, duplicate, stale Nov 10, 2023
@github-actions github-actions bot locked and limited conversation to collaborators Dec 10, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
Archived in project
Development

No branches or pull requests

4 participants