Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable jemalloc pool test with BA_GLOBAL_PROVIDER #902

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

vinser52
Copy link
Contributor

@vinser52 vinser52 commented Nov 17, 2024

Description

Enable Jemalloc pool test with BA_GLOBAL_PROVIDER

Ref: #903

Checklist

  • Code compiles without errors locally
  • All tests pass locally
  • CI workflows execute properly

@vinser52
Copy link
Contributor Author

@bratpiorka @lplewa I created this PR as a reproducer for the issue with jemalloc (See #903 ). It works only from a certain version of jemalloc, but looks like our CI needs to be fixed to use the "proper" version of jemalloc.

@vinser52 vinser52 force-pushed the svinogra_jemalloc_ba_provider branch 2 times, most recently from 01ce947 to a262693 Compare November 27, 2024 11:01
@ldorau
Copy link
Contributor

ldorau commented Dec 2, 2024

@vinser52 Please test this PR on top of #941

Copy link
Contributor

@ldorau ldorau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vinser52 Please test this PR on top of #941

@vinser52 I've checked - this PR works on top of #941

@vinser52
Copy link
Contributor Author

vinser52 commented Dec 3, 2024

@vinser52 Please test this PR on top of #941

@vinser52 I've checked - this PR works on top of #941

thank you, @ldorau, Sorry I was busy yesterday and wanted to check it today.

@ldorau
Copy link
Contributor

ldorau commented Dec 5, 2024

@vinser52 #941 has been merged, so you can rebase this PR now

@vinser52 vinser52 force-pushed the svinogra_jemalloc_ba_provider branch from a262693 to 1b99bc7 Compare December 5, 2024 10:36
@vinser52
Copy link
Contributor Author

vinser52 commented Dec 5, 2024

@vinser52 #941 has been merged, so you can rebase this PR now

Done. Works locally on my machine. Waiting for the CI.

@bratpiorka
Copy link
Contributor

I guess this would require to add the wrapper for hwloc_topology_init() with attribute((disable_sanitizer_instrumentation)) or similar

@vinser52 vinser52 force-pushed the svinogra_jemalloc_ba_provider branch from 1b99bc7 to 48736b6 Compare December 6, 2024 16:43
@vinser52 vinser52 force-pushed the svinogra_jemalloc_ba_provider branch from 48736b6 to 0169514 Compare January 14, 2025 13:19
@ldorau
Copy link
Contributor

ldorau commented Jan 24, 2025

@vinser52 I've tried to fix that, but compiler attributes disabling AddressSanitizer do not work (build still fails):

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 70ac087..8835c40 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -452,6 +452,8 @@ endif()
 # Sanitizer flags
 if(UMF_USE_ASAN)
     add_sanitizer_flag(address)
+    set(UMF_COMMON_COMPILE_DEFINITIONS ${UMF_COMMON_COMPILE_DEFINITIONS}
+                                       UMF_USE_ASAN=1)
 endif()
 if(UMF_USE_UBSAN)
     add_sanitizer_flag(undefined)
diff --git a/src/provider/provider_os_memory.c b/src/provider/provider_os_memory.c
index 1fe4679..c26d3ed 100644
--- a/src/provider/provider_os_memory.c
+++ b/src/provider/provider_os_memory.c
@@ -1,5 +1,5 @@
 /*
- * Copyright (C) 2022-2024 Intel Corporation
+ * Copyright (C) 2022-2025 Intel Corporation
  *
  * Under the Apache License v2.0 with LLVM Exceptions. See LICENSE.TXT.
  * SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
@@ -17,6 +17,20 @@
 #include <umf/memory_provider_ops.h>
 #include <umf/providers/provider_os_memory.h>

+#ifdef UMF_USE_ASAN
+#ifdef _WIN32
+#define ATTR_NO_ASAN __declspec(noinline) __declspec(no_sanitize_address)
+#else /* !_WIN32 */
+#ifdef __clang__
+#define ATTR_NO_ASAN __attribute__((no_sanitize("address"), noinline))
+#else /* !__clang__ */
+#define ATTR_NO_ASAN __attribute__((no_sanitize_address, noinline))
+#endif /* !__clang__ */
+#endif /* !_WIN32 */
+#else  /* !UMF_USE_ASAN */
+#define ATTR_NO_ASAN
+#endif /* !UMF_USE_ASAN */
+
 // OS Memory Provider requires HWLOC
 #if defined(UMF_NO_HWLOC)

@@ -568,6 +582,10 @@ static umf_result_t translate_params(umf_os_memory_provider_params_t *in_params,
     return UMF_RESULT_SUCCESS;
 }

+static int ATTR_NO_ASAN hwloc_topology_init_no_asan(hwloc_topology_t *topo) {
+    return hwloc_topology_init(topo);
+}
+
 static umf_result_t os_initialize(void *params, void **provider) {
     umf_result_t ret;

@@ -594,7 +612,7 @@ static umf_result_t os_initialize(void *params, void **provider) {

     memset(os_provider, 0, sizeof(*os_provider));

-    int r = hwloc_topology_init(&os_provider->topo);
+    int r = hwloc_topology_init_no_asan(&os_provider->topo);
     if (r) {
         LOG_ERR("HWLOC topology init failed");
         ret = UMF_RESULT_ERROR_OUT_OF_HOST_MEMORY;

@ldorau
Copy link
Contributor

ldorau commented Jan 24, 2025

BTW: on Ubuntu 24.04 it works well

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants