-
Notifications
You must be signed in to change notification settings - Fork 423
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(ci, profiling): build native macos arm64 wheel #12029
Conversation
|
Datadog ReportBranch report: ✅ 0 Failed, 130 Passed, 1468 Skipped, 5m 14.57s Total duration (35m 44.84s time saved) |
BenchmarksBenchmark execution time: 2025-01-27 18:24:51 Comparing candidate commit c548e12 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 394 metrics, 2 unstable metrics. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I ran into an ImportError when trying this on my M1 macbook. I downloaded the python 3.13 macos arm64 wheel from the build artifacts (link) and installed it in a virtual environment, then ran a trivial program with ddtrace-run:
% python3 --version
Python 3.13.0
% pip install ddtrace-2.21.0.dev58+g664c6b2a9-cp313-cp313-macosx_12_0_arm64.whl
... stuff ...
% env DD_PROFILING_ENABLED=true ./venv/bin/ddtrace-run python3 test.py
failed to enable profiling
Traceback (most recent call last):
File "/Users/nick.ripley/sandbox/python/ddtracepy-pr-12029/venv/lib/python3.13/site-packages/ddtrace/bootstrap/preload.py", line 66, in <module>
import ddtrace.profiling.auto # noqa: F401
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/nick.ripley/sandbox/python/ddtracepy-pr-12029/venv/lib/python3.13/site-packages/ddtrace/internal/module.py", line 295, in _exec_module
self.loader.exec_module(module)
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
File "/Users/nick.ripley/sandbox/python/ddtracepy-pr-12029/venv/lib/python3.13/site-packages/ddtrace/profiling/__init__.py", line 5, in <module>
from .profiler import Profiler # noqa:F401
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/nick.ripley/sandbox/python/ddtracepy-pr-12029/venv/lib/python3.13/site-packages/ddtrace/internal/module.py", line 295, in _exec_module
self.loader.exec_module(module)
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
File "/Users/nick.ripley/sandbox/python/ddtracepy-pr-12029/venv/lib/python3.13/site-packages/ddtrace/profiling/profiler.py", line 28, in <module>
from ddtrace.profiling.collector import stack
File "/Users/nick.ripley/sandbox/python/ddtracepy-pr-12029/venv/lib/python3.13/site-packages/ddtrace/internal/module.py", line 234, in _create_module
return self.loader.create_module(spec)
~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
ImportError: dlopen(/Users/nick.ripley/sandbox/python/ddtracepy-pr-12029/venv/lib/python3.13/site-packages/ddtrace/profiling/collector/stack.cpython-313-darwin.so, 0x0002): symbol not found in flat namespace '__PyThread_CurrentExceptions'
Is that expected to work?
@nsrip-dd Profiling doesn't support 3.13 yet but it also failed on 3.12 on my m1 mac. I thought I checked this, but turns out not. Thank you for catching this. Found the missing link flag to properly build on mac and not explicitly include libpython as a dependency. Updated cmake and we run delocate-wheel as we used to |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Confirmed the fix as follows:
Installed a macos Python 3.12 wheel prior to this fix. Ran with the following command and got an error like so:
% env DD_PROFILING_TIMELINE_ENABLED=true DD_PROFILING_ENABLED=true ./venv/bin/ddtrace-run python3 test.py
Failed to load libdd (module 'ddtrace.internal.datadog.profiling.ddup' has no attribute 'config') (dlopen(/Users/nick.ripley/sandbox/python/ddtracepy-pr-12029/venv/lib/python3.12/site-packages/ddtrace/internal/datadog/profiling/ddup/_ddup.cpython-312-darwin.so, 0x0002): Library not loaded: /Library/Frameworks/Python.framework/Versions/3.12/Python
Referenced from: <7DC51F2E-55D6-3BEA-BCEE-8F3192B9B2EC> /Users/nick.ripley/sandbox/python/ddtracepy-pr-12029/venv/lib/python3.12/site-packages/ddtrace/internal/datadog/profiling/ddup/_ddup.cpython-312-darwin.so
Reason: tried: '/Library/Frameworks/Python.framework/Versions/3.12/Python' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Library/Frameworks/Python.framework/Versions/3.12/Python' (no such file), '/Library/Frameworks/Python.framework/Versions/3.12/Python' (no such file)), falling back to legacy mode
Then I installed a wheel after the fix:
% env DD_PROFILING_TIMELINE_ENABLED=true DD_PROFILING_ENABLED=true ./venv/bin/ddtrace-run python3 test.py
% # ran to completion with no error
(cherry picked from commit b922886)
Use macos-latest (M1) runner to build arm64 wheel, instead of building universal2 wheel on macos-13 (intel x86_64).
universal2 wheels didn't have native modules to run profiling with certain features.
Checklist
Reviewer Checklist