Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade profiler to use libdatadog v2.0.0 #2599

Merged
merged 5 commits into from
Feb 9, 2023
Merged

Conversation

ivoanjo
Copy link
Member

@ivoanjo ivoanjo commented Feb 3, 2023

What does this PR do?:

This PR updates the profiler to use the new libdatadog v2.0.0 APIs.

In particular:

  • The "local root span id" is now attached to samples as a number
  • The error reporting API changed, and I tweaked how we got error messages from libdatadog
  • Some of the libdatadog memory release methods ("drop" methods") were replaced, and I've adopted the new ones where needed.

Motivation:

Enable the Ruby profiler to use the latest libdatadog release.

Additional Notes:

A few of the changes were committed incrementally, so it may be easier to review commit-by-commit.

How to test the change?:

The changes are covered by our existing test suite. In fact, this branch helped pinpoint a libdatadog bug that was fixed before release.

…tring

This is a newly-added feature to both the profiling backend, as well as
libdatadog.

Instead of sending the local root span ids as strings (that take up
space in the string table, etc), we can now send them as numbers.

We already did a similar migration for span ids in #2476, but at
the time we did not change the local root span ids because the
libdatadog `ddog_prof_Profile_set_endpoint` API could not properly
handle the numeric ids (this has been changed in
DataDog/libdatadog#80 ).

As far as testing goes, I could have chosen to abstract away this
change from our specs BUT I chose to make it explicit and changed the
specs to match. I like having visibility in the specs of the values
being encoded and passed around as numbers and not strings.
This will allow us to have early warning for when we're using the
API incorrectly.

See also DataDog/libdatadog#87 .
One of the drop methods had to be commented out due to a libdatadog bug,
I'll re-enable it once libdatadog gets fixed.
@ivoanjo ivoanjo requested a review from a team February 3, 2023 14:58
Comment on lines 171 to 175
if (push_result.tag == DDOG_VEC_TAG_PUSH_RESULT_ERR) {
VALUE err_details = ruby_string_from_vec_u8(push_result.err);
ddog_Vec_Tag_PushResult_drop(push_result);

// libdatadog validates tags and may catch invalid tags that ddtrace didn't actually catch.
// We warn users about such tags, and then just ignore them.
safely_log_failure_to_process_tag(tags, err_details);
} else {
ddog_Vec_Tag_PushResult_drop(push_result);
safely_log_failure_to_process_tag(tags, get_error_details_and_drop(&push_result.err));
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The else branch is not actually needed because there's nothing to drop when the ddog_Vec_Tag_push operation was successful.

Comment on lines -290 to +288
VALUE encoded_pprof = ruby_string_from_prof_vec_u8(serialized_profile.ok.buffer);
VALUE encoded_pprof = ruby_string_from_vec_u8(serialized_profile.ok.buffer);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a workaround for a header issue in 1.0.1, and it's been fixed.

@github-actions github-actions bot added the profiling Involves Datadog profiling label Feb 3, 2023
@ivoanjo ivoanjo requested a review from morrisonlevi February 8, 2023 08:20
Copy link

@morrisonlevi morrisonlevi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. I did more than a glance, but not a particularly thorough review. If you would like more scrutiny, throw some time on calendar to go over it together!

@ivoanjo
Copy link
Member Author

ivoanjo commented Feb 9, 2023

Sounds good, thanks for the review!

@ivoanjo ivoanjo merged commit 234c5fe into master Feb 9, 2023
@ivoanjo ivoanjo deleted the ivoanjo/wip-libdatadog-2 branch February 9, 2023 08:48
@github-actions github-actions bot added this to the 1.10.0 milestone Feb 9, 2023
@TonyCTHsu TonyCTHsu mentioned this pull request Mar 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
profiling Involves Datadog profiling
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants