Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PROF-9470] Enable "heap clean after GC" profiler optimization by default #4085

Merged

Conversation

ivoanjo
Copy link
Member

@ivoanjo ivoanjo commented Nov 7, 2024

What does this PR do?

This PR changes the optimization added in #4020 to be enabled by default.

I've collected a fresh set of benchmarking results for this feature in this google doc.

The TL;DR is that results seem to be... very close. E.g. sometimes we slightly improve things, but often the numbers seem too close to tell.

But on the other hand this also means that there are no regressions, and thus no reason not to enable the feature by default.

Motivation:

As a recap, without this optimization, the Ruby heap profiler works by sampling allocated objects, collecting and keeping metadata about these objects (stack trace, etc). Then, at serialization time (every 60 seconds), the profiler checks which objects are still alive; any objects still alive get included in the heap profile; any objects that have since been garbage collected get their metadata dropped.

The above scheme has a weak-point: some objects are allocated and almost immediately become garbage collected. Because the profiler only checks for object liveness at serialization time, this can mean that in the extreme, an object born and collected at the beginning of the profiling period can still be tracked for almost 60 seconds until the profiler finally figures out that the object is no longer alive.

This has two consequences:

  1. The profiler uses more memory, since it’s collecting metadata for already-dead objects
  2. The profiler has more work to do at the end of the 60-second period – it needs to check an entire 60 seconds of sampled objects

The heap profiling clean after GC optimization adds an extra mechanism that, based on Ruby GC activity, triggers periodic checking of young objects (e.g. objects that have been alive for few GC generations). Thus:

a. The profiler identifies and clears garbage objects faster,
thus overall needing less memory
b. The profiler has less work to do at the end of the 60-second
period
...trading it off with a smaller periodic pass

Change log entry

Enabled "heap clean after GC" profiler optimization by default

Additional Notes:

I've also removed the separate benchmarking configuration, to avoid having too many long-running benchmarking variants.

How to test the change?

I've updated the specs for the setting, and the optimization itself has existing test coverage that was added back in #4020.

…ault

**What does this PR do?**

This PR changes the optimization added in #4020 to be enabled by
default.

I've collected a fresh set of benchmarking results for this feature in
[this google doc](https://docs.google.com/document/d/143jmyzB7rMJ9W2hKN0JoDbjo2m3oCVCzvPToHVjLRAM/edit?tab=t.0#heading=h.f00wz5x8kwg6).

The TL;DR is that results seem to be... very close. E.g. sometimes we
slightly improve things, but often the numbers seem too close to tell.

But on the other hand this also means that there are no regressions,
and thus no reason not to enable the feature by default.

**Motivation:**

As a recap, without this optimization, the Ruby heap profiler works
by sampling allocated objects, collecting and keeping metadata about
these objects (stack trace, etc). Then, at serialization time
(every 60 seconds), the profiler checks which objects are still alive;
any objects still alive get included in the heap profile; any objects
that have since been garbage collected get their metadata dropped.

The above scheme has a weak-point: some objects are allocated and
almost immediately become garbage collected. Because the profiler
only checks for object liveness at serialization time, this can mean
that in the extreme, an object born and collected at the beginning
of the profiling period can still be tracked for almost 60 seconds
until the profiler finally figures out that the object is no longer
alive.

This has two consequences:
1. The profiler uses more memory, since it’s collecting metadata
   for already-dead objects
2. The profiler has more work to do at the end of the 60-second
    period – it needs to check an entire 60 seconds of sampled objects

The heap profiling clean after GC optimization adds an extra
mechanism that, based on Ruby GC activity, triggers periodic checking
of young objects (e.g. objects that have been alive for few GC
generations). Thus:

a. The profiler identifies and clears garbage objects faster,
   thus overall needing less memory
b. The profiler has less work to do at the end of the 60-second
    period
    ...trading it off with a smaller periodic pass

**Additional Notes:**

I've also removed the separate benchmarking configuration, to
avoid having too many long-running benchmarking variants.

**How to test the change?**

I've updated the specs for the setting, and the optimization itself
has existing test coverage that was added back in #4020.
@ivoanjo ivoanjo requested a review from a team as a code owner November 7, 2024 15:06
@github-actions github-actions bot added the core Involves Datadog core libraries label Nov 7, 2024
@ivoanjo ivoanjo added performance Involves performance (e.g. CPU, memory, etc) profiling Involves Datadog profiling and removed core Involves Datadog core libraries labels Nov 7, 2024
@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.73%. Comparing base (d7cd566) to head (da099a9).
Report is 8 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #4085   +/-   ##
=======================================
  Coverage   97.72%   97.73%           
=======================================
  Files        1338     1338           
  Lines       80251    80259    +8     
  Branches     4017     4017           
=======================================
+ Hits        78428    78438   +10     
+ Misses       1823     1821    -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member

@Strech Strech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@pr-commenter
Copy link

pr-commenter bot commented Nov 7, 2024

Benchmarks

Benchmark execution time: 2024-11-07 15:45:43

Comparing candidate commit da099a9 in PR branch ivoanjo/prof-9470-enable-heap-clean-after-gc-default with baseline commit f679de4 in branch master.

Found 1 performance improvements and 0 performance regressions! Performance is the same for 28 metrics, 2 unstable metrics.

scenario:profiler - sample timeline=false

  • 🟩 throughput [+0.589op/s; +0.601op/s] or [+9.461%; +9.658%]

@ivoanjo ivoanjo merged commit 27f7d0a into master Nov 7, 2024
268 checks passed
@ivoanjo ivoanjo deleted the ivoanjo/prof-9470-enable-heap-clean-after-gc-default branch November 7, 2024 15:46
@github-actions github-actions bot added this to the 2.7.0 milestone Nov 7, 2024
@TonyCTHsu TonyCTHsu mentioned this pull request Nov 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Involves performance (e.g. CPU, memory, etc) profiling Involves Datadog profiling
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants