Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEBUG-3182 Rework DI loading #4239

Open
wants to merge 13 commits into
base: master
Choose a base branch
from
Open

DEBUG-3182 Rework DI loading #4239

wants to merge 13 commits into from

Conversation

p-datadog
Copy link
Member

What does this PR do?

This PR separates the part of DI that is required for code tracker (which is the code tracker itself and the DI.current_component group of methods, plus exception classes) to be self-contained. Currently in master loading datadog/di/init would bring in datadog/di as well which is too much (for example, that would cause DI to attempt to load ActiveRecord contrib, which is definitely not the right time for it).

Motivation:

Ensuring DI code tracker is loaded early without causing the rest of dd-trace-rb to be loaded early.

Change log entry

Yes: Improved early loading mechanism of dynamic instrumentation (datadog/di/init).

Additional Notes:

While this PR changes the behavior of datadog/di/init, this behavior has not yet been documented anywhere and should not be used by customers.

How to test the change?

Integration tests specifically for datadog/di/init will be added in a follow-up PR. The existing unit tests verify that basic DI functionality continues to operate correctly.

@p-datadog p-datadog marked this pull request as ready for review December 19, 2024 14:52
@p-datadog p-datadog requested a review from a team as a code owner December 19, 2024 14:52
@datadog-datadog-prod-us1
Copy link
Contributor

datadog-datadog-prod-us1 bot commented Dec 19, 2024

Datadog Report

Branch report: di-loading
Commit report: d3b854c
Test service: dd-trace-rb

❌ 15 Failed (0 Known Flaky), 20871 Passed, 1327 Skipped, 3m 14.57s Total Time
❄️ 49 New Flaky

❌ Failed Tests (15)

This report shows up to 5 failed tests.

  • Rails cache #delete - rspec - Details

    Expand for error
     undefined method 'embed_authenticity_token_in_remote_forms=' for module ActionView::Helpers::FormTagHelper
     
     Failure/Error: initialize!
     
     NoMethodError:
       undefined method 'embed_authenticity_token_in_remote_forms=' for module ActionView::Helpers::FormTagHelper
     /usr/local/bundle/gems/actionview-6.1.7.10/lib/action_view/railtie.rb:17:in 'block in <class:Railtie>'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:68:in 'block in ActiveSupport::LazyLoadHooks#execute_hook'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:61:in 'ActiveSupport::LazyLoadHooks#with_execution_control'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:66:in 'ActiveSupport::LazyLoadHooks#execute_hook'
     ...
    
  • Rails cache #delete when cache_key.enabled is false - rspec - Details

    Expand for error
     undefined method 'embed_authenticity_token_in_remote_forms=' for module ActionView::Helpers::FormTagHelper
     
     Failure/Error: initialize!
     
     NoMethodError:
       undefined method 'embed_authenticity_token_in_remote_forms=' for module ActionView::Helpers::FormTagHelper
     /usr/local/bundle/gems/actionview-6.1.7.10/lib/action_view/railtie.rb:17:in 'block in <class:Railtie>'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:68:in 'block in ActiveSupport::LazyLoadHooks#execute_hook'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:61:in 'ActiveSupport::LazyLoadHooks#with_execution_control'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:66:in 'ActiveSupport::LazyLoadHooks#execute_hook'
     ...
    
  • Rails cache #fetch_multi when the method is defined with exception - rspec

  • Rails cache #fetch_multi when the method is defined with exception and when cache_key.enabled is false - rspec - Details

    Expand for error
     undefined method 'embed_authenticity_token_in_remote_forms=' for module ActionView::Helpers::FormTagHelper
     
     Failure/Error: initialize!
     
     NoMethodError:
       undefined method 'embed_authenticity_token_in_remote_forms=' for module ActionView::Helpers::FormTagHelper
     /usr/local/bundle/gems/actionview-6.1.7.10/lib/action_view/railtie.rb:17:in 'block in <class:Railtie>'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:68:in 'block in ActiveSupport::LazyLoadHooks#execute_hook'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:61:in 'ActiveSupport::LazyLoadHooks#with_execution_control'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:66:in 'ActiveSupport::LazyLoadHooks#execute_hook'
     ...
    
  • Rails cache #fetch_multi when the method is not defined - rspec - Details

    Expand for error
     undefined method 'embed_authenticity_token_in_remote_forms=' for module ActionView::Helpers::FormTagHelper
     
     Failure/Error: initialize!
     
     NoMethodError:
       undefined method 'embed_authenticity_token_in_remote_forms=' for module ActionView::Helpers::FormTagHelper
     /usr/local/bundle/gems/actionview-6.1.7.10/lib/action_view/railtie.rb:17:in 'block in <class:Railtie>'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:68:in 'block in ActiveSupport::LazyLoadHooks#execute_hook'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:61:in 'ActiveSupport::LazyLoadHooks#with_execution_control'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:66:in 'ActiveSupport::LazyLoadHooks#execute_hook'
     ...
    

New Flaky Tests (49)

  • Rails cache #delete behaves like an instrumented cache method behaves like measured span for integration does not set _dd.measured on the span - rspec - Last Failure

    Expand for error
     undefined method 'embed_authenticity_token_in_remote_forms=' for module ActionView::Helpers::FormTagHelper
     
     Failure/Error: initialize!
     
     NoMethodError:
       undefined method 'embed_authenticity_token_in_remote_forms=' for module ActionView::Helpers::FormTagHelper
     Shared Example Group: "measured span for integration" called from ./spec/datadog/tracing/contrib/rails/cache_spec.rb:84
     Shared Example Group: "an instrumented cache method" called from ./spec/datadog/tracing/contrib/rails/cache_spec.rb:329
     /usr/local/bundle/gems/actionview-6.1.7.10/lib/action_view/railtie.rb:17:in 'block in <class:Railtie>'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:68:in 'block in ActiveSupport::LazyLoadHooks#execute_hook'
     ...
    
  • Rails cache #delete behaves like an instrumented cache method disabled at integration level does not instrument - rspec - Last Failure

    Expand for error
     undefined method 'embed_authenticity_token_in_remote_forms=' for module ActionView::Helpers::FormTagHelper
     
     Failure/Error: initialize!
     
     NoMethodError:
       undefined method 'embed_authenticity_token_in_remote_forms=' for module ActionView::Helpers::FormTagHelper
     Shared Example Group: "an instrumented cache method" called from ./spec/datadog/tracing/contrib/rails/cache_spec.rb:329
     /usr/local/bundle/gems/actionview-6.1.7.10/lib/action_view/railtie.rb:17:in 'block in <class:Railtie>'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:68:in 'block in ActiveSupport::LazyLoadHooks#execute_hook'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:61:in 'ActiveSupport::LazyLoadHooks#with_execution_control'
     ...
    
  • Rails cache #delete behaves like an instrumented cache method disabled at tracer level does not instrument - rspec - Last Failure

    Expand for error
     undefined method 'embed_authenticity_token_in_remote_forms=' for module ActionView::Helpers::FormTagHelper
     
     Failure/Error: initialize!
     
     NoMethodError:
       undefined method 'embed_authenticity_token_in_remote_forms=' for module ActionView::Helpers::FormTagHelper
     Shared Example Group: "an instrumented cache method" called from ./spec/datadog/tracing/contrib/rails/cache_spec.rb:329
     /usr/local/bundle/gems/actionview-6.1.7.10/lib/action_view/railtie.rb:17:in 'block in <class:Railtie>'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:68:in 'block in ActiveSupport::LazyLoadHooks#execute_hook'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:61:in 'ActiveSupport::LazyLoadHooks#with_execution_control'
     ...
    
  • Rails cache #delete behaves like an instrumented cache method with a cache different from Rails' default store returns the matching backend type - rspec - Last Failure

    Expand for error
     undefined method 'embed_authenticity_token_in_remote_forms=' for module ActionView::Helpers::FormTagHelper
     
     Failure/Error: initialize!
     
     NoMethodError:
       undefined method 'embed_authenticity_token_in_remote_forms=' for module ActionView::Helpers::FormTagHelper
     Shared Example Group: "an instrumented cache method" called from ./spec/datadog/tracing/contrib/rails/cache_spec.rb:329
     /usr/local/bundle/gems/actionview-6.1.7.10/lib/action_view/railtie.rb:17:in 'block in <class:Railtie>'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:68:in 'block in ActiveSupport::LazyLoadHooks#execute_hook'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:61:in 'ActiveSupport::LazyLoadHooks#with_execution_control'
     ...
    
  • Rails cache #delete behaves like an instrumented cache method with a cache not in the ActiveSupport::Cache:: namespace returns the matching backend type - rspec - Last Failure

    Expand for error
     undefined method 'embed_authenticity_token_in_remote_forms=' for module ActionView::Helpers::FormTagHelper
     
     Failure/Error: initialize!
     
     NoMethodError:
       undefined method 'embed_authenticity_token_in_remote_forms=' for module ActionView::Helpers::FormTagHelper
     Shared Example Group: "an instrumented cache method" called from ./spec/datadog/tracing/contrib/rails/cache_spec.rb:329
     /usr/local/bundle/gems/actionview-6.1.7.10/lib/action_view/railtie.rb:17:in 'block in <class:Railtie>'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:68:in 'block in ActiveSupport::LazyLoadHooks#execute_hook'
     /usr/local/bundle/gems/activesupport-6.1.7.10/lib/active_support/lazy_load_hooks.rb:61:in 'ActiveSupport::LazyLoadHooks#with_execution_control'
     ...
    

@pr-commenter
Copy link

pr-commenter bot commented Dec 19, 2024

Benchmarks

Benchmark execution time: 2024-12-25 17:33:11

Comparing candidate commit d3b854c in PR branch di-loading with baseline commit 44f36d3 in branch master.

Found 0 performance improvements and 2 performance regressions! Performance is the same for 29 metrics, 2 unstable metrics.

scenario:profiler - sample timeline=false

  • 🟥 throughput [-0.568op/s; -0.558op/s] or [-8.660%; -8.507%]

scenario:tracing - Propagation - Trace Context

  • 🟥 throughput [-4051.506op/s; -3951.822op/s] or [-10.453%; -10.196%]

Comment on lines +80 to +113
# DI code tracker is instantiated globally before the regular set of
# components is created, but the code tracker needs to call out to the
# "current" DI component to perform instrumentation when application
# code is loaded. Because this call may happen prior to Datadog
# components having been initialized, we maintain the "current component"
# which contains a reference to the most recently instantiated
# DI::Component. This way, if a DI component hasn't been instantiated,
# we do not try to reference Datadog.components.
# In other words, this method exists so that we never attempt to call
# Datadog.components from the code tracker.
def current_component
LOCK.synchronize do
@current_components&.last
end
end

# To avoid potential races with DI::Component being added and removed,
# we maintain a list of the components. Normally the list should contain
# either zero or one component depending on whether DI is enabled in
# Datadog configuration. However, if a new instance of DI::Component
# is created while the previous instance is still running, we are
# guaranteed to not end up with no component when one is running.
def add_current_component(component)
LOCK.synchronize do
@current_components ||= []
@current_components << component
end
end

def remove_current_component(component)
LOCK.synchronize do
@current_components&.delete(component)
end
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand your use-case well, you can maybe replace this with something along the lines of

        components = Datadog.send(:components, allow_initialization: false)
        di = components.di if components

This allows you to access the components without triggering initialization. This is something we use in for instance Datadog::Tracing.correlation to allow the API to be used, but not cause initialization as a side-effect.

@codecov-commenter
Copy link

codecov-commenter commented Dec 24, 2024

Codecov Report

Attention: Patch coverage is 78.72340% with 10 lines in your changes missing coverage. Please review.

Project coverage is 97.74%. Comparing base (44f36d3) to head (4cc2d79).

Files with missing lines Patch % Lines
lib/datadog/di/base.rb 71.42% 8 Missing ⚠️
lib/datadog/di/code_tracker.rb 50.00% 1 Missing ⚠️
spec/datadog/di/init_spec.rb 90.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4239      +/-   ##
==========================================
- Coverage   97.74%   97.74%   -0.01%     
==========================================
  Files        1355     1357       +2     
  Lines       82365    82384      +19     
  Branches     4230     4232       +2     
==========================================
+ Hits        80510    80527      +17     
- Misses       1855     1857       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants