1.20.0

Highlights

Dynamic sampling for Alpha Allocations and Heap Profiling

Allocation and heap profiling were introduced as experimental/alpha features in ddtrace 1.19.0 to help you optimize your application to allocate and use less memory, as well as to track down memory leaks.

In 1.20.0, we've introduced a new dynamic sampler that tunes sampling parameters to keep sampling overhead under control. Target overhead is governed by the overhead_target_percentage setting. This sampling mechanism replaces the one included in release 1.19.0 which relied on fixed sampling rates and thus required manual tuning to adapt to the specificities of your application.

As a reminder, you can enable these memory profiling features:

• Using the environment variables DD_PROFILING_EXPERIMENTAL_ALLOCATION_ENABLED=true,
  DD_PROFILING_EXPERIMENTAL_HEAP_ENABLED=true and DD_PROFILING_EXPERIMENTAL_HEAP_SIZE_ENABLED=true
• Or via code by adding to your Datadog.configure block:

Datadog.configure do |c|
  # ... existing configuration ...
  c.profiling.advanced.experimental_allocation_enabled = true
  c.profiling.advanced.experimental_heap_enabled = true
  c.profiling.advanced.experimental_heap_size_enabled = true
end

Known issues and limitations:

• This feature only works on Ruby 2.7 and above.
• Ruby versions 3.2.0, 3.2.1 and 3.2.2 are not supported due to a Ruby VM bug. The bug has been fixed on Ruby 3.2.3, released on January 18.
• This feature is currently incompatible with applications using Ractors due to a Ruby VM bug.
• The three options above are not fully independent; experimental_heap_size_enabled requires experimental_heap_enabled,
  and in turn experimental_heap_enabled requires experimental_allocation_enabled.

We're very interested in feedback on how this feature works for you. Give it a try, let us know how it's working!

Test visibility

Test visibility in Ruby is feature complete and generally available now. This release includes:

• Test sessions and suites are reported by default
• Source code integration
• CODEOWNERS support
• Parametrized tests in Cucumber

See full release notes from datadog-ci-rb for more information.

Changelog

Added

• Tracing: Add Trilogy instrumentation (#3274)
• Rack: Add remote configuration boot tags (#3315)
• Faraday: Add on_error option (#3431)
• Profiling: Add dynamic allocation sampling (#3395)

Changed

• Bump datadog-ci dependency to 0.7.0 (#3408)
• Improve performance of gathering ClassCount metric (#3386)

Fixed

• OpenTelemetry: Fix internal loading (#3400)
• Core: Fix logger deadlock (#3426)
• Rack: Fix missing active trace (#3420)
• Redis: Fix instance configuration (#3278)

Read the full changeset and the release milestone.

1.19.0

Highlights

Alpha support for Memory Profiling (Allocations and Heap)

As of ddtrace 1.19.0, the Profiler supports three new profile types:

• Allocations
• Heap Live Objects
• Heap Live Size

You can use these to optimize your application to allocate and use less memory, as well as to track down memory leaks.

We're still working on tuning the overhead impact and sampling frequency for these new profile types, and that's why this feature is marked as experimental.

You can enable these features:

• Using the environment variables DD_PROFILING_EXPERIMENTAL_ALLOCATION_ENABLED=true,
  DD_PROFILING_EXPERIMENTAL_HEAP_ENABLED=true and DD_PROFILING_EXPERIMENTAL_HEAP_SIZE_ENABLED=true
• Or via code by adding to your Datadog.configure block:

Datadog.configure do |c|
  # ... existing configuration ...
  c.profiling.advanced.experimental_allocation_enabled = true
  c.profiling.advanced.experimental_heap_enabled = true
  c.profiling.advanced.experimental_heap_size_enabled = true
end

Known issues and limitations:

• This feature only works on Ruby 2.7 and above.
• Ruby versions 3.2.0, 3.2.1 and 3.2.2 are not supported due to a Ruby VM bug. The bug has been fixed upstream and the upcoming Ruby 3.2.3 will be supported.
• This feature is currently incompatible with applications using Ractors due to a Ruby VM bug.
• The three options above are not fully independent; experimental_heap_size_enabled requires experimental_heap_enabled,
  and in turn experimental_heap_enabled requires experimental_allocation_enabled.
• We have added experimental tunables for lowering the overhead of these features. For more details, check the documentation for the experimental_allocation_sample_rate and experimental_heap_sample_rate settings.
• The experimental_allocation_enabled feature replaces the allocation_counting_enabled setting, which is now deprecated.

We're very interested in feedback on how this feature works for you. Give it a try, let us know how it's working!

Performance improvements to Garbage Collection Profiling + Timeline view support

The off-by-default Garbage Collection profiling feature has been redesigned to lower its overhead, and to support showing time spent doing Garbage Collection in the Profiler Timeline view.

See #3313 for some nice screenshots of how it looks!

You can enable this feature using the DD_PROFILING_FORCE_ENABLE_GC=true environment variable, or using the
c.profiling.advanced.force_enable_gc_profiling = true setting via code.

Added

• Tracing: Add on_error settings for mysql2 (#3316)
• Core: Add install_signature to app-started telemetry event (#3349)
• Profiling: Heap Profiling (#3281) (#3287) (#3328) (#3329) (#3333) (#3360)
• Profiling: Redesign GC profiling to add timeline support and reduce overhead (#3313)
• Core: Use Ruby 3.3 stable for CI testing (#3354)

Changed

• Core: Bump datadog-ci dependency to 0.6.0 (#3361)
• Core: Bump debase-ruby_core_source dependency to 3.3.1 (#3373)
• Docs: Backport "List Ruby 3.3 as supported in the docs" to master branch (#3374)
• Profiling: Import upstream rb_profile_frames fix (#3352)
• Profiling: Allow the dynamic sampling rate overhead target to be set (#3310)
• Profiling: Split profiling tests into ractor and non-ractor suites. (#3320)

Fixed

• Docs: Fix pg doc markdown format (#3317)
• Tracing: Fix recursive require in Railtie (#3365)
• Profiling: Fix issues stemming from rb_gc_force_recycle (#3366)
• Profiling: Fix Ruby 3.3 CI being broken in master due to profiler (#3356)
• Profiling: Fix "no signals" workaround detection when mariadb is in use (#3362)

Read the full changeset and the release milestone.

1.18.0

Added

• Tracing: Support lib injection for ARM64 architecture (#3307)
• Tracing: Add error_handler for pg instrumentation (#3303)
• Appsec: Enable "Trusted IPs", a.k.a passlist with optional monitoring (#3229)

Changed

• Mark ddtrace threads as fork-safe (#3279)
• Bump datadog-ci dependency to 0.5.0 (#3308)
• Bump debase-ruby_core_source dependency to 3.2.3 (#3284)
• Profiling: Disable profiler on Ruby 3.3 when running with RUBY_MN_THREADS=1 (#3259)
• Profiling: Run without "no signals" workaround on passenger 6.0.19+ (#3280)

Fixed

• Tracing: Fix pg instrumentation enabled settings (#3271)
• Profiling: Fix potential crash by importing upstream rb_profile_frames fix (#3289)
• Appsec: Call devise RegistrationsController block (#3286)

Read the full changeset and the release milestone.

1.17.0

For W3C Trace Context, this release adds tracecontext to the default trace propagation extraction and injection styles. The new defaults are:

• Extraction: Datadog,b3multi,b3,tracecontext
• Injection: Datadog,tracecontext

And to increase interoperability with tracecontext, 128-bit Trace ID generation is now the default.

For OpenTelemetry, this release adds support for converting OpenTelemetry Trace Semantic Conventions into equivalent Datadog trace semantics. Also, it's now possible to configure top-level Datadog span fields using OpenTelemetry span attributes (#3262).

For CI Visibility, you can now manually create CI traces and spans with the newly released API.

Added

• OpenTelemetry: Parse OpenTelemetry semantic conventions to Datadog's (#3273)
• OpenTelemetry: Support span reserved attribute overrides (#3262)
• Tracing: Ensure W3C tracestate is always propagated (#3255)

Changed

• Tracing: Set 128-bit trace_id to true by default (#3266)
• Tracing: Default trace propagation styles to Datadog,b3multi,b3,tracecontext (#3248,#3267)
• Ci-App: Upgraded datadog-ci dependency to 0.4 (#3270)

Read the full changeset and the release milestone.

1.16.2

This release reverts a change to appsec response body parsing that was introduced in 1.16.0 that may cause memory leaks.

Fixed

• Appsec: Revert parse response body fix introduced in 1.16.0 (#3252)

Read the full changeset and the release milestone.

1.16.1

Fixed

• Tracing: Fix concurrent-ruby future propagation without active_trace (#3242)
• Tracing: Fix host injection error handling (#3240)

Read the full changeset and the release milestone.

1.16.0

This release includes a security change for the Tracing Redis integration:

Currently, the Datadog Agent removes command arguments from the resource name. However there are cases, like Redis compressed keys, where this obfuscation cannot correctly remove command arguments. To safeguard that situation, the resource name set by the tracer will only be the command (e.g. SET) with no arguments. To retain the previous behavior and keep arguments in the span resource, with the potential risk of some command arguments not being fully obfuscated, set DD_REDIS_COMMAND_ARGS=true or set the option c.instrument :redis, command_args: true.

Added

• Tracing: Propagate trace through Concurrent::Promises.future (#1522)
• Core: Name Datadog::Core::Remote::Worker thread (#3207)

Changed

• Tracing: Redis - Omit command arguments from span.resource by default (#3235)
• Ci-app: Bump datadog-ci dependency from 0.2.0 to 0.3.0 (#3223)

Fixed

• Appsec: ASM parse response body (#3153)
• Appsec: ASM make sure to append content type and length information (#3204)
• Appsec: Make sure function that checks content-type header value accepts nil content-type header value (#3234)
• Profiling: Shut down profiler if any components failed (#3197)
• Tracing: Fix ActiveSupport instrumentation of custom cache stores (#3206)

Read the full changeset and the release milestone.

1.15.0

Highlights

Timeline view for Profiler beta

As of ddtrace 1.15.0, the Profiler now supports gathering data for the new Timeline view.

The Timeline view allows you to look at time-based patterns and work distribution over the period of a single profile: you can look at what individual threads were doing, and when 🎉

You can use the timeline view both when looking at individual profiles, as well as when scoped to a given trace.

You can enable it:

• Using an environment variable by setting DD_PROFILING_EXPERIMENTAL_TIMELINE_ENABLED=true
• Or via code by adding to your Datadog.configure block:

Datadog.configure do |c|
  # … existing configuration …
  c.profiling.advanced.experimental_timeline_enabled = true
end

Give it a try, let us know what you think!

(Note: We do not recommend enabling this feature prior to 1.15.0!)

google-protobuf dependency is no longer needed by the Profiler

As of ddtrace version 1.15.0, the google-protobuf gem is no longer needed to enable the Profiler.

If you've added this gem to your Gemfile/gems.rb file as part of enabling the Profiler, you can
remove it now. (If you're curious, we've internally replaced this dependency with the libdatadog gem.)

Configure blocking responses for AppSec via configuration or Remote Configuration

As of dd-trace-rb 1.15.0, AppSec supports configuring the blocking response.

You can configure the blocking response via:

• Using the ENV variables: DD_APPSEC_HTTP_BLOCKED_TEMPLATE_HTML=#{file_name}, and DD_APPSEC_HTTP_BLOCKED_TEMPLATE_JSON=#{file_name}
• Via code by adding to your Datadog.configure block:

Datadog.configure do |c|
  # … existing configuration …
  c.appsec.block.templates.html = "#{file_name}"
  c.appsec.block.templates.json = "#{file_name}"
end

• Using the Remote configuration UI. This option allows you to configure the status code and the blocking behaviour. You can redirect malicious attackers to custom pages.
  You can find more information on the official documentation

Configure agentless mode for CI visibility

If you are using CI visibility with a cloud CI provider without access to the underlying worker nodes, such as GitHub Actions or CircleCI, configure the library to use the Agentless mode.

For this, set the following environment variables:

• DD_CIVISIBILITY_AGENTLESS_ENABLED=true
• DD_API_KEY=<your_api_key>

Additionally, configure which Datadog site you want to send your data to:

• DD_SITE (default: datadoghq.com)

You can also enable agentless mode with Datadog.configure block:

Datadog.configure do |c|
  # … existing configuration …
  c.ci.agentless_mode_enabled = true
  # don't forget to set DD_API_KEY env variable!
end

Added

• Enable allocation counting feature by default for some Ruby 3 versions (#3176)
• Detect WebMock Cucumber and Rails.env to disable telemetry and remote configuration for development environment (#3065 , #3062 , #3145)
• Profiling: Import java-profiler PID controller and port it to C (#3190)
• Profiling: Record allocation type when sampling objects (#3096)
• Profiling: Include ruby vm type in profiler allocation samples (#3074)
• Tracing: Support Rack 3 (#3132 )
• Tracing: Support Opensearch 3 (#3189)
• Tracing: grpc adds client_error_handler option (#3095)
• Tracing: Add async option for test_mode configuration (#3158)
• Tracing: Implements _dd.base_service tag (#3018)
• Appsec: Allow blocking response template configuration via ENV variables (#2975)
• Appsec: ASM API security. Schema extraction (#3131, #3166, #3177)
• Appsec: Enable configuring blocking response via Remote Configuration (#3099)
• Ci-app: Validate git tags (#3100)
• Ci-app: Add agentless mode (#3186 )

Changed

• Appsec: Skip passing waf addresses when the value is empty (#3188)
• Profiling: Restore support for Ruby 3.3 (#3167)
• Profiling: Add approximate thread state categorization for timeline (#3162)
• Profiling: Wire up allocation sampling into CpuAndWallTimeWorker (#3103)
• Tracing: dalli disable memcached command tag by default (#3171)
• Tracing: Use first valid extracted style for distributed tracing (#2879 )
• Tracing: Rename configuration option on_set to after_set (#3107)
• Tracing: Rename experimental_default_proc to default_proc (#3091)
• Tracing: Use peer.service for sql comment propagation (#3127)
• Ci-app: Fix Datadog::CI::Environment to support the new CI specs (#3080)
• Bump datadog-ci dependency to 0.2 (#3186)
• Bump debase-ruby_core_source dependency to 3.2.2 (#3163)
• Upgrade libdatadog 5 (#3169, #3104)
• Upgrade libddwaf-rb 1.11.0 (#3087 )
• Update AppSec rules to 1.8.0 (#3140, #3139)

Fixed

• Profiling: Add workaround for incorrect invoke location when logging gem is in use (#3183)
• Profiling: Fix missing endpoint profiling when request_queuing is enabled in rack instrumentation (#3109)
• Appsec: Span tags reporting the number of WAF failed loaded rules (#3106)
• Tracing: Fix tagging with empty data (#3102)
• Tracing: Fix rails.cache.backend span tag with multiple stores (#3060)

Removed

• Profiling: Remove legacy profiler codepath (#3172)
• Ci-app: Remove CI module and add a dependency on datadog-ci gem (#3128)
• Tracing: Remove depends_on option from configuration DSL (#3085)
• Tracing: Remove delegate_to option from configuration DSL (#3086)

Read the full changeset and the release milestone.

1.14.0

Added

• Cucumber 8.0.0 support, test CI visibility with cucumber versions 6-8 (#3061)
• Tracing: Add ddsource to #to_log_format (#3025)
• Core: include peer service configurations in telemetry payload (#3056)
• Tracing: Improve quantization (#3041)

Changed

• Profiling: Disable profiler on Ruby 3.3 due to incompatibility (#3054)
• Core: EnvironmentLogger adjustments (#3020, #3057)

Fixed

• Appsec: Fix ASM setting for automated user events. (#3070)
• Tracing: Fix ActiveRecord adapter name for Rails 7 (#3051)

Read the full changeset and the release milestone.

1.13.1

Fixed

• Tracing: net/http instrumentation excludes query string for http.url tag (#3045)
• Tracing: Remove log_tags warning when given hash for log injection (#3022)
• Tracing: Fix OpenSearch integration loading (#3019)
• Core: Fix default hostname/port when mixing http and uds configuration (#3037)
• Core: Disable Telemetry and Remote Configuration in development environments (#3039)
• Profiling: Improve Datadog::Profiling::HttpTransport error logging (#3038)
• Docs: Document known issues with hanging Resque workers (#3033)

Read the full changeset and the release milestone.