feat(django): add DD_DJANGO_TRACING_MINIMAL configuration option (#14468)

The new `DD_DJANGO_TRACING_MINIMAL` configuration option (off by
default) will disable Django ORM, cache, and template spans. This has a
sizable improvement on performance overhead of the traced application.

Django applications generally have both the Django ORM/cache/template
spans generated as well as the underlying engines (psycopg, redis,
jinja2, etc). This means we will create duplicate spans for every db
query, cache call, or template render. Setting
`DD_DJANGO_TRACING_MINIMAL=true` will completely disable these
additional spans and should help measurably with performance overhead of
the application. Our microbenchmarks show about a 20% reduction in mean
request latency when this setting is enabled (my local load tests were
showing a 34% reduction in mean request latency).

Note: Future major versions of our library may set this setting by
default. That way you only need to explicitly enable this additional
Django tracing if we do not yet support tracing of the underlying engine
used for db/cache/template.


## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

---------

Co-authored-by: Alberto Vara <alberto.vara@datadoghq.com>
Co-authored-by: Gabriele N. Tornetta <P403n1x87@users.noreply.github.com>
Co-authored-by: Taegyun Kim <taegyun.kim@datadoghq.com>
Co-authored-by: dd-octo-sts[bot] <200755185+dd-octo-sts[bot]@users.noreply.github.com>
Co-authored-by: quinna-h <175135214+quinna-h@users.noreply.github.com>
Co-authored-by: Gary Huang <garyhuang@hotmail.ca>
Co-authored-by: Yun Kim <35776586+Yun-Kim@users.noreply.github.com>
Co-authored-by: Christophe Papazian <114495376+christophe-papazian@users.noreply.github.com>
Co-authored-by: ncybul <124532568+ncybul@users.noreply.github.com>
Co-authored-by: Rachel Yang <rachel.yang@datadoghq.com>
Co-authored-by: Sam Brenner <106700075+sabrenner@users.noreply.github.com>
Co-authored-by: wantsui <wan.tsui@datadoghq.com>
Co-authored-by: Nick Ripley <nick.ripley@datadoghq.com>
Co-authored-by: Munir Abdinur <munir.abdinur@datadoghq.com>
Co-authored-by: Emmett Butler <723615+emmettbutler@users.noreply.github.com>
Co-authored-by: Manuel Álvarez Álvarez <manuel-alvarez-alvarez@users.noreply.github.com>
Co-authored-by: Joey Zhao <5253430+joeyzhao2018@users.noreply.github.com>
Co-authored-by: Tyler Finethy <tyler.finethy@datadoghq.com>
Co-authored-by: Federico Mon <federico.mon@datadoghq.com>
Co-authored-by: Tyler Finethy <tylfin@gmail.com>
Co-authored-by: Florentin Labelle <florentin.labelle@datadoghq.com>

Author: 21 people

.gitlab/benchmarks/bp-runner.microbenchmarks.fail-on-breach.yml

@@ -57,6 +57,10 @@ experiments:
             thresholds:
               - execution_time < 21.75 ms
               - max_rss_usage < 66.00 MB
+          - name: djangosimple-tracer-minimal
+            thresholds:
+              - execution_time < 17.50 ms
+              - max_rss_usage < 66.00 MB
           - name: djangosimple-tracer-native
             thresholds:
               - execution_time < 21.75 ms

benchmarks/django_simple/config.yaml

@@ -10,6 +10,7 @@ baseline: &baseline
   always_create_database_spans: true
   django_instrument_templates: true
   native_writer: false
+  django_minimal: false
 tracer: &tracer
   <<: *baseline
   tracer_enabled: true
@@ -52,3 +53,6 @@ tracer-no-templates:
 tracer-native:
   <<: *tracer
   native_writer: true
+tracer-minimal:
+  <<: *tracer
+  django_minimal: true

benchmarks/django_simple/scenario.py

@@ -17,6 +17,7 @@ class DjangoSimple(bm.Scenario):
     always_create_database_spans: bool
     django_instrument_templates: bool
     native_writer: bool
+    django_minimal: bool
 
     def run(self):
         os.environ["DJANGO_SETTINGS_MODULE"] = "app"
@@ -26,6 +27,14 @@ def run(self):
         os.environ["DD_DJANGO_ALWAYS_CREATE_DATABASE_SPANS"] = "1" if self.always_create_database_spans else "0"
         os.environ["DD_DJANGO_INSTRUMENT_TEMPLATES"] = "1" if self.django_instrument_templates else "0"
 
+        # Use only the minimal setting and the defaults for everything else (mostly based on the value of "minimal")
+        if self.django_minimal:
+            os.environ["DD_DJANGO_TRACING_MINIMAL"] = "1"
+            del os.environ["DD_DJANGO_INSTRUMENT_CACHES"]
+            del os.environ["DD_DJANGO_INSTRUMENT_DATABASES"]
+            del os.environ["DD_DJANGO_ALWAYS_CREATE_DATABASE_SPANS"]
+            del os.environ["DD_DJANGO_INSTRUMENT_TEMPLATES"]
+
         if self.profiler_enabled:
             os.environ.update(
                 {"DD_PROFILING_ENABLED": "1", "DD_PROFILING_API_TIMEOUT": "0.1", "DD_PROFILING_UPLOAD_INTERVAL": "10"}

ddtrace/contrib/internal/django/__init__.py

@@ -85,6 +85,29 @@
 
    Default: ``False``
 
+.. envvar:: DD_DJANGO_TRACING_MINIMAL
+
+   Enables minimal tracing mode for performance-sensitive applications. When enabled, this disables
+   Django ORM, cache, and template instrumentation while keeping middleware instrumentation enabled.
+   This can significantly reduce overhead by removing Django-specific spans while preserving visibility
+   into the underlying database drivers, cache clients, and other integrations.
+
+   This is equivalent to setting:
+   - ``DD_DJANGO_INSTRUMENT_TEMPLATES=false``
+   - ``DD_DJANGO_INSTRUMENT_DATABASES=false``
+   - ``DD_DJANGO_INSTRUMENT_CACHES=false``
+
+   For example, with ``DD_DJANGO_INSTRUMENT_DATABASES=false``, Django ORM query spans are disabled
+   but database driver spans (e.g., psycopg, MySQLdb) will still be created, providing visibility
+   into the actual database queries without the Django ORM overhead.
+
+   Consider using this option if your application is performance-sensitive and the additional
+   Django-layer spans are not required for your observability needs.
+
+   Default: ``False``
+
+   *New in version v3.15.0.*
+
 .. py:data:: ddtrace.config.django['instrument_middleware']
 
    Whether or not to instrument middleware.

ddtrace/contrib/internal/django/patch.py

@@ -39,6 +39,9 @@
 
 log = get_logger(__name__)
 
+# TODO[4.0]: Change this to True by default
+DJANGO_TRACING_MINIMAL = asbool(_get_config("DD_DJANGO_TRACING_MINIMAL", default=False))
+
 config._add(
     "django",
     dict(
@@ -49,10 +52,11 @@
         trace_fetch_methods=asbool(os.getenv("DD_DJANGO_TRACE_FETCH_METHODS", default=False)),
         distributed_tracing_enabled=True,
         instrument_middleware=asbool(os.getenv("DD_DJANGO_INSTRUMENT_MIDDLEWARE", default=True)),
-        instrument_templates=asbool(os.getenv("DD_DJANGO_INSTRUMENT_TEMPLATES", default=True)),
-        instrument_databases=asbool(os.getenv("DD_DJANGO_INSTRUMENT_DATABASES", default=True)),
+        instrument_templates=asbool(os.getenv("DD_DJANGO_INSTRUMENT_TEMPLATES", default=not DJANGO_TRACING_MINIMAL)),
+        instrument_databases=asbool(os.getenv("DD_DJANGO_INSTRUMENT_DATABASES", default=not DJANGO_TRACING_MINIMAL)),
+        # TODO[4.0]: remove this option and make it the default behavior when databases are instrumented
         always_create_database_spans=asbool(os.getenv("DD_DJANGO_ALWAYS_CREATE_DATABASE_SPANS", default=True)),
-        instrument_caches=asbool(os.getenv("DD_DJANGO_INSTRUMENT_CACHES", default=True)),
+        instrument_caches=asbool(os.getenv("DD_DJANGO_INSTRUMENT_CACHES", default=not DJANGO_TRACING_MINIMAL)),
         trace_query_string=None,  # Default to global config
         include_user_name=asm_config._django_include_user_name,
         include_user_email=asm_config._django_include_user_email,

releasenotes/notes/add-django-tracing-minimal-d995af59e59e748e.yaml

@@ -0,0 +1,9 @@
+---
+features:
+  - |
+    django: This introduces the ``DD_DJANGO_TRACING_MINIMAL`` environment variable for performance-sensitive applications.
+    When enabled, this disables Django ORM, cache, and template instrumentation while keeping middleware instrumentation enabled.
+    This significantly reduces overhead by removing Django-specific spans while preserving visibility into the underlying
+    database drivers, cache clients, and other integrations. For example, with this enabled, Django ORM query spans are
+    disabled but database driver spans (e.g., psycopg, MySQLdb) will still be created. To enable minimal tracing,
+    set ``DD_DJANGO_TRACING_MINIMAL=true``.