MDC & Baggage AutoConfiguration for Micrometer Tracing with Brave and OpenTelemetry #32480

marcingrzejszczak · 2022-09-22T22:41:24Z

Rationale

In order to provide the users with the basic tracing observability story we need the following, additional features

W3C propagation type as the default propagation option (there would be 2 options, W3C and B3)
Baggage support (remote fields only (these are passed over the wire) - no local field support, no tagged field support)
- Correlation fields support (automatically ads to MDC given baggage entries)
MDC updating for OTel and Brave (we will NOT modify the logging format but we WILL prepare the MDC so that the logging format would pick those entries)

Conditionals

If Micrometer Tracing IS NOT on the classpath
- For Brave
  - [Context-Propagation] We will setup Brave B3 Propagation as default (there's no W3C support out of the box in Brave)
- For OTel
  - [Baggage] [Context-Propagation] We will setup W3C Propagation with Baggage as default
  - [Context-Propagation] We will setup W3C Propagation without Baggage when Baggage disabled via property
  - [Context-Propagation] We will setup B3 Propagation without Baggage when B3 propagation property turned on (there's no option to have baggage with B3 without Micrometer Tracing)
If Micrometer Tracing IS on the classpath
- For Brave
  - If baggage is not opt-out (management.tracing.baggage.enabled)
    - [Context-Propagation] [Baggage] By default we setup W3C context propagation with Baggage
    - [Context-Propagation] [Baggage] We can switch to B3 context propagation (via a property) with Baggage
    - [Baggage] If provided up-front via properties, we will set remote baggage entries
    - [Baggage] [MDC] If MDC is on the classpath
      - We will setup injecting correlated baggage entries to MDC (those need to be provided up-front)
  - If baggage is opt-out
    - [Context-Propagation] By default we setup W3C context propagation without Baggage (that will be possible with Tracing snapshots - W3CPropagation allows not having baggage micrometer-metrics/tracing#72)
    - [Context-Propagation] We can switch to B3 context propagation (via a property)
  - [MDC] If MDC is on the classpath
    - We will setup injecting spanId / traceId to MDC context
- For OTel:
  - [Baggage] [Context-Propagation] We will setup W3C Propagation with Baggage as default
  - [Context-Propagation] We will setup W3C Propagation without Baggage when Baggage disabled via property
  - [Baggage] [Context-Propagation] We will setup B3 Propagation with Baggage when B3 propagation property turned on
  - [Context-Propagation] We will setup B3 Propagation without Baggage when B3 propagation property turned on and baggage option disabled
  - [Baggage] [MDC] If MDC is on the classpath
    - We will setup injecting correlated baggage entries to MDC (those need to be provided up-front)
  - [MDC] If MDC is on the classpath
    - We will setup injecting spanId / traceId to MDC context

prerequisites

superseeds #32214

wilkinsona · 2022-09-26T13:18:28Z

...main/java/org/springframework/boot/actuate/autoconfigure/tracing/BraveAutoConfiguration.java

+			@ConditionalOnProperty(value = "management.tracing.propagation.type", havingValue = "W3C",
+					matchIfMissing = true)
+			Factory w3cPropagationNoBaggageFactory() {
+				return new W3CPropagation(BRAVE_BAGGAGE_MANAGER, List.of()); // TODO: Use


micrometer-metrics/tracing@29feef8 is the change that we need so that there's no baggage.

Correct, I mean if we leave this as it is the components for baggage (i.e. BraveBaggageManager) will be called, but no baggage will be propagated cause there's nothing configured. With this change in tracing we are nulling anything baggage related so that will not be called at all

wilkinsona · 2022-09-26T13:21:56Z

.../src/main/java/org/springframework/boot/actuate/autoconfigure/tracing/TracingProperties.java

+		/**
+		 * Whether to enable correlation of the baggage context with logging contexts.
+		 */
+		private boolean correlationEnabled = true;
+
+		/**
+		 * List of fields that should be correlated with the logging context. That means
+		 * that these fields would end up as key-value pairs in e.g. MDC.
+		 */
+		private List<String> correlationFields = new ArrayList<>();


It looks as if there could be a correlation group with enabled and fields properties.

Yeah, that's a good idea!

wilkinsona · 2022-09-26T13:22:58Z

.../src/main/java/org/springframework/boot/actuate/autoconfigure/tracing/TracingProperties.java

@@ -53,4 +74,81 @@ public void setProbability(float probability) {

 	}

+	public static class Baggage {


This class could have an enabled property, removing the need for some additional metadata.

wilkinsona · 2022-09-26T13:24:16Z

...ator-autoconfigure/src/main/resources/META-INF/additional-spring-configuration-metadata.json

+    },
+    {
+      "name": "management.tracing.sampling.probability",
+      "defaultValue": "0.10"


Does the annotation processor not pick up this default automatically?

Intellij told me that it doesn't 🤷

It appears to be wrong (https://docs.spring.io/spring-boot/docs/3.0.0-M5/reference/html/application-properties.html#application-properties.actuator.management.tracing.sampling.probability) so this can be reverted.

Please check default values are produced by building on the command-line.

See gh-32480

…d Baggage" See gh-32480

See spring-projectsgh-32480

marcingrzejszczak mentioned this pull request Sep 22, 2022

MDC auto-configuration for Brave and OpenTelemetry #32214

Closed

spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Sep 22, 2022

marcingrzejszczak force-pushed the mdcAutoConfiguration branch 2 times, most recently from 0bce626 to 4552de2 Compare September 23, 2022 12:05

This was referenced Sep 23, 2022

MDC trace and span IDs for Brave and OTel #32488

Closed

Context Propagation via W3C and B3 for OTel and Brave #32487

Closed

marcingrzejszczak force-pushed the mdcAutoConfiguration branch 3 times, most recently from c549826 to 3e1424b Compare September 23, 2022 14:28

wilkinsona reviewed Sep 26, 2022

View reviewed changes

[Observability] MDC, Context Propagation (via B3 and W3C) and Baggage

a257408

marcingrzejszczak force-pushed the mdcAutoConfiguration branch from 3e1424b to 1b21cb2 Compare September 26, 2022 13:46

Applied changes following the review

e8730ac

marcingrzejszczak force-pushed the mdcAutoConfiguration branch from 1b21cb2 to e8730ac Compare September 26, 2022 13:49

wilkinsona self-assigned this Sep 26, 2022

wilkinsona added type: enhancement A general enhancement and removed status: waiting-for-triage An issue we've not yet triaged labels Sep 26, 2022

wilkinsona added this to the 3.0.x milestone Sep 26, 2022

Requires Micrometer Tracing as a dependency for Brave Autoconfiguration

891d148

wilkinsona modified the milestones: 3.0.x, 3.0.0-RC1 Sep 27, 2022

wilkinsona pushed a commit that referenced this pull request Sep 27, 2022

Add support for MDC, Context Propagation (via B3 and W3C), and Baggage

52d1436

See gh-32480

wilkinsona added a commit that referenced this pull request Sep 27, 2022

Polish "Add support for MDC, Context Propagation (via B3 and W3C), an…

36a4b36

…d Baggage" See gh-32480

wilkinsona closed this in 3acd9b8 Sep 27, 2022

This was referenced Oct 3, 2022

Add log correlation for Micrometer Tracing #31468

Closed

Add support for baggage in tracing #31522

Closed

izeye mentioned this pull request Oct 6, 2022

Polish Micrometer tracing changes #32627

Closed

izeye added a commit to izeye/spring-boot that referenced this pull request Oct 6, 2022

Polish Micrometer tracing changes

1d31ec4

See spring-projectsgh-32480

braunsonm mentioned this pull request Jan 10, 2023

RestTemplateBuilder and WebClient.Builder do not have instrumentation to propagate traces #33742

Closed

szpak mentioned this pull request Apr 13, 2023

Improve documentation for baggage propagation into MDC #34977

Closed

wilkinsona mentioned this pull request May 18, 2023

OpenTelemetryAutoConfiguration: allow to override SpanProcessor bean #35558

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

MDC & Baggage AutoConfiguration for Micrometer Tracing with Brave and OpenTelemetry #32480

MDC & Baggage AutoConfiguration for Micrometer Tracing with Brave and OpenTelemetry #32480

Uh oh!

marcingrzejszczak commented Sep 22, 2022 •

edited

Loading

Uh oh!

wilkinsona Sep 26, 2022

Uh oh!

marcingrzejszczak Sep 26, 2022

Uh oh!

wilkinsona Sep 26, 2022

Uh oh!

marcingrzejszczak Sep 26, 2022

Uh oh!

wilkinsona Sep 26, 2022

Uh oh!

wilkinsona Sep 26, 2022

Uh oh!

marcingrzejszczak Sep 26, 2022

Uh oh!

wilkinsona Sep 26, 2022

Uh oh!

snicoll Sep 27, 2022

Uh oh!

Uh oh!

		@@ -53,4 +74,81 @@ public void setProbability(float probability) {

		}

		public static class Baggage {

MDC & Baggage AutoConfiguration for Micrometer Tracing with Brave and OpenTelemetry #32480

MDC & Baggage AutoConfiguration for Micrometer Tracing with Brave and OpenTelemetry #32480

Uh oh!

Conversation

marcingrzejszczak commented Sep 22, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale

Conditionals

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

marcingrzejszczak commented Sep 22, 2022 •

edited

Loading