[Java/Logging] Questions about attributes emitted by log statements.

[hagbard]

Sorry if these questions are already answered in the specification, but I've not been able to find definitive answers.

As part of the planning for a Flogger specific backend for OTel, I've been reading through the specification to try and determine the exact semantics. However, I'm still not entirely clear on the expectations a user should have when logging is integrated with OTel.

I see the example code in:
https://github.com/open-telemetry/opentelemetry-demo/blob/42d2ed94217707200037a60acdb9466c73392a2b/src/adservice/src/main/java/oteldemo/AdService.java
but it's not obvious how calls to the Span class are intended to interact with the log statements.

Span span = Span.current();
try {
  ...
  span.setAttribute("app.ads.contextKeys", req.getContextKeysList().toString());
  span.setAttribute("app.ads.contextKeys.count", req.getContextKeysCount());
  if (req.getContextKeysCount() > 0) {
    logger.info("Targeted ad request received for " + req.getContextKeysList());
    ...
  } else {
    logger.info("Non-targeted ad request received, preparing random response.");
    ...
  }
  ...
  span.setAttribute("app.ads.count", allAds.size());
  span.setAttribute("app.ads.ad_request_type", adRequestType.name());
  span.setAttribute("app.ads.ad_response_type", adResponseType.name());

  adRequestsCounter.add(
      1,
      Attributes.of(
          adRequestTypeKey,
          adRequestType.name(),
          adResponseTypeKey,
          adResponseType.name()));
  if (checkAdFailure()) {
    logger.warn(ADSERVICE_FAIL_FEATURE_FLAG + " fail feature flag enabled");
    throw new StatusRuntimeException(Status.RESOURCE_EXHAUSTED);
  }
  ...
} catch (StatusRuntimeException e) {
  span.addEvent(
      "Error",
      Attributes.of(AttributeKey.stringKey("exception.message"), e.getMessage()));
  span.setStatus(StatusCode.ERROR);
  logger.log(Level.WARN, "GetAds Failed with status {}", e.getStatus());
  ...
}

In this code various attributes are set on the span during its lifetime (implying spans are mutable).
And various log statements are made at different points, before and after attributes are set.

My main question is:

Should a user expect each log statement made in this context to have all current attributes attached to the current span at the time the log statement is made emitted with it? Or, are attributes for logging only those which are somehow attached to a logging specific context (e.g. Log4J's MDC)?

As someone potentially writing a bridging API, I'd like to think that the OTel Logger handles any automatically emitted attributes from the current context, but it's not really clear from that example (perhaps there's a better bit of example Java code I missed however).

The example of the adding of an error event to the span just prior to logging (at the bottom of the code example) is interesting to me. Should the log statement somehow pick up the "Error" event and treat it as if the log statement had an exception passed to it? Or is this a replacement for passing the exception to the log statement?

I'm assuming that not all "general logging metadata" should be emitted to OTel though (Flogger has metadata for things like the degree of rate limiting on a log statement, which would likely not be known to an OTel schema).

It feels like logging of OTel attributes needs to be essentially "opt in".

[jkwatson]

AFAIK, there is no expectation that span attributes will automatically be stuck into structured logs. The only things that I know of would be the trace id and span id, and not attributes unless specifically configured (or explicitly) to be included.

[trask]

baggage or (maybe in the future) open-telemetry/oteps#207 could be options for propagating attributes to both spans and (child) logs