Upgrading from 1.2 to 1.4: <if> nested into <appender> is disallowed

[ziomill]

Hi all,

I'm upgrading from logback 1.2 to 1.4.

Our logback config has some IF nested into the APPENDER and, as described here, it seems that from the logback 1.3 version, this configuration is disallowed:

<configuration>

    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
            <providers>
                <!-- TIMESTAMP field (UTC) -->
                <timestamp>
                    <fieldName>Timestamp</fieldName>
                    <timeZone>UTC</timeZone>
                </timestamp>
                <!-- Include TRACE-ID field from MDC -->
                <pattern>
                    <omitEmptyFields>true</omitEmptyFields>
                    <pattern>
                        {"TraceId":"%mdc{trace_id}"}
                    </pattern>
                </pattern>
                <!-- Include SPAN-ID field from MDC -->
                <pattern>
                    <omitEmptyFields>true</omitEmptyFields>
                    <pattern>
                        {"SpanId":"%mdc{span_id}"}
                    </pattern>
                </pattern>
                <!-- LOG_LEVEL field -->
                <logLevel>
                    <fieldName>SeverityText</fieldName>
                </logLevel>
                <!-- MESSAGE field -->
                <message>
                    <fieldName>Body</fieldName>
                </message>
                <!-- RESOURCE nested object -->
                <nestedField>
                    <fieldName>Resource</fieldName>
                    <providers>
                        <!-- NAMESPACE field -->
                        <if condition='isDefined("CONTAINER_NAMESPACE")'>
                            <then>
                                <pattern>
                                    <pattern>
                                        {"container-namespace":"${CONTAINER_NAMESPACE}"}
                                    </pattern>
                                </pattern>
                            </then>
                        </if>
                        <!-- CONTAINER_NAME field -->
                        <if condition='isDefined("CONTAINER_NAME")'>
                            <then>
                                <pattern>
                                    <pattern>
                                        {"container-name":"${CONTAINER_NAME}"}
                                    </pattern>
                                </pattern>
                            </then>
                        </if>
                        <!-- CONTAINER_IMAGE_VERSION field -->
                        <if condition='isDefined("CONTAINER_IMAGE_VERSION")'>
                            <then>
                                <pattern>
                                    <pattern>
                                        {"container-version":"${CONTAINER_IMAGE_VERSION}"}
                                    </pattern>
                                </pattern>
                            </then>
                        </if>
                    </providers>
                </nestedField>
                <!-- ATTRIBUTES nested object -->
                <nestedField>
                    <fieldName>Attributes</fieldName>
                    <providers>
                        <!-- TAG field -->
                        <pattern>
                            <pattern>
                                {"tag":"applicative_log"}
                            </pattern>
                        </pattern>
                        <!-- THREAD field -->
                        <threadName>
                            <fieldName>thread</fieldName>
                        </threadName>
                        <!-- CLASS field -->
                        <loggerName>
                            <fieldName>class</fieldName>
                        </loggerName>
                        <stackTrace>
                            <fieldName>stacktrace</fieldName>
                        </stackTrace>
                    </providers>
                </nestedField>
            </providers>
        </encoder>
    </appender>

    <root level="INFO">
        <appender-ref ref="STDOUT" />
    </root>

</configuration>

As you can see, we have to add some JSON properties to a log's record, only if some ENV variables have been provided.

I tried using the "surrounding-if form" as suggested, but the original appender is completely overwritten by the last IF which evaluates to TRUE:

<configuration>

    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
            <providers>
                <!-- TIMESTAMP field (UTC) -->
                <timestamp>
                    <fieldName>Timestamp</fieldName>
                    <timeZone>UTC</timeZone>
                </timestamp>
                <!-- Include TRACE-ID field from MDC -->
                <pattern>
                    <omitEmptyFields>true</omitEmptyFields>
                    <pattern>
                        {"TraceId":"%mdc{trace_id}"}
                    </pattern>
                </pattern>
                <!-- Include SPAN-ID field from MDC -->
                <pattern>
                    <omitEmptyFields>true</omitEmptyFields>
                    <pattern>
                        {"SpanId":"%mdc{span_id}"}
                    </pattern>
                </pattern>
                <!-- LOG_LEVEL field -->
                <logLevel>
                    <fieldName>SeverityText</fieldName>
                </logLevel>
                <!-- MESSAGE field -->
                <message>
                    <fieldName>Body</fieldName>
                </message>
                <!-- RESOURCE nested object -->
                <nestedField>
                    <fieldName>Resource</fieldName>
                    <providers>
                    </providers>
                </nestedField>
                <!-- ATTRIBUTES nested object -->
                <nestedField>
                    <fieldName>Attributes</fieldName>
                    <providers>
                        <!-- TAG field -->
                        <pattern>
                            <pattern>
                                {"tag":"applicative_log"}
                            </pattern>
                        </pattern>
                        <!-- THREAD field -->
                        <threadName>
                            <fieldName>thread</fieldName>
                        </threadName>
                        <!-- CLASS field -->
                        <loggerName>
                            <fieldName>class</fieldName>
                        </loggerName>
                        <stackTrace>
                            <fieldName>stacktrace</fieldName>
                        </stackTrace>
                    </providers>
                </nestedField>
            </providers>
        </encoder>
    </appender>

   <!-- NAMESPACE field -->
    <if condition='isDefined("CONTAINER_NAMESPACE")'>
        <then>
            <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
                <encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
                    <providers>
                        <nestedField>
                            <fieldName>Resource</fieldName>
                                <providers>
                                    <pattern>
                                        <pattern>
                                            {"container-namespace":"${CONTAINER_NAMESPACE}"}
                                        </pattern>
                                    </pattern>
                                </providers>
                        </nestedField>
                    </providers>
                </encoder>
            </appender>
        </then>
    </if>
    <!-- CONTAINER_NAME field -->
    <if condition='isDefined("CONTAINER_NAME")'>
        <then>
            <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
                <encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
                    <providers>
                        <nestedField>
                            <fieldName>Resource</fieldName>
                            <providers>
                                <pattern>
                                    <pattern>
                                        {"container-name":"${CONTAINER_NAME}"}
                                    </pattern>
                                </pattern>
                            </providers>
                        </nestedField>
                    </providers>
                </encoder>
            </appender>
        </then>
    </if>
    <!-- CONTAINER_IMAGE_VERSION field -->
    <if condition='isDefined("CONTAINER_IMAGE_VERSION")'>
        <then>
            <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
                <encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
                    <providers>
                        <nestedField>
                            <fieldName>Resource</fieldName>
                                <providers>
                                    <pattern>
                                        <pattern>
                                            {"container-version":"${CONTAINER_IMAGE_VERSION}"}
                                        </pattern>
                                    </pattern>
                                </providers>
                        </nestedField>
                    </providers>
                </encoder>
            </appender>
        </then>
    </if>

    <root level="INFO">
        <appender-ref ref="STDOUT" />
    </root>


</configuration>

For example, if I provide the CONTAINER_NAME env property, the printed log is:

{"Resource":{"container-name":"my-container"}}

How to reach the same result according to the new rules?

Thank you

[ceki]

@ziomill Thank you for this detailed description.

Are the CONTAINER_NAMESPACE, CONTAINER_NAME and CONTAINER_IMAGE_VERSION mutually exclusive or mostly independent? For example, can they all be defined simultaneously?

[ziomill]

Hi @ceki ,

The properties are independent.

Anyway, I could avoid using nested IF in the appender if the logback library would allow me to evaluate ENV variables to null (or to an empty string) when they are not provided, so that I can configure something this:

  ....
  ....
<pattern>
    <omitEmptyFields>true</omitEmptyFields>
    <pattern>
        {
            "Resource": {"container-namespace":"${CONTAINER_NAMESPACE:-}"}
        }
    </pattern>
</pattern>
....
....

However logback evaluates not provided ENV variables to "_IS_UNDEFINED" and I can't rely on the xml<omitEmptyFields>true</omitEmptyFields> strategy provided by the LoggingEventCompositeJsonEncoder

The problem, which is related to this logback issue, has been explained with details here

Any help would be appreciated
Thank you!

[ceki]

I think the following should do the trick:

<configuration>

  <variable name="empty" value="" />
  ...  
    <pattern>
      <omitEmptyFields>true</omitEmptyFields>
      <pattern>
          {
            "Resource": {"container-namespace":"${CONTAINER_NAMESPACE:-${empty}}"}
         }
      </pattern>
   </pattern>
   ....
</configuration>

[ziomill]

HI @ceki

I have tried your configuration, but it doesn't work:

"Resource":{"container-namespace":"empty_IS_UNDEFINED"

Thank you for your support!

[ziomill]

Hi @ceki

Just to add that, after some tests, your trick seems to work only if the {empty} variable is referenced with the %property{empty}.
Accessing it through the ${empty} doesn't work:

<variable name="empty" value="" />
<pattern>
   <omitEmptyFields>true</omitEmptyFields>
      <pattern>
         { "container-namespace" : "${CONTAINER_NAMESPACE:-%property{empty}}" }
      </pattern>
</pattern>