Add StructuredLogFormatter wrapper for JsonEncoder (logback)

[sustacek]

Hi, I wanted to be able to use the JsonEncoder for Logback natively with SpringBoot, but for now I always have to create a logback.xml. SpringBoot 3.4 introduced support for structured formatting, without the need to write logback.xml / log4j.xml.

To support his, I created a simple wrapper. Then user will be able to use the encoder as a formatter for structured logging with just a regular application.properties / .yaml, like this:

# application.yaml
logging:
  structured:
    format:
      console: com.sap.hcp.cf.logback.boot.CloudFoundryStructuredLogFormatter
      #console: ecs
  level:
    root: INFO
    com.sap.cloud.sdk: INFO
    com.sap.cloud.security: DEBUG

Let me know what you think. In our application, I've created a similar wrapper, but since JsonEncoder does not expose the getJson(event) method, it's a little bit cumbersome -- I had to use reflection. I didn't want to use the public encode(event) because I'd need to decode it right back using UTF-8 into a String instance, every time.

Alternatively, if you could expose the JsonEncoder.getJson(event) method (or a new one, returning JSON as String), that'd help us a lot as well. I know this commit drags with itself a dependency on SpringBoot, which might not be ideal.

[cla-assistant]

All committers have signed the CLA.

[KarstenSchnitter]

Hi,

thanks for raising the issue. I had a look at the participating classes and interfaces and must say, this raises some eyebrows. Let's start with ch.qos.logback.core.encoder.Encoder which is implemented by com.sap.hcp.cf.logback.encoder.JsonEncoder.

public interface Encoder<E> extends ContextAware, LifeCycle {
    byte[] headerBytes();

    byte[] encode(E var1);

    byte[] footerBytes();
}

Logback is expecting an Encoder to transform an event of type E (usually ILoggingEvent) into a byte array. Spring on the other hand defines org.springframework.boot.logging.structured.StructuredLogFormatter.

@FunctionalInterface
public interface StructuredLogFormatter<E> {

	String format(E event);

	default byte[] formatAsBytes(E event, Charset charset) {
		return format(event).getBytes(charset);
	}
}

Spring is expecting implementations of StructuredLogFormatter to emit String instances instead of byte arrays but provides a default implementation to transform the String into a byte array. This is a strange decision given what Logback provides. But comparing against Log4j2 it is very similar to the interfaces we find in that framework.

Now the interesting part happens, when we look where the StructuredLogFormatter is actually used: org.springframework.boot.logging.logback.StructuredLogEncoder:

public class StructuredLogEncoder extends EncoderBase<ILoggingEvent> {

	@Override
	public byte[] headerBytes() {
		return null;
	}

	@Override
	public byte[] encode(ILoggingEvent event) {
		return this.formatter.formatAsBytes(event, (this.charset != null) ? this.charset : StandardCharsets.UTF_8);
	}

	@Override
	public byte[] footerBytes() {
		return null;
	}
}

If you use Logback, the String based method is only called indirectly over the default method. That means in your wrapper you can overwrite the formatAsBytes method to directly call the JsonEncoder. For format you can transform the byte array into a String without performance penalty, since this code path will not be executed. You could throw an UnsupportedOperationException for testing.

For this library, I am open to change the visibility of JsonEncoder.getJson to public to ease the integration with Spring. Due to the viable work-around explained above, I do not think, that this warrants a release of the library on its own. I am also open to provide a StructuredLogFormatter in a separate module of cf-java-logging-support, but only in the new major release 4.x.