High log volume can cause serious performance degredation

[CamdenClark]

When using this library in a project, an upstream library was using apache.http.*, which has significant trace logging. It looks like on every log message this library calls .getStackTrace, which causes a really significant spike in CPU usage, even if our log level isn't at trace.

[rufoa]

Thanks, will look into this

[rufoa]

Does putting apache.http.* in timbre's :ns-blacklist resolve the issue?

I realise this isn't a proper solution but the blacklist should be checked before the call to getStackTrace occurs.

[CamdenClark]

Yes, that was a successful temporary fix for us.

[CamdenClark]

We were scared that there was unknown impact--any time any new upstream dependency is added we're nervous about the potential for new spats of trace logging that causes a performance degredation that's opaque unless you're explicitly looking for it.

[rufoa]

Yes I understand your concern. Thanks for confirming the blacklist works.

Please could you now check timbre's :ns-log-level works?

Put :ns-log-level [["apache.http.*" :error]] in your config, and check the calls to getStackTrace are skipped

[ieugen]

Is this still an issue?
Should we check if we have "apache.http.*" logging in our setup?
Is there a better/more reliable way to check without "watching the CPU usage" ?

[greglook]

We (Amperity) eventually wound up writing a new logging library that combines some of the concepts in timbre with a direct SLF4J implementation, so it doesn't suffer from some of the handoff pain this ticket is an example of.

[ieugen]

@greglook : thanks for the suggestion.
Will check it out in depth.

We have a big app that uses libraries with all sorts of logging.
As of right now, we can't filter logging from other logging frameworks.
This is kind of a deal breaker.

[ptaoussanis]

Hi folks - in case this is still relevant for you, you might want to try Timbre v6.6.0-RC1 which now includes built-in SLF4J(v2) support that may be more performant.

Alternatively, if general performance is a concern - I'd consider Telemere which also has built-in SLF4J interop but can be dramatically faster (~350 nanosecs / call with full filtering). It includes a shim for easily migrating from Timbre.