Consider moving away from Bunyan

[NuSkooler]

Bunyan has worked amazingly well and I'm very happy with it overall. However, it at least appears to have become a dead project. See here and [here](https://github.com/trentm/node-bunyan/issues/586], etc.

Consider moving away from Bunyan to something similar:

• JSON based logging
• Pretty-printer
• Similar enough syntax that the upgrade is minimal. The general signature for log methods is ([object], message)
• Log rotation (may need to be separate, but should be built in to ENiGMA)

Pino seems to be a continuation of the Bunyan spirit. One thing that is missing is log rotation, so something else would need to be created here.

Others?

[davestephens]

Funnily enough I'm working on a massive structured logging project at work at the moment, and the general consensus is "JSON sucks for logs", mostly because it's difficult to read without additional tooling. (Enigma docs re logging case in point). It's fine if you're jamming everything into the likes of ELK, but nobody running ENiG is. We've also come to the consensus that to keep structure but whilst still maintaining readability, logfmt is the way to go. Heroku maintain node-logfmt and it looks pretty decent...

…

On Thu, 4 Jun 2020, 04:50 Bryan Ashby, ***@***.***> wrote: Bunyan has worked amazingly well and I'm very happy with it overall. However, it at least appears to have become a dead project. See here <trentm/node-bunyan#640> and [here]( trentm/node-bunyan#586], etc. Consider moving away from Bunyan to something similar: - JSON based logging - Pretty-printer - Similar enough syntax that the upgrade is minimal. The general signature for log methods is ([object], message) - Log rotation (may need to be separate, but should be built in to ENiGMA) Pino <http://getpino.io/#/> seems to be a continuation of the Bunyan spirit. One thing that is missing is log rotation, so something else would need to be created here. Others? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#286>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFDGMHZB6HDHD4SWMYCU3DRU4K2BANCNFSM4NSHTTOA> .

[NuSkooler]

Funnily enough I'm working on a massive structured logging project at work
at the moment, and the general consensus is "JSON sucks for logs"

At my work place we just moved to JSON for a number of logs for the exact opposite reason. 😆

I'm not sure when you don't have the most basic of tooling (tail is a tool; piping to bunyan currently is part of the same pipeline). I suppose if you load a log into a text editor, but I just dump it all through the processor first even if handed a .log file. With systemd you have a pure binary format, but use journalctl to view it (and extract to text if you want in a number of formats including JSON). I've looked at logfmt a number of times. From their main page:

In development, a log output formatter can then give the msg field special treatment by displaying it in way that a human can easily read (along with other special fields like level)

(they suggest using a formatter because this sucks):

level=info msg="Stopping all fetchers"
  tag=stopping_fetchers id=ConsumerFetcherManager-1382721708341
  module=kafka.consumer.ConsumerFetcherManager

But this is nice:

info | Stopping all fetchers          module=kafka.consumer.ConsumerFetcherManager
info | Performing log compaction      module=kafka.compacter.LogCompactionManager
info | Performing garbage collection  module=kafka.cleaner.GarbageCollectionManager
info | Starting all fetchers          module=kafka.consumer.ConsumerFetcherManager

Perhaps it would be nice if the log allowed backends for particular formats?

[coolacid]

In $day job, my team uses python, so you may have to extrapolate a bit.
The python logging system supports a msg, and extra

• msg is a human readable message that describes the error. This is what is used in normal file/console logging handlers
• extra is a dict with structured important info

We then built a custom logging handler that takes the msg and extra bits for forwarding to a message queue for handling.

If nodejs doesn't have something like this directly, it might be worth having an internal class that implements the same sort of details - thus, the end user can get the info they need.

[NuSkooler]

@coolacid This is essentially what ENiGMA already does:

Example:

const extra = { // usually you'd inline this, but for illustration
  foo : "bar",
  baz : 1234,
};

log.error(extra, "Something went terribly wrong!");

[coolacid]

@NuSkooler This is what I get for reading email and not looking at code (that I haven't looked at in a while).

Having said that, as a part time dev, part time pipeline designer, I'd go with the last statement in your last message Perhaps it would be nice if the log allowed backends for particular formats?

[davestephens]

Yeah, agree with that. JSON truly is poo on the terminal (imo), I could configure it to use my choice and then Bryan wouldn't have to listen to me whine.

…

On Thu, 4 Jun 2020, 17:55 Jason Kendall, ***@***.***> wrote: @NuSkooler <https://github.com/NuSkooler> This is what I get for reading email and not looking at code (that I haven't looked at in a while). Having said that, as a part time dev, part time pipeline designer, I'd go with the last statement in your last message Perhaps it would be nice if the log allowed backends for particular formats? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#286 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFDGMB7LSSALN7H7FTBQ7LRU7GZJANCNFSM4NSHTTOA> .

[NuSkooler]

It looks like Pino allows for most of what is being asked for via transports/etc. E.g. one could simply plug a transport that outputs to format $foo or sends to $bar, so on.

[vordimous]

https://github.com/trentm/node-bunyan/releases/tag/2.0.4 release is out. should this ticket be closed?