Be able to use loguru as logging/logger

[lucasfcnunes]

Description

https://github.com/Delgan/loguru

loguru is a popular third-party logging framework for Python on GitHub, with about 17.2k stars at the time of writing.
It aims to ease the process of setting up a logging system in your project and provide a simpler alternative to the default Python logging module, which is sometimes criticized for having a convoluted configuration setup.

Use case/motivation

loguru is much easier to set up than the standard logging module, and it has many useful features (e.g. easily outputting every log in JSON) that will help you collect as much information from your application as needed.

Related issues

#8848
#35622
#16579

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[dirrao]

@lucasfcnunes You are welcome to submit the MR.

[potiuk]

Actually I am not sure if we want to do it at all. I don't see who could benefit from that.

Not our users, I am quite sure of that. For users it's better to focus on what has already been approved in https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-49+OpenTelemetry+Support+for+Apache+Airflow - i.e. open-telemetry logging integration. This is waiting for someone to implement it and if someone was to focus on improving our logging, that would be it. What we want to achieve is that none of our users shoud focus on writing their own Handlers, they should simply configure the right OpenTelemetry integration if they want to do something with logs. And if someone would like spend their effort there, that would be way more efficient way of improving lifes of our users.

Contributors and Maintainers? I doubt the effort and cost needed would be justified by the benefits.

I believe that framework could potentially make live a bit easier for maintainers of Airflow, but sinve we already have working implementation of almost-declarative way of configuring logging and handlers, and we've already made a lot of effort into not only making our handlers task-aware and also being able to log things to memory for Triggerer (asyncio) and to merge the logs coming from diferent context from diffferent contexts.

Not mentioning the fact that just basic task handlers of Airflow already support much more than regular handlers - they not only allow to write logs to local or remote storage, but also allow to read the logs written for the webserver to retrieve them via API from workers that produced them so that they can show up in a UI of a webserver running on a completely diffierent machine.

Not mentioning filters (which loguru mentions as not needed) that we are using for secrets masking and our automated way of coloring logs.

Not even mentioning the custom handlers we have that are handling the cases that in POSIX system by default writing to log files by default fills up the filesyste, cache and (even if it is not really causing memory issues, only scares away some people) we have custom handlers that write logs to files while avoiding to fill up the cache.

Not mentioning that we are doing custom redirections of stdin and stderr from subprocesses and redirecting them back to log handler in order to handle the situation when we want to capture all the stdout/stderr printed and prentend it has been logged using context-aware logger.

Simply the scope of "logging" in Airflow and amount of engineering efforts spent in that is so huge, and the amount of code you'd need to rewrite is so big and prone to errors that it's a huge effort to be able to use something like loguru. But if @lucasfcnunes, you feel like creating a pull request for that is a good idea, then you can always try - maybe I am exaggerating the effort needed, after just looking at the demo and docs of loguru, who knows.

Converting it into discussion anyway, because maybe indeed, I am wrong and there are benefits that will outhweigh the cost needed.