Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add ConsoleFormatter #3

Merged
merged 1 commit into from
Aug 21, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 33 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@
![pypi](https://img.shields.io/pypi/v/loggingx-py)
![language](https://img.shields.io/github/languages/top/hhk7734/loggingx.py)

## loggingx.py
# loggingx.py

`loggingx` is a drop-in replacement for Python's built-in `logging` module. Even better, once you've imported `loggingx`, you don't need to modify your existing `logging` module.

```shell
python3 -m pip install loggingx-py
```

### Additional Format
## Additional Format

- https://docs.python.org/3/library/logging.html#logrecord-attributes

Expand All @@ -19,7 +19,7 @@ python3 -m pip install loggingx-py
| caller | %(caller)s | Caller(`<pathname>:<lineno>`) |
| ctxFields | %(ctxFields)s | Context fields |

### Optimization
## Optimization

| Configuration | Description |
| ---------------------------- | -------------------------------------------------------------- |
Expand All @@ -28,15 +28,14 @@ python3 -m pip install loggingx-py
| `logging.logMultiprocessing` | If `False`, Record will not collect `processName`. |


### Context
## Context

```python
import loggingx as logging

logging.basicConfig(
level=logging.INFO,
format="%(asctime)s\t%(levelname)s\t%(caller)s\t%(message)s\t%(ctxFields)s",
)
handler = logging.StreamHandler()
handler.setFormatter(logging.ConsoleFormatter())
logging.basicConfig(level=logging.INFO, handlers=[handler])


def A() -> None:
Expand All @@ -60,11 +59,13 @@ if __name__ == "__main__":
```

```shell
2023-07-19 01:15:33,981 INFO loggingx.py/main.py:10 A {}
2023-07-19 01:15:33,981 INFO loggingx.py/main.py:16 B {'A': 'a'}
2023-07-19 01:15:33,982 INFO loggingx.py/main.py:22 C {'A': 'a', 'B': 'b'}
2024-08-22T02:46:38.257+09:00 INFO main.py:9 A {}
2024-08-22T02:46:38.257+09:00 INFO main.py:15 B {"A": "a"}
2024-08-22T02:46:38.258+09:00 INFO main.py:21 C {"A": "a", "B": "b"}
```

## Formatter

### JSONFormatter

```python
Expand All @@ -84,15 +85,33 @@ if __name__ == "__main__":
{
"time": 1689697694.9980711,
"level": "info",
"caller": "loggingx.py/main.py:10",
"caller": "main.py:10",
"msg": "test",
"ctx": "ctx",
"thread_name": "MainThread",
"extra": "extra"
}
```

### With `logging`
### ConsoleFormatter

```python
import loggingx as logging

handler = logging.StreamHandler()
handler.setFormatter(logging.ConsoleFormatter())
logging.basicConfig(level=logging.INFO, handlers=[handler])

if __name__ == "__main__":
with logging.addFields(ctx="ctx"):
logging.info("test", extra={"extra": "extra"})
```

```shell
2024-08-22T02:48:17.868+09:00 INFO main.py:9 test {"ctx": "ctx", "extra": "extra"}
```

## With `logging`

```python
import logging
Expand All @@ -112,7 +131,7 @@ if __name__ == "__main__":
logging.info("test", extra={"extra": "extra"})
```

### jq
## jq

```shell
alias log2jq="jq -rRC --unbuffered '. as \$line | try fromjson catch \$line' | sed 's/\\\\n/\\n/g; s/\\\\t/\\t/g'"
Expand Down
4 changes: 2 additions & 2 deletions loggingx/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

from .context import CtxRecord as _CtxRecord
from .context import addFields
from .formatter import Information, JSONFormatter
from .formatter import ConsoleFormatter, Information, JSONFormatter

try:
__version__ = version("loggingx-py")
Expand All @@ -13,7 +13,7 @@
pass

__all__ = _logging.__all__.copy()
__all__ += ["addFields", "Information", "JSONFormatter"]
__all__ += ["addFields", "Information", "JSONFormatter", "ConsoleFormatter"]


setLogRecordFactory(_CtxRecord) # noqa: F405
77 changes: 76 additions & 1 deletion loggingx/formatter.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
import json
import logging
import re
from datetime import datetime, timezone
from enum import Enum
from logging import Formatter
from typing import List, Union

from loggingx.context import CtxRecord
from .context import CtxRecord

# https://docs.python.org/3/library/logging.html#logrecord-attributes
_DEFAULT_KEYS = (
Expand Down Expand Up @@ -40,6 +41,8 @@
logging.NOTSET: "notset",
}

_LEVEL_TO_UPPER_NAME = {k: v.upper() for k, v in _LEVEL_TO_LOWER_NAME.items()}


class Information(str, Enum):
NAME = "name"
Expand Down Expand Up @@ -99,3 +102,75 @@ def format(self, record: CtxRecord) -> str: # type: ignore[override]

# Set ensure_ascii to False to output the message as it is typed.
return json.dumps(msg_dict, ensure_ascii=False)


_RESET = "\033[0m"
_RED_BG = "\033[30;41m"
_GREEN_BG = "\033[30;42m"
_YELLOW_BG = "\033[30;43m"


class ConsoleFormatter(Formatter):
def __init__(
self,
additional_infos: Union[Information, List[Information], None] = None,
color: bool = True,
) -> None:
super().__init__()
if additional_infos is None:
additional_infos = []
elif isinstance(additional_infos, Information):
additional_infos = [additional_infos]

self._excludes = {x.value for x in Information if x not in additional_infos}
self._key_map = {x.value: re.sub(r"(?<!^)(?=[A-Z])", "_", x.value).lower() for x in additional_infos}

self._color = color

def format(self, record: CtxRecord) -> str: # type: ignore[override]
rfc3339 = datetime.now(timezone.utc).astimezone().isoformat(sep="T", timespec="milliseconds")

extra = {}
for k, v in record.ctxFields.items():
extra[k] = v

# extra
for k, v in record.__dict__.items():
if k in _DEFAULT_KEYS:
continue
if k.startswith("_"):
continue
if k in self._excludes:
continue
if k in self._key_map:
k = self._key_map[k]
extra[k] = v

color = ""
reset = ""
if self._color:
if record.levelno >= logging.ERROR:
color = _RED_BG
elif record.levelno >= logging.WARNING:
color = _YELLOW_BG
elif record.levelno >= logging.INFO:
color = _GREEN_BG

if record.levelno >= logging.INFO:
reset = _RESET

# Set ensure_ascii to False to output the message as it is typed.
msg = f"{rfc3339} {color}{_LEVEL_TO_UPPER_NAME[record.levelno]:<6}{reset} {record.caller}\t{record.getMessage()} {json.dumps(extra, ensure_ascii=False)}"

if record.exc_info:
# Cache the traceback text to avoid converting it multiple times
# (it's constant anyway)
if not record.exc_text:
record.exc_text = self.formatException(record.exc_info)
if record.exc_text:
msg += f"\n{record.exc_text}"

if record.stack_info:
msg += f"\n{self.formatStack(record.stack_info)}"

return msg