Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce our own rich console logging handler #2277

Closed
Tracked by #2205
antonymilne opened this issue Feb 2, 2023 · 0 comments · Fixed by #2592
Closed
Tracked by #2205

Introduce our own rich console logging handler #2277

antonymilne opened this issue Feb 2, 2023 · 0 comments · Fixed by #2592
Assignees

Comments

@antonymilne
Copy link
Contributor

antonymilne commented Feb 2, 2023

Note. Non-breaking, goes in 0.18.x. Easiest to complete after #2206.

We want people to be able to e.g. turn rich tracebacks on/off from logging.yml. However, we don't want to start adding our own custom keys to logging.yml (the format is specified already by Python).

The way to do this is to write our own rich console logging handler. It will be very simple, just a thin wrapper for rich.logging.RichHandler. We need to move all the custom behaviour out of _ProjectLogging and into the init method of the new logging handler. Let's call it kedro.logging.RichHandler (tbc exactly where it should live - we don't have extras/logging in develop so don't just put it there).

Since we're piggybacking off RichHandler's rich_tracebacks argument, which defaults to False, we'll need to change the logging config everywhere (default_logging.yml + all starters) to set this to True:

handlers:
  rich:
    class: kedro.logging.RichHandler
    rich_tracebacks: True

Add docs to describe how you can set rich_tracebacks: False if you'd like to disable them.

The new handler should look something like this:

class RichHandler(rich.logging.RichHandler):
    """Identical to rich's logging handler but with a few extra behaviours:
        * warnings issued by the `warnings` module are redirected to logging
        * pretty printing is enabled on the Python REPL (including IPython and Jupyter)
        * all tracebacks are handled by rich when rich_tracebacks=True
    """
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        logging.captureWarnings(True)
        rich.pretty.install()
        if self.rich_tracebacks:
            # We suppress click here to hide tracebacks related to it conversely,
            # kedro is not suppressed to show its tracebacks for easier debugging.
            # sys.executable is used to get the kedro executable path to hide the
            # top level traceback.
            # Rich traceback handling does not work on databricks. Hopefully this will be
            # fixed on their side at some point, but until then we disable it.
            # See https://github.com/Textualize/rich/issues/2455

            if "DATABRICKS_RUNTIME_VERSION" not in os.environ:
                rich.traceback.install(
                    suppress=[click, str(Path(sys.executable).parent)])
            ###                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            ### NOTE YOU MAY WELL NEED TO CHANGE THIS PATH DEPENDING ON WHERE
            ### THIS FILE ENDS UP SO THAT IT POINTS TO THE RIGHT PLACE

Following #2206, _ProjectLogging would be left looking something like this:

class _ProjectLogging(UserDict):
    # pylint: disable=super-init-not-called
    def __init__(self):
        """Initialise project logging. The path to logging configuration is given in
        environment variable KEDRO_LOGGING_CONFIG (defaults to default_logging.yml)."""
        path = os.environ.get("KEDRO_LOGGING_CONFIG", Path(__file__).parent / "default_logging.yml")
        logging_config = Path(path).read_text(encoding="utf-8")
        self.configure(yaml.safe_load(logging_config))
@antonymilne antonymilne added the Stage: Technical Design 🎨 Ticket needs to undergo technical design before implementation label Feb 6, 2023
@merelcht merelcht removed the Stage: Technical Design 🎨 Ticket needs to undergo technical design before implementation label Mar 1, 2023
@merelcht merelcht moved this to To Do in Kedro Framework Mar 16, 2023
@noklam noklam moved this from To Do to In Progress in Kedro Framework Apr 28, 2023
@noklam noklam self-assigned this Apr 28, 2023
@merelcht merelcht moved this from In Progress to To Do in Kedro Framework May 2, 2023
@merelcht merelcht removed the status in Kedro Framework May 2, 2023
@noklam noklam moved this to To Do in Kedro Framework May 15, 2023
@noklam noklam moved this from To Do to In Progress in Kedro Framework May 18, 2023
@noklam noklam self-assigned this May 18, 2023
@noklam noklam linked a pull request May 19, 2023 that will close this issue
11 tasks
@noklam noklam mentioned this issue May 19, 2023
11 tasks
@NeroOkwa NeroOkwa moved this from In Progress to In Review in Kedro Framework May 24, 2023
@github-project-automation github-project-automation bot moved this from In Review to Done in Kedro Framework May 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

3 participants