Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the Kedro RichHandler #2592

Merged
merged 45 commits into from
May 30, 2023

Conversation

noklam
Copy link
Contributor

@noklam noklam commented May 19, 2023

NOTE: Kedro datasets are moving from kedro.extras.datasets to a separate kedro-datasets package in
kedro-plugins repository. Any changes to the dataset implementations
should be done by opening a pull request in that repository.

Description

Fix #2277

It should also open up the solutions for these logging issues

This will enable users to customise rich logging and optionally disable the rich's traceback if needed (Some platform doesn't play very well with it, e.g. Databricks and we have some logic to disable it by default)

Bonus: It now also supports more arguments that is compatible with RichHandler and rich.traceback.install

Questions for Reviewer

  • Where should we put the logger? We have kedro.extras.logging but we plan to remove this namespace, it is already removed in dev. Do we need to put it in kedro.extras.logging and move it to kedro.logging in 0.19.0 or just do it now since it's not breaking?
  • If some error happen before the log was set up it won't be rich-formatted properly, i.e. invalid RichHandler. I think it's fine but worth to mention.
  • I did some extra work to map arguments between RichHandler to rich.traceback.install to maximize the flexibility (Tests are not written yet, but if we agree this is useful then I will go on to write them). For example tracebacks_show_locals is mapped as show_locals for rich.pretty.install

Note: I haven't created unit tests since there are many decisions about what should be enable/disable, I will add the test after the 1st round of review.

Development notes

  • Add some unit tests (test rich is enable by default, test the option to disable rich, and test all known available arguments of RichHandler)
  • Check if we can simplify the test, we manually delete sys.modules for testing _ProjectLogging, it may be easier now because this is configurable we don't have to reload module.
  • Add changes to all logging.yml (template)
  • Add changes to all starters logging.yml (starters)
  • Add documentation about the changes
  • Implementation of RichLogger

Checklist

Manual Testing

  1. Run kedro new -s pandas-iris
  2. Edit conf/base/logging.yml
  rich:
    class: kedro.extras.logging.RichHandler
    # class: rich.logging.RichHandler
    rich_tracebacks: True
   # tracebacks_show_locals: True
  1. kedro run
  • Read the contributing guidelines
  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added a description of this change in the RELEASE.md file
  • Added tests to cover my changes

Note:

Currently user need to set KEDRO_LOGGING_CONFIG to disable rich's traceback, because the default logging installed it and there is no way to uninstall it.

We need to solve this by either implementing it ourselves or raise a PR to rich

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
@noklam noklam linked an issue May 19, 2023 that may be closed by this pull request
noklam added 8 commits May 19, 2023 17:44
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
…RichHandler

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
class: rich.logging.RichHandler
class: kedro.extras.logging.RichHandler
rich_tracebacks: True
# tracebacks_show_locals: False
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like settings.py, we could have some commented default here to help user discover possible options. I put show_locals because initially when we introduce rich, we want to enable it as True. It turns out to be a bit noisy for some users, and there was no good way to turn it off by the user.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this idea!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also like the idea, but remember this file won't exist in the default template in 0.19 and it will move to docs instead - see #2426 (this was a relatively recent decision in tech design: #2281 (comment)).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also note you changed the test_starter here, not the actual kedro template starter.

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Comment on lines 37 to 48
# Mapping Arguments from RichHandler's Constructor to rich.traceback.install
for key, value in kwargs.items():
prefix = "tracebacks_"
if key.startswith(prefix):
key_prefix_removed = key[len(prefix) :]
if key_prefix_removed == "suppress":
mapped_kwargs[key_prefix_removed] += value
else:
mapped_kwargs[key_prefix_removed] = value
elif key in ("locals_max_length", "locals_max_string"):
mapped_kwargs[key] = value

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original issue propose to support self.rich_tracebacks only, I take it further because it is possible to support most of the arguments as long as they both exists in RichHandler and rich.traceback.install, the signature are slightly different so we need to map it.

Comment on lines 49 to 53
if self.rich_tracebacks:
if "DATABRICKS_RUNTIME_VERSION" not in os.environ:
# https://rich.readthedocs.io/en/stable/reference/logging.html?highlight=rich%20handler#rich.logging.RichHandler
# Support compatible arguments between RichHandler and rich.traceback.install
rich.traceback.install(**mapped_kwargs)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, we can also remove the special logic about Databricks but just change the default to False in the starter because it is now customisable.

I didn't do it because it's still a breaking change, and there's no point to use Rich's traceback on Databricks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather keep it as you have it, so the default is True and we include the special logic that disables it on Databricks.

Comment on lines -224 to -235
logging.captureWarnings(True)

# We suppress click here to hide tracebacks related to it conversely,
# kedro is not suppressed to show its tracebacks for easier debugging.
# sys.executable is used to get the kedro executable path to hide the
# top level traceback.
# Rich traceback handling does not work on databricks. Hopefully this will be
# fixed on their side at some point, but until then we disable it.
# See https://github.com/Textualize/rich/issues/2455
if "DATABRICKS_RUNTIME_VERSION" not in os.environ:
rich.traceback.install(suppress=[click, str(Path(sys.executable).parent)])
rich.pretty.install()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rich's traceback won't be installed when kedro.framework.project is imported, it's now just a logging handler. This is an improvement because importing a module no longer has side-effect.

noklam added 2 commits May 23, 2023 17:02
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Copy link
Member

@merelcht merelcht left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To answer your questions:

Where should we put the logger? We have kedro.extras.logging but we plan to remove this namespace, it is already removed in dev. Do we need to put it in kedro.extras.logging and move it to kedro.logging in 0.19.0 or just do it now since it's not breaking?

I would just put it in kedro.logging straightaway, so we don't need to move it again for the 0.19.0 release.

If some error happen before the log was set up it won't be rich-formatted properly, i.e. invalid RichHandler. I think it's fine but worth to mention.

That makes sense to me and I don't think there's a way around that.

I did some extra work to map arguments between RichHandler to rich.traceback.install to maximize the flexibility (Tests are not written yet, but if we agree this is useful then I will go on to write them). For example tracebacks_show_locals is mapped as show_locals for rich.pretty.install

I don't know that much about Rich so I'm not entirely sure yet what this means, but it sounds good 😄 But it would be good to get @antonymilne 's view on this.

class: rich.logging.RichHandler
class: kedro.extras.logging.RichHandler
rich_tracebacks: True
# tracebacks_show_locals: False
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this idea!

@antonymilne
Copy link
Contributor

Let me start by answering the questions and will then review the code.

  • Where should we put the logger? We have kedro.extras.logging but we plan to remove this namespace, it is already removed in dev. Do we need to put it in kedro.extras.logging and move it to kedro.logging in 0.19.0 or just do it now since it's not breaking?

Agree with @merelcht that we should just put it in kedro.logging to begin with. Note there's no need to create a subpackage for it - just a single file logging.py should be good.

  • If some error happen before the log was set up it won't be rich-formatted properly, i.e. invalid RichHandler. I think it's fine but worth to mention.

This is unavoidable really and should be fine. Users will be able to fix it using the KEDRO_LOGGING_CONFIG environment variable to point to a different configuration anyway.

  • I did some extra work to map arguments between RichHandler to rich.traceback.install to maximize the flexibility (Tests are not written yet, but if we agree this is useful then I will go on to write them). For example tracebacks_show_locals is mapped as show_locals for rich.pretty.install

I think this is a good idea 👍

  • Add some unit tests (test rich is enable by default, test the option to disable rich, and test all known available arguments of RichHandler)

Probably you don't need to test all the known available arguments - just one of them should be fine.

Copy link
Contributor

@antonymilne antonymilne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good in general! I will try it out but basically I think you're on the right track here and should go ahead and write tests. The only big thing is to put the code in kedro/logging.py instead of in extras.

Edit: tried it out and all seems to work as expected 👍 Note you need to set KEDRO_LOGGING_CONFIG=conf/base/logging.yml to really test this properly at the moment. Again, things will be waaaaay simpler soon when we've done #2426.

class: rich.logging.RichHandler
class: kedro.extras.logging.RichHandler
rich_tracebacks: True
# tracebacks_show_locals: False
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also like the idea, but remember this file won't exist in the default template in 0.19 and it will move to docs instead - see #2426 (this was a relatively recent decision in tech design: #2281 (comment)).

class: rich.logging.RichHandler
class: kedro.extras.logging.RichHandler
rich_tracebacks: True
# tracebacks_show_locals: False
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also note you changed the test_starter here, not the actual kedro template starter.

"""Identical to rich's logging handler but with a few extra behaviours:
* warnings issued by the `warnings` module are redirected to logging
* pretty printing is enabled on the Python REPL (including IPython and Jupyter)
* all tracebacks are handled by rich when rich_tracebacks=True
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should have anoter point here to describe the extra rich.install behaviour you added.

Comment on lines 49 to 53
if self.rich_tracebacks:
if "DATABRICKS_RUNTIME_VERSION" not in os.environ:
# https://rich.readthedocs.io/en/stable/reference/logging.html?highlight=rich%20handler#rich.logging.RichHandler
# Support compatible arguments between RichHandler and rich.traceback.install
rich.traceback.install(**mapped_kwargs)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather keep it as you have it, so the default is True and we include the special logic that disables it on Databricks.

kedro/extras/logging/rich_logger.py Outdated Show resolved Hide resolved
Comment on lines 31 to 33
# Rich traceback handling does not work on databricks. Hopefully this will be
# fixed on their side at some point, but until then we disable it.
# See https://github.com/Textualize/rich/issues/2455
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment should be moved down to where it belongs.

kedro/extras/logging/rich_logger.py Outdated Show resolved Hide resolved
mapped_kwargs[key_prefix_removed] += value
else:
mapped_kwargs[key_prefix_removed] = value
elif key in ("locals_max_length", "locals_max_string"):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the set of arguments that are the same for rich.logging.RichHandler and rich.tracebacks.install? Might be worth a comment explaining why they're here,

kedro/extras/logging/rich_logger.py Outdated Show resolved Hide resolved
kedro/extras/logging/rich_logger.py Outdated Show resolved Hide resolved

Kedro's `kedro.extras.logging.RichHandler` is a subclass of [`rich.logging.RichHandler`](https://rich.readthedocs.io/en/stable/reference/logging.html#rich.logging.RichHandler) and supports the same set of arguments. By default, `rich_tracebacks` is set to `True` to use `rich` to render exceptions. However, you can disable it by setting `rich_tracebacks: False`.

When `rich_tracebacks` is set to `True`, the configuration is propagated to [`rich.traceback.install`](https://rich.readthedocs.io/en/stable/reference/traceback.html#rich.traceback.install). If an argument is compatible with `rich.pretty.install`, it will be passed to the traceback's settings.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
When `rich_tracebacks` is set to `True`, the configuration is propagated to [`rich.traceback.install`](https://rich.readthedocs.io/en/stable/reference/traceback.html#rich.traceback.install). If an argument is compatible with `rich.pretty.install`, it will be passed to the traceback's settings.
When `rich_tracebacks` is set to `True`, the configuration is propagated to [`rich.traceback.install`](https://rich.readthedocs.io/en/stable/reference/traceback.html#rich.traceback.install). If an argument is compatible with `rich.traceback.install`, it will be passed to the traceback's settings.

Unless I misunderstood something?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad! I notice this yesterday but the changes are still in my local branch, good catch!

noklam and others added 4 commits May 24, 2023 11:43
Co-authored-by: Antony Milne <49395058+antonymilne@users.noreply.github.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
noklam added 8 commits May 25, 2023 13:29
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
noklam added 4 commits May 25, 2023 17:54
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
noklam added 4 commits May 30, 2023 12:40
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
.. autoclass:: RichHandler
:members:
:undoc-members:
:inherited-members:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we'll have to remove the :undoc-members: and :inherited-members:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! This seems to work. Sphinx is a puzzle, I cannot reproduce the same error in my local environment, but I am glad it's fixed now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't believe it worked 😂 I guess I'm a Sphinx black belt now 🥷

noklam added 2 commits May 30, 2023 15:19
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
@noklam noklam enabled auto-merge (squash) May 30, 2023 14:58
@noklam noklam disabled auto-merge May 30, 2023 14:58
@noklam noklam enabled auto-merge (squash) May 30, 2023 14:59
@noklam noklam merged commit 4a4bd73 into main May 30, 2023
@noklam noklam deleted the 2277-introduce-our-own-rich-console-logging-handler branch May 30, 2023 15:12
noklam added a commit that referenced this pull request Jun 7, 2023
* Remove unnecessary files and subheaders in documentation (#2563)

Tidying up.

* Leverage PEP-585 (#2540)

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>

* [READY] Bring deployment docs up-to-date and add new pages for additional targets (#2557)

Ready at last. Still some conversation to be had about how we reflect "confidence" in the plugins we describe (what version they were last tested against) but the text is ready and better than what's there currently, so let's go!

* Remove mentions to `.egg` and stop producing them from `kedro package` (#2568)

* Stop producing `.egg` files from `kedro package`

Fix gh-2273.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Upgrade import-linter

Fix gh-2570.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Partially remove reliance on setup.py for micropackaging

Incomplete effort, see gh-2414.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Revert "Partially remove reliance on setup.py for micropackaging"

This reverts commit fd3efa8.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* OmegaConfigLoader returns `Dict` instead of `DictConfig`, resolves runtime_params properly (#2467)

* Fix typehint

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* test push

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* POC of fix to solve the runtime param resolution problem

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix OmegaConfigLoadaer - resolve runtime_params early

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Delegate the intialization of runtime_params to AbstractConfigLoader

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Add test for interpolated value and globals

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* add more test and linting

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* refactor

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* update release note

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Apply comments and refactor the test

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Update RELEASE.md

Co-authored-by: Antony Milne <49395058+antonymilne@users.noreply.github.com>

---------

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Co-authored-by: Antony Milne <49395058+antonymilne@users.noreply.github.com>

* Revert special logic handling DictConfig (#2582)

* Partially revert #2378 as OCL no longer returning DictConfig

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Revert #2176 with special logging handling

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* remove unused import

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

---------

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Re-title and modify introduction to databricks testing guide in contribution section (#2579)

* modify databricks testing guide for contributors

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/contribution/development_for_databricks.md

Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>

* Update development_for_databricks.md

---------

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>

* Quick fix .gitpod.yml (#2585)

* `OmegaConfigLoader` should not show `MissingConfigException` when the file is empty (#2584)

* Allow empty config files

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Remove condition for empty config

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Revert "Remove condition for empty config"

This reverts commit 7961155.

* Update release notes

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add unit test

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

---------

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Proposal of updated settings.py (#2587)

* Proposal of updated settings.py

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Update settings after comment

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Collaborative Experiment Tracking Docs  (#2589)

* Add metadata attribute to `kedro.io` datasets (#2537)

* Add metadata attribute to kedro.io datasets

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Update specs in pipeline hooks docstrings (#2598)

* Update pipeline hooks docstrings

Signed-off-by: Tomas Van Pottelbergh <github@tomasvanpottelbergh.com>

* Acknowledge community contribution

---------

Signed-off-by: Tomas Van Pottelbergh <github@tomasvanpottelbergh.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Miscellaneous hooks improvements (#2596)

* Add debug level logging for hook execution order

Fix gh-1942.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Clarify role of setuptools in hooks

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Remove mentions to outdated and unused `pkg_resources`

Fix gh-2391.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Adapt example hook to spec signature

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Grammar and style fixes in docs

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Revert "Adapt example hook to spec signature"

This reverts commit ac1b495.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Minor modification to titles of deployment docs to replace "deprecated" with "legacy" (#2606)

* FIX: Typo in markdown for kedro notebooks docs (#2610)

* FIX: Typo in markdown for kedro notebooks docs

Signed-off-by: debugger24 <rahulcomp24@gmail.com>

* Acknowledge community contribution

---------

Signed-off-by: debugger24 <rahulcomp24@gmail.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Add the Kedro RichHandler (#2592)

* Add the Kedro RichHandler

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Clean up imports and comments

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Map arguments to add flexibility to customize arguments available in RichHandler

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Clean up rich_logger.py and unused imports

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Add docs - first iteration

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix imports and small refactoring

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Improve the doc

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Update the project's template

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* format docstring

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Apply Jo's comment on the doc

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix the link with RichHandler

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Apply suggestions from code review

Co-authored-by: Antony Milne <49395058+antonymilne@users.noreply.github.com>

* Fix broken doc

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* update template defaults

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* refactoring RichHandler and improve doc

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix broken test due to new default RichHandler

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix doc

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Move kedro.extras.logging.RichHandler -> kedro.logging.RichHandler

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* add kedro/logging.py and remove the unnecessary import that cause namespace collision

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Refactor test

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Add more tests

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix lots of import, docstring

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Reorder docstring

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix tests

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Link options to doc

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Clean up doc and unused logger

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix spelling `customise`

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Refactor test with fixture

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Remove unnecessary copy() because it is now a fixture

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Split up test_rich_traceback_configuration() to smaller tests

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Add RELEASE.md

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Update doc

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix template logging

* Mention awesome-kedro in docs (#2620)

* Mention awesome-kedro in docs

See kedro-org/kedro-devrel#40

* Update CONTRIBUTING.md

* Spelling and style fixes

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

---------

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update SnowparkTableDataSet docs (#2485)

* Update SnowparkTableDataSet docs

* Add missing snowpark dependencies for docs building

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Release 0.18.9 (#2619)

* Update __init__.py

* Update release note

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Update Version Numbers across docs

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Update RELEASE.md

* Update Release Note

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

---------

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Add documentation for deploying packaged Kedro projects on Databricks (#2595)

* Add deployment workflow page

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add table of contents, entry point guide, data and conf upload guide

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add detailed instructions for creating a job on Databricks

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add images and automated deployment resources

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Remove use of 'allows', add summary

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Remove link to missing image

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add deployment workflow to toctree

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Lint and fix missing link

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Minor style, syntax and grammar improvements

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Fixes for correctness during validation

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add instructions for creating log output location

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Lint

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Lint databricks_run

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Minor wording change in reference to logs

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Modify reference to Pyspark-Iris

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Fix linter errors to enable docs build for inspection

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update build-docs.sh

* Fix broken link

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Remove spurious word

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Add advantages subheading

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update docs/source/integrations/databricks_deployment_workflow.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Add alternative ways to upload data to DBFS

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Move note on unpackaged config and data

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Fix broken links

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Move databricks back into deployment section

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Remove references to PySpark Iris (pyspark-iris) starter

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Graphics links fixes, revise titles

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Fix broken internal link

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Fix links broken by new folder

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Remove logs directory

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Update image of final job configuration

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Add full stops in list.

Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Fix conda environment name.

Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Modify wording and image for creating a new job cluster

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Modify wording in guide to create new job cluster

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Remove --upgrade option

Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Add both ways of creating a new job

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

---------

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>
Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Update docs font family to Inter and swap the logo (#2611)

* Update docs font family to Inter

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Match weights to previous Google font

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Update logo images; revert css formatting for easier PR review

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Update text styles

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Add trailing whitespace (?)

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Update Kedro image path

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Update font size

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* update font to font-family: 'Inter', sans-serif;

Signed-off-by: huongg <huongg1409@gmail.com>

* testing with important for font-family

Signed-off-by: huongg <huongg1409@gmail.com>

* Make project name lowercase in docs

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* revert back to kedro/develop

Signed-off-by: huongg <huongg1409@gmail.com>

---------

Signed-off-by: Tynan DeBold <thdebold@gmail.com>
Signed-off-by: huongg <huongg1409@gmail.com>
Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Co-authored-by: Huong Nguyen <32060364+Huongg@users.noreply.github.com>
Co-authored-by: huongg <huongg1409@gmail.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Enable variable interpolation for the catalog with `OmegaConfigLoader` (#2621)

* Enable variable interpolation in catalog by escaping _

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>

* Add archive link on docs home page (#2636)

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* add ravi-kumar-pilla to tsc (#2629)

Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Add file for search console verification (#2631)

* add verification file and change to conf

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Fix conf.py so google file is moved to root of docs build

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

---------

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Chore/update assets for docs (#2625)

* Update docs font family to Inter

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Match weights to previous Google font

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Update logo images; revert css formatting for easier PR review

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Update text styles

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Add trailing whitespace (?)

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Update Kedro image path

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Update font size

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* update font to font-family: 'Inter', sans-serif;

Signed-off-by: huongg <huongg1409@gmail.com>

* testing with important for font-family

Signed-off-by: huongg <huongg1409@gmail.com>

* update assets for experiment tracking, docs, pipeline, focus mode, matplotlib and plotly

* update modular pipelines section

Signed-off-by: huongg <huongg1409@gmail.com>

* autoreload

Signed-off-by: huongg <huongg1409@gmail.com>

* reupload autoreload gif

Signed-off-by: huongg <huongg1409@gmail.com>

* revert back to develop branch

Signed-off-by: huongg <huongg1409@gmail.com>

* gif for ET comparison

Signed-off-by: huongg <huongg1409@gmail.com>

* remove jupyter notebook screenshot since its not used anywhere

Signed-off-by: huongg <huongg1409@gmail.com>

* Update databricks viz demo

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* pipeline_show_metrics gif

Signed-off-by: huongg <huongg1409@gmail.com>

* update different gif format

Signed-off-by: huongg <huongg1409@gmail.com>

* point to main instead of develop for kedro-banner

Signed-off-by: huongg <huongg1409@gmail.com>

---------

Signed-off-by: Tynan DeBold <thdebold@gmail.com>
Signed-off-by: huongg <huongg1409@gmail.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>
Co-authored-by: Tynan DeBold <thdebold@gmail.com>
Co-authored-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Fix the level heading for collab exp tracking so it shows in sidebar (#2647)

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update Kedro icons in docs with for rebrand (#2652)

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Modify logos (#2651)

Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>

* Fix lint

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>

* remove the subsection of kedro.extras.extension

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Fix lint

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* remove the subsection of kedro.extras.extension

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

---------

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>
Signed-off-by: Tomas Van Pottelbergh <github@tomasvanpottelbergh.com>
Signed-off-by: debugger24 <rahulcomp24@gmail.com>
Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com>
Signed-off-by: Tynan DeBold <thdebold@gmail.com>
Signed-off-by: huongg <huongg1409@gmail.com>
Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>
Co-authored-by: Antony Milne <49395058+antonymilne@users.noreply.github.com>
Co-authored-by: Ankita Katiyar <110245118+ankatiyar@users.noreply.github.com>
Co-authored-by: rashidakanchwala <37628668+rashidakanchwala@users.noreply.github.com>
Co-authored-by: Ahdra Merali <90615669+AhdraMeraliQB@users.noreply.github.com>
Co-authored-by: Tomas Van Pottelbergh <85937396+tomasvanpottelbergh@users.noreply.github.com>
Co-authored-by: Rahul Kumar <rahulcomp24@gmail.com>
Co-authored-by: Tynan DeBold <thdebold@gmail.com>
Co-authored-by: Huong Nguyen <32060364+Huongg@users.noreply.github.com>
Co-authored-by: huongg <huongg1409@gmail.com>
Co-authored-by: Ravi Kumar Pilla <ravi_kumar_pilla@mckinsey.com>
Co-authored-by: Jannic Holzer <jannic.holzer@quantumblack.com>
Co-authored-by: Merel Theisen <merel.theisen@quantumblack.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Introduce our own rich console logging handler
5 participants