Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Miscellaneous hooks improvements #2596

Merged
merged 6 commits into from
May 22, 2023
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/source/extend_kedro/plugins.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Kedro plugins allow you to create new features for Kedro and inject additional c

## Overview

Kedro uses [`setuptools`](https://setuptools.readthedocs.io/en/latest/setuptools.html), which is a collection of enhancements to the Python `distutils` to allow developers to build and distribute Python packages. Kedro uses various entry points in [`pkg_resources`](https://setuptools.readthedocs.io/en/latest/setuptools.html) to provide plugin functionality.
Kedro's extension mechanism is built on [`pluggy`](https://pluggy.readthedocs.io/), a solid plugin management library that was created for the [pytest](https://docs.pytest.org/) ecosystem. `pluggy` relies on [entry points](https://packaging.python.org/en/latest/specifications/entry-points/), a Python mechanism for packages to provide components that can be discovered by other packages using [`importlib.metadata`](https://docs.python.org/3/library/importlib.metadata.html#entry-points).

## Example of a simple plugin

Expand Down Expand Up @@ -189,7 +189,7 @@ When you are ready to submit your code:
2. Choose a command approach: `global` and / or `project` commands:
- All `global` commands should be provided as a single `click` group
- All `project` commands should be provided as another `click` group
- The `click` groups are declared through the [`pkg_resources` entry_point system](https://setuptools.readthedocs.io/en/latest/setuptools.html)
- The `click` groups are declared through the [entry points mechanism](https://setuptools.pypa.io/en/latest/userguide/entry_point.html)
3. Include a `README.md` describing your plugin's functionality and all dependencies that should be included
4. Use GitHub tagging to tag your plugin as a `kedro-plugin` so that we can find it

Expand Down
2 changes: 1 addition & 1 deletion docs/source/hooks/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ class DataCatalogHooks:
return logging.getLogger(self.__class__.__name__)

@hook_impl
def after_catalog_created(self, catalog: DataCatalog) -> None:
def after_catalog_created(self, catalog: DataCatalog, **kwargs) -> None:
astrojuanlu marked this conversation as resolved.
Show resolved Hide resolved
self._logger.info(catalog.list())
```

Expand Down
6 changes: 5 additions & 1 deletion kedro/framework/cli/hooks/manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@
from .markers import CLI_HOOK_NAMESPACE
from .specs import CLICommandSpecs

logger = logging.getLogger(__name__)

_cli_hook_manager = None

_CLI_PLUGIN_HOOKS = "kedro.cli_hooks"
Expand All @@ -17,6 +19,8 @@ def get_cli_hook_manager():
global _cli_hook_manager
if _cli_hook_manager is None:
_cli_hook_manager = CLIHooksManager()
_cli_hook_manager.trace.root.setwriter(logger.debug)
_cli_hook_manager.enable_tracing()
return _cli_hook_manager
astrojuanlu marked this conversation as resolved.
Show resolved Hide resolved


Expand All @@ -42,7 +46,7 @@ def _register_cli_hooks_setuptools(self) -> None:
}

if plugin_names:
logging.getLogger(__name__).debug(
logger.debug(
"Registered CLI hooks from %d installed plugin(s): %s",
len(plugin_names),
", ".join(sorted(plugin_names)),
Expand Down
7 changes: 6 additions & 1 deletion kedro/framework/hooks/manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@
def _create_hook_manager() -> PluginManager:
"""Create a new PluginManager instance and register Kedro's hook specs."""
manager = PluginManager(HOOK_NAMESPACE)
manager.trace.root.setwriter(logger.debug)
manager.enable_tracing()
manager.add_hookspecs(NodeSpecs)
manager.add_hookspecs(PipelineSpecs)
manager.add_hookspecs(DataCatalogSpecs)
Expand Down Expand Up @@ -60,10 +62,13 @@ def _register_hooks_setuptools(

"""
already_registered = hook_manager.get_plugins()
# Method name is misleading:
# entry points are standard and don't require setuptools,
# see https://packaging.python.org/en/latest/specifications/entry-points/
hook_manager.load_setuptools_entrypoints(_PLUGIN_HOOKS)
disabled_plugins = set(disabled_plugins)

# Get list of plugin/distinfo tuples for all setuptools registered plugins.
# Get list of plugin/distinfo tuples for all registered plugins.
plugininfo = hook_manager.list_plugin_distinfo()
plugin_names = set()
disabled_plugin_names = set()
Expand Down