Skip to content

Commit

Permalink
feat: Add Hooks Extension Framework tooling
Browse files Browse the repository at this point in the history
* Pipeline runner for actions and filters
* Triggers for actions and filters
  • Loading branch information
mariajgrimaldi committed Mar 26, 2021
1 parent a8cc383 commit b50a5f0
Show file tree
Hide file tree
Showing 11 changed files with 1,107 additions and 0 deletions.
108 changes: 108 additions & 0 deletions docs/decisions/0005-hooks-config-location.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
Configuration for the Hooks Extension Framework
===============================================

Status
------

Draft

Context
-------

Context taken from the Discuss thread `Configuration for the Hooks Extension Framework <https://discuss.openedx.org/t/configuration-for-the-hooks-extension-framework/4527>`_

We need a way to configure a list of functions (actions or filters) that will be called at different places (triggers) in the code of edx-platform.

So, for a string like:

"openedx.lms.auth.post_login.action.v1"

We need to define a list of functions:

.. code-block:: python
[
"from_a_plugin.actions.action_1",
"from_a_plugin.actions.action_n",
"from_some_other_package.actions.action_1",
# ... and so.
]
And also some extra variables:

.. code-block:: python
{
"async": True, # ... and so.
}
We have considered two alternatives:

* A dict in the Django settings.
* Advantages:
* It is very standard, everyone should know how to change it by now.
* Can be altered without installing plugins.
* Disadvantages:
* It is hard to document a large dict.
* Could grow into something difficult to manage.

* In a view of the AppConfig of your plugin.
* Advantages:
* Each plugin can extend the config to add its own actions and filters without collisions.
* Disadvantages:
* It’s not possible to control the ordering of different actions being connected to the same trigger by different plugins.
* For updates, an operator must install a new version of the dependency which usually is longer and more difficult than changing vars and restart.
* Not easy to configure by tenant if you use site configs.
* Requires a plugin.

Decision
--------

We decided to use a dictionary with a flexible format defined using Django settings.

Consequences
------------

1. The only way to configure Hooks Extension Framework is via Django settings using one of these three formats:

**Option 1**: this is the more detailed option and from it, the others can be derived. Through this configuration can be configured the list of functions to be executed and how
to execute them.

.. code-block:: python
HOOKS_EXTENSION_CONFIG = {
"openedx.service.trigger_context.location.trigger_type.vi": {
"pipeline": [
"from_a_plugin.actions.action_1",
"from_a_plugin.actions.action_n",
"from_some_other_package.actions.action_1",
],
"async": False,
}
}
**Option 2**: this option only considers the configuration of the list of functions to be executed and how to execute them is set to the default.

.. code-block:: python
HOOKS_EXTENSION_CONFIG = {
"openedx.service.trigger_context.location.trigger_type.vi": {
[
"from_a_plugin.actions.action_1",
"from_a_plugin.actions.action_n",
"from_some_other_package.actions.action_1",
],
}
}
**Option 3**: this option considers that there's just one function to be executed. As above, how to execute it is set to the default.

.. code-block:: python
HOOKS_EXTENSION_CONFIG = {
"openedx.service.trigger_context.location.trigger_type.vi": {
"from_a_plugin.actions.action_1",
}
}
2. Given that Site Configurations is not available in this repository, it can't be used to configure hooks.
44 changes: 44 additions & 0 deletions docs/decisions/0006-hooks-tooling-pipeline.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
Hooks tooling: pipeline
=======================

Status
------

Draft

Context
-------

Taking into account the design considerations outlined in OEP-50 Hooks extension framework about

1. The order of execution for multiple actions must be respected
2. The result of a previous action must be available to the current one

We must implement a pattern that follows this considerations: a Pipeline for the set of functions, i.e actions or filters,
listening on a trigger.

Checkout https://github.com/edx/open-edx-proposals/pull/184 for more information.

To do this, we considered three similar approaches for the pipeline implementation:

1. Unix-like (how unix pipe operator works): the output of the previous function is the input of the next one. For the first function, includes the initial arguments.
2. Unix-like modified: the output of the previous function is the input of the next one including the initial arguments for all functions.
3. Accumulative: the output of every (i, …, i-n) function is accumulated using a data structure and fed into the next function i-n+1, including the initial arguments. We draw inspiration from
python-social-auth/social-core, please checkout their repository: https://github.com/python-social-auth/social-core

These approaches follow the pipeline pattern and have as main difference what each function receive as input.

It is important to emphasize that the main objectives with this implementation are: to have function interchangeability and to maintain the signature of the functions across the Hooks Extension Framework.

Decision
--------

We decided to use the accumulative approach as the only pipeline for both actions and filters.

Consequences
------------

1. Actions listening on a trigger must return None. Either way their result will be ignored.
2. Given that we are using just one pipeline with actions and filters listening on triggers, the behavior when executing them will be the same.
3. Given that actions and filters will expect the same input arguments, i.e accumulated output plus initial arguments, their signature will stay the same. And for this reason, these functions are interchangeable.
4. For the above reasons, actions and filters must have \*args and \*\*kwargs in their signature.
4 changes: 4 additions & 0 deletions edx_django_utils/hooks/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
"""
File used to expose available triggers.
"""
from .triggers import trigger_action, trigger_filter
26 changes: 26 additions & 0 deletions edx_django_utils/hooks/exceptions.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
"""
Exceptions thrown by Hooks.
"""


class HookException(Exception):
"""
Base exception for hooks. It is re-raised by the Pipeline Runner if any of
the actions/filters that is executing raises it.
Arguments:
message (str): message describing why the exception was raised.
redirect_to (str): redirect URL.
status_code (int): HTTP status code.
keyword arguments (kwargs): extra arguments used to customize exception.
"""

def __init__(self, message="", redirect_to=None, status_code=None, **kwargs):
super().__init__()
self.message = message
self.redirect_to = redirect_to
self.status_code = status_code
self.kwargs = kwargs

def __str__(self):
return "HookException: {}".format(self.message)
84 changes: 84 additions & 0 deletions edx_django_utils/hooks/pipeline.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
"""
Pipeline runner used to execute list of functions (actions or filters).
"""
from logging import getLogger

from .exceptions import HookException
from .utils import get_functions_for_pipeline

log = getLogger(__name__)


def run_pipeline(pipeline, *args, raise_exception=False, **kwargs):
"""
Given a list of functions paths, this function will execute them using the Accumulative Pipeline
pattern defined in docs/decisions/0006-hooks-tooling-pipeline.rst
Example usage:
result = run_pipeline(
[
'my_plugin.hooks.filters.test_function',
'my_plugin.hooks.filters.test_function_2nd'
],
raise_exception=True,
request=request,
user=user,
)
>>> result
{
'result_test_function': Object,
'result_test_function_2nd': Object_2nd,
}
Arguments:
pipeline (list): paths where each function is defined.
Keyword arguments:
raise_exception (bool): used to determine whether the pipeline will raise HookExceptions. Default is set
to False.
Returns:
out (dict): accumulated outputs of the functions defined in pipeline.
result (obj): return object of one of the pipeline functions. This will be the return object for the pipeline
if one of the functions returns an object different than Dict o None.
Exceptions raised:
HookException: custom exception re-raised when a function raised an exception of
this type and raise_exception is set to True. This behavior is common when using filters.
This pipeline implementation was inspired by: Social auth core. For more information check their Github
repository: https://github.com/python-social-auth/social-core
"""
functions = get_functions_for_pipeline(pipeline)

out = kwargs.copy()
for function in functions:
try:
result = function(*args, **out) or {}
if not isinstance(result, dict):
log.info(
"Pipeline stopped by '%s' for returning an object.",
function.__name__,
)
return result
out.update(result)
except HookException as exc:
if raise_exception:
log.exception(
"Exception raised while running '%s':\n %s", function.__name__, exc,
)
raise exc
except Exception as exc: # pylint: disable=broad-except
# We're catching this because we don't want the core to blow up when a
# hook is broken. This exception will probably need some sort of
# monitoring hooked up to it to make sure that these errors don't go
# unseen.
log.exception(
"Exception raised while running '%s': %s\n%s",
function.__name__,
exc,
"Continuing execution.",
)
continue

return out
Empty file.
Loading

0 comments on commit b50a5f0

Please sign in to comment.