Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature]: Add error handling for deferred functions #98

Open
tsv1 opened this issue Sep 22, 2024 · 0 comments
Open

[Feature]: Add error handling for deferred functions #98

tsv1 opened this issue Sep 22, 2024 · 0 comments

Comments

@tsv1
Copy link
Member

tsv1 commented Sep 22, 2024

Summary:

Currently, if a deferred function raises an exception in the DeferrerPlugin, subsequent deferred functions are not executed, and no error information is logged. This behavior can lead to silent failures, making it difficult to diagnose issues and ensuring that some deferred actions are not performed. This feature request proposes adding error handling to ensure that, if a deferred function fails, the error is logged, and the remaining deferred functions are still executed. Additionally, users should have the option to configure whether execution should stop or continue when an error occurs in a deferred function.

Proposed Solution:

  1. Error Handling:
    Modify the DeferrerPlugin to catch exceptions when executing deferred functions. If an exception is raised by a function or coroutine, the plugin should log the error, including details about the function and the exception. Despite the error, execution of subsequent deferred functions should continue.

  2. Optional Configuration:
    Add a configuration option to allow users to control whether the plugin should continue or stop execution if a deferred function raises an exception:

    • --defer-stop-on-error (default: False): When set to True, the plugin should stop executing deferred functions upon encountering an error. When set to False, the plugin should log the error and continue executing the remaining deferred functions.
  3. Logging:
    If a deferred function fails, log the function name, arguments, and the exception traceback for easier debugging.

Example Usage:

  1. Command-line usage:

    vedro run --defer-stop-on-error

    This option will cause the plugin to stop executing remaining deferred functions if one fails.

  2. Configuration via plugin setup:

    class CustomDeferrer(Deferrer):
        defer_stop_on_error = True  # Stop execution on error
  3. Error logging example:

    ERROR: Deferred function <function_name> failed with arguments: (arg1, arg2), kwargs: {key: value}.
    Traceback (most recent call last):
    ...
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant