Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scheduler exceptions when Sentry and schedule_after_task_execution are enabled #36957

Closed
2 tasks done
jkramer-ginkgo opened this issue Jan 22, 2024 · 1 comment · Fixed by #37002
Closed
2 tasks done
Labels
area:core kind:bug This is a clearly a bug

Comments

@jkramer-ginkgo
Copy link
Contributor

Apache Airflow version

2.8.1

If "Other Airflow 2 version" selected, which one?

2.8.0

What happened?

Seeing exceptions in code at:

File "/opt/pyenv/versions/3.11.7/lib/python3.11/site-packages/airflow/sentry.py", line 127, in add_tagging
  task = task_instance.task
         ^^^^^^^^^^^^^^^^^^
AttributeError: type object 'TaskInstance' has no attribute 'task'

What you think should happen instead?

No response

How to reproduce

Have sentry_on = True and schedule_after_task_execution = True. Run a DAG that has tasks depending on other tasks. Scheduling of any downstream task will hit this bug.

Operating System

Linux (Ubuntu 22.04)

Versions of Apache Airflow Providers

No response

Deployment

Docker-Compose

Deployment details

No response

Anything else?

I was able to attach a debugger and root cause this:

_schedule_downstream_tasks uses @Sentry.enrich_errors which is expecting to be decorating a function that either has self as a TaskInstance or for self to have an attribute called task_instance.

A recent refactoring of this code broke this assumption. There are two paths forward to resolve as I see it:

  1. Update enrich_errors to look for ti in kwargs
  2. Remove @Sentry.enrich_errors from _schedule_downstream_tasks. I don't have the full context here, but I imagine the semantics of enriching errors for scheduling (as opposed to running tasks) doesn't make sense.

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@jkramer-ginkgo jkramer-ginkgo added area:core kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet labels Jan 22, 2024
Copy link

boring-cyborg bot commented Jan 22, 2024

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:core kind:bug This is a clearly a bug
Projects
None yet
2 participants