Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DAG Import Errors remain in the WebUI even after fix error #39762

Closed
1 of 2 tasks
sanromeo opened this issue May 22, 2024 · 6 comments
Closed
1 of 2 tasks

DAG Import Errors remain in the WebUI even after fix error #39762

sanromeo opened this issue May 22, 2024 · 6 comments
Labels
affected_version:2.9 area:core Can't Reproduce The problem cannot be reproduced kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet pending-response

Comments

@sanromeo
Copy link

Apache Airflow version

2.9.1

If "Other Airflow 2 version" selected, which one?

2.9.0, 2.9.1

What happened?

DAG Import Errors remain in the WebUI even after fix error

What you think should happen instead?

DAG Import Errors will disappear in WebUI after fix errors

How to reproduce

  1. Create DAG with broken parameters that will cause DAG Import Errors.
  2. Look at Airflow WebUI, you will see error.
  3. Fix it :)
  4. Error will remain the same in WebUI but should disappear after fix.

Operating System

MacOS or Linux

Versions of Apache Airflow Providers

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@sanromeo sanromeo added area:core kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet labels May 22, 2024
Copy link

boring-cyborg bot commented May 22, 2024

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

@eladkal
Copy link
Contributor

eladkal commented May 22, 2024

I can't reproduce this.
Please provide reproduce example

@sanromeo
Copy link
Author

sanromeo commented May 23, 2024

Sorry, it's not related to all errors, only to ValidationError with pydantic for me
airflow dags list-import-errors -o json show no errors
but in Web UI there is still error appear
Also link to reddit thread:
https://www.reddit.com/r/apache_airflow/comments/14oji77/airflow_262_and_pydantic_warningserrors/

Seems it happenned after upgrate pydantic to 2.+ version

@KulykDmytro
Copy link
Contributor

Data remains in import_errors table despite of fixed DAG code
Started to appear since 2.9.0 upgrade

@Taragolis
Copy link
Contributor

Check on both 2.9.0 and 2.9.1 by use sample docker compose

With this DAG it worked as expected

from __future__ import annotations

from datetime import datetime, timezone

from airflow.decorators import task
from airflow.models.dag import DAG

raise RuntimeError("Boooom!")

START_DATE = datetime(2024, 2, 29, tzinfo=timezone.utc)


with DAG("issue_39762", schedule="@once", tags=["issue", "39762", "dag-import"], start_date=START_DATE):
    @task
    def do_nothing(): ...

    do_nothing()

Gone from both DB and UI as soon as I comment out / remove raise RuntimeError("Boooom!")


Expected to see some examples and more information about your configurations

@Taragolis Taragolis added the Can't Reproduce The problem cannot be reproduced label May 23, 2024
@sanromeo
Copy link
Author

The root cause was that we have our own DagFactory and did not use the processor_subdir parameter in errors.ImportError since we have one DAG-processor, so we specified the folder with dags as processor_subdir and the problem was solved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affected_version:2.9 area:core Can't Reproduce The problem cannot be reproduced kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet pending-response
Projects
None yet
Development

No branches or pull requests

4 participants