Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

OpenLineage: Add AirflowRunFacet to DAG run events #40798

Closed
2 tasks done
dolfinus opened this issue Jul 15, 2024 · 1 comment · Fixed by #40854
Closed
2 tasks done

OpenLineage: Add AirflowRunFacet to DAG run events #40798

dolfinus opened this issue Jul 15, 2024 · 1 comment · Fixed by #40854
Assignees
@dolfinus
Labels
area:providers kind:feature Feature Requests provider:openlineage AIP-53

Comments

@dolfinus
Copy link
Contributor

dolfinus commented Jul 15, 2024 •
edited
Loading

Description

Add AirflowDagRunFacet facet for DAG run events.

Use case/motivation

Currently AirflowRunFacet is present only RunEvents produces by Airflow tasks (containing fields dag, dag_run, task, task_instance). But DAG can be started too, and it currently contains only AirflowJobFacet (fields like taskTree, taskGroups and so on).

Having dag_run field allows custom OpenLineage backend implementations to have URL for Airflow UI page of a particular DagRun, as well as have custom logic based on run_type field value.

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct
@dolfinus dolfinus added kind:feature Feature Requests needs-triage label for new issues that we didn't triage yet labels Jul 15, 2024
@dolfinus dolfinus changed the title Add AirflowRunFacet to DAG run events OpenLineage: Add AirflowRunFacet to DAG run events Jul 15, 2024
@dosubot dosubot bot added the provider:openlineage AIP-53 label Jul 15, 2024
@shahar1 shahar1 removed the needs-triage label for new issues that we didn't triage yet label Jul 15, 2024
@dolfinus dolfinus mentioned this issue Jul 18, 2024
Merged
@dolfinus
Copy link
Contributor Author

Implementation: #40854

This was referenced Jul 26, 2024
Merged
Closed
@eladkal eladkal mentioned this issue Aug 3, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dolfinus dolfinus

Labels
area:providers kind:feature Feature Requests provider:openlineage AIP-53
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

openlineage: Add AirflowDagRunFacet to dag runEvents dolfinus/airflow
2 participants
@dolfinus @shahar1