Skip to content

[Airflow 2.x] check if stand alone dag processor is active in get_health endpoint#48612

Merged
pierrejeambrun merged 5 commits intoapache:v2-11-testfrom
rawwar:kalyan/dag_processor_metrics_fix
May 19, 2025
Merged

[Airflow 2.x] check if stand alone dag processor is active in get_health endpoint#48612
pierrejeambrun merged 5 commits intoapache:v2-11-testfrom
rawwar:kalyan/dag_processor_metrics_fix

Conversation

@rawwar
Copy link
Contributor

@rawwar rawwar commented Apr 1, 2025

If I have a stand alone dag-processor and I turn it off later, health metrics will keep showing that the dag-processor is unhealthy. This PR fixes it to show None to indicate dag-processor no longer exists

EDIT: only for Airflow 2.x

@eladkal
Copy link
Contributor

eladkal commented Apr 1, 2025

If I have a stand alone dag-processor and I turn it off later

In Airflow 3 we support only standalone dag processor so you won't be able to turn it off.
https://lists.apache.org/thread/cx1mg5g1pw4k1km56hzr06d0o39kdk8d

@rawwar
Copy link
Contributor Author

rawwar commented Apr 1, 2025

If I have a stand alone dag-processor and I turn it off later

In Airflow 3 we support only standalone dag processor so you won't be able to turn it off. https://lists.apache.org/thread/cx1mg5g1pw4k1km56hzr06d0o39kdk8d

Sorry. This is for 2.x. I raised the PR to 2.x only

@rawwar rawwar changed the title check if stand alone dag processor is active in get_health endpoint [Airflow 2.x] check if stand alone dag processor is active in get_health endpoint Apr 1, 2025
Copy link
Contributor

@bugraoz93 bugraoz93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Small nit, and could you please update the unit tests? Some of them failing related to this change

Co-authored-by: Bugra Ozturk <bugraoz93@users.noreply.github.com>
@rawwar rawwar marked this pull request as draft April 4, 2025 03:48
@rawwar
Copy link
Contributor Author

rawwar commented Apr 4, 2025

Converting to draft as I need to fix tests and I can't work on it right now.

Copy link
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice

@rawwar rawwar changed the base branch from v2-10-test to v2-11-test May 18, 2025 21:52
@rawwar rawwar marked this pull request as ready for review May 18, 2025 21:53
@pierrejeambrun pierrejeambrun added this to the Airflow 2.11.1 milestone May 19, 2025
@pierrejeambrun pierrejeambrun merged commit 362409a into apache:v2-11-test May 19, 2025
47 of 70 checks passed
@rawwar rawwar deleted the kalyan/dag_processor_metrics_fix branch May 19, 2025 09:46
potiuk pushed a commit that referenced this pull request Feb 16, 2026
…lth endpoint (#48612)

* check if stand alone dag processor is used

* Update airflow/api/common/airflow_health.py

Co-authored-by: Bugra Ozturk <bugraoz93@users.noreply.github.com>

* fix tests

* update tests

---------

Co-authored-by: Bugra Ozturk <bugraoz93@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

Comments