Skip to content

Conversation

@nrnavaneet
Copy link
Contributor

@nrnavaneet nrnavaneet commented Jun 26, 2025

This PR updates the systemd integration to align with changes introduced in Apache Airflow 3.0.

Changes made:

  • Replaced deprecated webserver unit with the new api-server service
  • Added missing service unit files for:
    • airflow-triggerer
    • airflow-dag-processor
    • airflow-api
  • Updated all ExecStart commands to support virtual environment usage (e.g., sourcing a venv before running Airflow components)
  • Revised run-with-systemd.rst to:
    • Reflect Airflow 3.0 architectural changes
    • Include virtualenv usage examples
    • Link to the GitHub scripts/systemd folder directly

Testing

Manually tested Airflow 3.0 on macOS with a virtualenv:

  • Verified command structure in updated .service files
  • Confirmed successful startup of scheduler, api-server, and triggerer
  • Verified new CLI structure and DB setup in Airflow 3.0

closes #51117

nrnavaneet and others added 5 commits June 25, 2025 04:10
- Added a note under "Using Airflow Public Interfaces" to recommend using `airflow.sdk` as the official interface from Airflow 3.0.
- Referenced AIP-72 and linked related documentation.
- Encouraged users to prefer REST API and Python Client for integrations.
Great

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
- Updated systemd unit files to support Airflow 3.0 architecture.
- Replaced deprecated `webserver` with `api-server` in ExecStart.
- Added service units for new components: `triggerer`, `dag-processor`, and `api-server`.
- Adjusted ExecStart commands to support virtualenv-based deployments.
- Improved documentation (`run-with-systemd.rst`) to clarify service changes, virtualenv usage, and installation paths.
- Linked to the updated GitHub `scripts/systemd` directory for latest service files.

Tested Airflow 3.0 manually on macOS using virtualenv, confirming correct startup behavior for `scheduler`, `api-server`, and `triggerer`.
@boring-cyborg boring-cyborg bot added area:dev-tools backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch kind:documentation labels Jun 26, 2025
@potiuk
Copy link
Member

potiuk commented Jun 26, 2025

This is very cool, thanks - but there are some unrelated doc changes that should be reverted

@nrnavaneet nrnavaneet closed this Jun 27, 2025
@nrnavaneet nrnavaneet deleted the fix/systemd-airflow3 branch June 27, 2025 16:38
@nrnavaneet nrnavaneet restored the fix/systemd-airflow3 branch July 21, 2025 14:31
@nrnavaneet nrnavaneet reopened this Jul 21, 2025
@nrnavaneet
Copy link
Contributor Author

nrnavaneet commented Jul 21, 2025

Hey @potiuk, Not sure why I closed it.I've made all the changes needed which I feel shld be there. Kindly lmk if I have to make any other changes.
Thanks again :)

@nrnavaneet nrnavaneet requested a review from potiuk July 21, 2025 16:16
Copy link
Member

@potiuk potiuk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@potiuk potiuk merged commit 097ae45 into apache:main Jul 21, 2025
57 checks passed
github-actions bot pushed a commit that referenced this pull request Jul 21, 2025
… for Airflow 3.0 (#52294)

* docs: update public interface doc to reflect airflow.sdk and AIP-72

- Added a note under "Using Airflow Public Interfaces" to recommend using `airflow.sdk` as the official interface from Airflow 3.0.
- Referenced AIP-72 and linked related documentation.
- Encouraged users to prefer REST API and Python Client for integrations.

* Update airflow-core/docs/public-airflow-interface.rst

Great

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>

* fix(systemd): Update systemd service files and docs for Airflow 3.0

- Updated systemd unit files to support Airflow 3.0 architecture.
- Replaced deprecated `webserver` with `api-server` in ExecStart.
- Added service units for new components: `triggerer`, `dag-processor`, and `api-server`.
- Adjusted ExecStart commands to support virtualenv-based deployments.
- Improved documentation (`run-with-systemd.rst`) to clarify service changes, virtualenv usage, and installation paths.
- Linked to the updated GitHub `scripts/systemd` directory for latest service files.

Tested Airflow 3.0 manually on macOS using virtualenv, confirming correct startup behavior for `scheduler`, `api-server`, and `triggerer`.

* fix(systemd): Update systemd service files and docs for Airflow 3.0

* Minor test failure fixes

---------
(cherry picked from commit 097ae45)

Co-authored-by: N R Navaneet <156576749+nrnavaneet@users.noreply.github.com>
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
@github-actions
Copy link

Backport successfully created: v3-0-test

Status Branch Result
v3-0-test PR Link

@nrnavaneet
Copy link
Contributor Author

Thank you :)

@nrnavaneet nrnavaneet deleted the fix/systemd-airflow3 branch July 21, 2025 19:48
github-actions bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Jul 21, 2025
… for Airflow 3.0 (apache#52294)

* docs: update public interface doc to reflect airflow.sdk and AIP-72

- Added a note under "Using Airflow Public Interfaces" to recommend using `airflow.sdk` as the official interface from Airflow 3.0.
- Referenced AIP-72 and linked related documentation.
- Encouraged users to prefer REST API and Python Client for integrations.

* Update airflow-core/docs/public-airflow-interface.rst

Great

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>

* fix(systemd): Update systemd service files and docs for Airflow 3.0

- Updated systemd unit files to support Airflow 3.0 architecture.
- Replaced deprecated `webserver` with `api-server` in ExecStart.
- Added service units for new components: `triggerer`, `dag-processor`, and `api-server`.
- Adjusted ExecStart commands to support virtualenv-based deployments.
- Improved documentation (`run-with-systemd.rst`) to clarify service changes, virtualenv usage, and installation paths.
- Linked to the updated GitHub `scripts/systemd` directory for latest service files.

Tested Airflow 3.0 manually on macOS using virtualenv, confirming correct startup behavior for `scheduler`, `api-server`, and `triggerer`.

* fix(systemd): Update systemd service files and docs for Airflow 3.0

* Minor test failure fixes

---------
(cherry picked from commit 097ae45)

Co-authored-by: N R Navaneet <156576749+nrnavaneet@users.noreply.github.com>
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
potiuk added a commit that referenced this pull request Jul 21, 2025
… for Airflow 3.0 (#52294) (#53609)

* docs: update public interface doc to reflect airflow.sdk and AIP-72

- Added a note under "Using Airflow Public Interfaces" to recommend using `airflow.sdk` as the official interface from Airflow 3.0.
- Referenced AIP-72 and linked related documentation.
- Encouraged users to prefer REST API and Python Client for integrations.

* Update airflow-core/docs/public-airflow-interface.rst

Great



* fix(systemd): Update systemd service files and docs for Airflow 3.0

- Updated systemd unit files to support Airflow 3.0 architecture.
- Replaced deprecated `webserver` with `api-server` in ExecStart.
- Added service units for new components: `triggerer`, `dag-processor`, and `api-server`.
- Adjusted ExecStart commands to support virtualenv-based deployments.
- Improved documentation (`run-with-systemd.rst`) to clarify service changes, virtualenv usage, and installation paths.
- Linked to the updated GitHub `scripts/systemd` directory for latest service files.

Tested Airflow 3.0 manually on macOS using virtualenv, confirming correct startup behavior for `scheduler`, `api-server`, and `triggerer`.

* fix(systemd): Update systemd service files and docs for Airflow 3.0

* Minor test failure fixes

---------
(cherry picked from commit 097ae45)

Co-authored-by: N R Navaneet <156576749+nrnavaneet@users.noreply.github.com>
Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:dev-tools backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch kind:documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scripts/systemd weren't updated to Airflow 3

2 participants