Remove gunicorn daemonize for api-server #52860

jason810496 · 2025-07-04T10:21:10Z

related: #52581

Why

When I attempt to remove gunicorn dependency in #52581, I found that we are still using gunicorn for daemonization.

What

Replace gunicorn.daemonize with run_command_with_daemon_option, this also align the behavior of --daemon option for scheduler, triggerer ... and api-server
Add test for api-server with --daemon as well
- Rename _CommonCLIGunicornTestClass to _CommonCLIUvicornTestClass, since we are using uvicorn instead of gunicorn for api-server now.
Replace [api/access_logfile] config with [api/log_config], detail in Remove gunicorn daemonize for api-server #52860 (comment)

jason810496

access_log=access_logfile, # type: ignore[arg-type] this line is a strange part for me.

I add # type: ignore[arg-type] to ingore the mypy warning for now.

airflow-core/src/airflow/cli/commands/api_server_command.py

jason810496 · 2025-07-05T15:30:25Z

Hi @potiuk, may I might need hints for replacing config.

I thought replacing them in config.yaml and add the pair in deprecated_option in configuration will be all right.

Thanks in advance!

Update:

I found the root cause.

My step is correct but I set [api/log_config] config default as None in config.yml and I set default=conf.get("api", "log_config") in cli_config module.
There is not need to set default=conf.get("api", "log_config") or should set fallback=None for the --log-config flag in cli_config, since the it is not mandatory.

jason810496

Thanks @potiuk and @pierrejeambrun for the review!
Just resolved all comments and fixed the CI as well.

airflow-core/src/airflow/config_templates/config.yml

potiuk

Nice. One NIT about updating newsfragment.

- the logging before daemonization is not logging the correct PID so remove it - should_setup_logging is default as False

* Remove gunicorn daemonize for api-server * Rename _CommonCLIUvicornTestClass * Add test for api-server with daemon option * Fix setproctitle on MacOS * Replace api/access_logfile with api/log_config * Fix [api/log_config] cli_config * Remove logging and unsed option in api_server_command - the logging before daemonization is not logging the correct PID so remove it - should_setup_logging is default as False * Fix test_run_command_daemon * Add significant rst

* Remove gunicorn daemonize for api-server * Rename _CommonCLIUvicornTestClass * Add test for api-server with daemon option * Fix setproctitle on MacOS * Replace api/access_logfile with api/log_config * Fix [api/log_config] cli_config * Remove logging and unsed option in api_server_command - the logging before daemonization is not logging the correct PID so remove it - should_setup_logging is default as False * Fix test_run_command_daemon * Add significant rst (cherry picked from commit ab1ee72)

* [v3-0-test] Remove gunicorn daemonize for api-server (#52860) * Remove gunicorn daemonize for api-server * Rename _CommonCLIUvicornTestClass * Add test for api-server with daemon option * Fix setproctitle on MacOS * Replace api/access_logfile with api/log_config * Fix [api/log_config] cli_config * Remove logging and unsed option in api_server_command - the logging before daemonization is not logging the correct PID so remove it - should_setup_logging is default as False * Fix test_run_command_daemon * Add significant rst (cherry picked from commit ab1ee72) * Fix spelling wordlist * Fix configuration

jedcunningham · 2025-07-31T18:27:38Z

airflow-core/src/airflow/configuration.py

        ("api", "require_confirmation_dag_change"): ("webserver", "require_confirmation_dag_change", "3.1.0"),
        ("api", "instance_name"): ("webserver", "instance_name", "3.1.0"),
        ("dag_processor", "parsing_pre_import_modules"): ("scheduler", "parsing_pre_import_modules", "3.1.0"),
+        ("api", "log_config"): ("api", "access_logfile", "3.1.0"),


This is not safe to do - they are different config options. As mentioned in the comment above, this will use the old access_logfile value for log_config if it's set, which isn't what we want.

jedcunningham · 2025-07-31T19:12:11Z

airflow-core/newsfragments/52860.significant.rst

+Replace API server ``access_logfile`` configuration with ``log_config``
+
+The API server configuration option ``[api] access_logfile`` has been replaced with ``[api] log_config`` to align with uvicorn's logging configuration instead of the legacy gunicorn approach.
+The new ``log_config`` option accepts a path to a logging configuration file compatible with ``logging.config.fileConfig``, providing more flexible logging configuration for the API server.


Is there a way we can continue to support a simple access logfile config? I just worry a bit about backcompat, right now this is a breaking change. Maybe we leave access_logfile, build our own log_config file dynamically, and make log_config and access_logfile mutually exclusive? And we should emit a deprecation warning when access_logfile is used.

Well, good and bad news. Good news is no backcompat issues to worry about! I believe it got broken in #45103. Bad news is I don't think access_log ever worked in AF3.

…52860) (apache#53372)" This reverts commit 7212c88.

…#53372)" (#54002) This reverts commit 7212c88.

boring-cyborg bot added the area:CLI label Jul 4, 2025

jason810496 force-pushed the refactor/api-server/remove-gunicorn-daemonize branch from 1dfe755 to 7a44f9a Compare July 4, 2025 10:22

jason810496 added the area:API Airflow's REST/HTTP API label Jul 4, 2025

jason810496 marked this pull request as ready for review July 4, 2025 10:22

jason810496 commented Jul 4, 2025

View reviewed changes

airflow-core/src/airflow/cli/commands/api_server_command.py Outdated Show resolved Hide resolved

jason810496 requested review from pierrejeambrun and potiuk July 4, 2025 10:29

potiuk reviewed Jul 4, 2025

View reviewed changes

airflow-core/src/airflow/cli/commands/api_server_command.py Show resolved Hide resolved

potiuk reviewed Jul 4, 2025

View reviewed changes

airflow-core/src/airflow/cli/commands/api_server_command.py Outdated Show resolved Hide resolved

jason810496 marked this pull request as draft July 6, 2025 02:29

jason810496 force-pushed the refactor/api-server/remove-gunicorn-daemonize branch from 89b529e to 52151fe Compare July 6, 2025 03:08

jason810496 marked this pull request as ready for review July 6, 2025 04:30

jason810496 commented Jul 6, 2025

View reviewed changes

jason810496 mentioned this pull request Jul 6, 2025

Respect apps flags for api_server_command #52929

Merged

jason810496 self-assigned this Jul 6, 2025

potiuk reviewed Jul 6, 2025

View reviewed changes

airflow-core/src/airflow/config_templates/config.yml Show resolved Hide resolved

potiuk approved these changes Jul 6, 2025

View reviewed changes

jason810496 added 9 commits July 7, 2025 22:02

Remove gunicorn daemonize for api-server

8be9cc5

Rename _CommonCLIUvicornTestClass

bcdf81e

Add test for api-server with daemon option

215f40a

Fix setproctitle on MacOS

694ffa6

Replace api/access_logfile with api/log_config

9715977

Fix [api/log_config] cli_config

ebaada7

Remove logging and unsed option in api_server_command

0bf37d4

- the logging before daemonization is not logging the correct PID so remove it - should_setup_logging is default as False

Fix test_run_command_daemon

40c356c

Add significant rst

ef4c730

jason810496 force-pushed the refactor/api-server/remove-gunicorn-daemonize branch from 52151fe to ef4c730 Compare July 7, 2025 14:02

potiuk approved these changes Jul 7, 2025

View reviewed changes

potiuk merged commit ab1ee72 into apache:main Jul 7, 2025
57 checks passed

This was referenced Jul 10, 2025

Fix setproctitle usage #53122

Merged

Cleanup gunicorn_config module #53123

Merged

jason810496 mentioned this pull request Jul 15, 2025

[v3-0-test] Remove gunicorn daemonize for api-server (#52860) #53372

Merged

jedcunningham reviewed Jul 31, 2025

View reviewed changes

jedcunningham added a commit to astronomer/airflow that referenced this pull request Jul 31, 2025

Revert "[v3-0-test] Remove gunicorn daemonize for api-server (apache#…

3fdcefa

…52860) (apache#53372)" This reverts commit 7212c88.

jedcunningham mentioned this pull request Jul 31, 2025

Revert "[v3-0-test] Remove gunicorn daemonize for api-server (#52860) (#53372)" #54002

Merged

jedcunningham added a commit that referenced this pull request Aug 1, 2025

Revert "[v3-0-test] Remove gunicorn daemonize for api-server (#52860) (…

69bb85b

…#53372)" (#54002) This reverts commit 7212c88.

ashb mentioned this pull request Aug 5, 2025

Status of testing Apache Airflow 3.0.4rc2 and TaskSDK 1.0.4rc1 #54124

Closed

82 tasks

eladkal mentioned this pull request Sep 5, 2025

Status of testing Providers that were prepared on September 05, 2025 #55285

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Remove gunicorn daemonize for api-server #52860

Remove gunicorn daemonize for api-server #52860

Uh oh!

jason810496 commented Jul 4, 2025 •

edited

Loading

Uh oh!

jason810496 left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jason810496 commented Jul 5, 2025 •

edited

Loading

Uh oh!

jason810496 left a comment

Uh oh!

Uh oh!

potiuk left a comment

Uh oh!

Uh oh!

jedcunningham Jul 31, 2025

Uh oh!

jedcunningham Jul 31, 2025

Uh oh!

jedcunningham Jul 31, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Remove gunicorn daemonize for api-server #52860

Remove gunicorn daemonize for api-server #52860

Uh oh!

Conversation

jason810496 commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What

Uh oh!

jason810496 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jason810496 commented Jul 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jason810496 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

potiuk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jedcunningham Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

jedcunningham Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

jedcunningham Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jason810496 commented Jul 4, 2025 •

edited

Loading

jason810496 left a comment •

edited

Loading

jason810496 commented Jul 5, 2025 •

edited

Loading