use 1 worker for log server

[mattp-]

fixes stacktraces of the form in standalone & running directly for the components that launch logserver under a multiprocessing process:

    scheduler  | Process SpawnProcess-1:1:
    scheduler  | Traceback (most recent call last):
    scheduler  | File "/opt/bb/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    scheduler  | self.run()
    scheduler  | File "/opt/bb/lib/python3.10/multiprocessing/process.py", line 108, in run
    scheduler  | self._target(*self._args, **self._kwargs)
    scheduler  | File "/mnt/dev/repos/airflow/.venv/lib/python3.10/site-packages/uvicorn/_subprocess.py", line 73, in subprocess_started
    scheduler  | sys.stdin = os.fdopen(stdin_fileno)  # pragma: full coverage
    scheduler  | File "/opt/bb/lib/python3.10/os.py", line 1030, in fdopen
    scheduler  | return io.open(fd, mode, buffering, encoding, *args, **kwargs)
    scheduler  | OSError: [Errno 9] Bad file descriptor

there is a bug in uvicorn that causes a crash when uvicorn.run is being spawned from a process that itself was spawned by multiprocessing, as we do with serve_logs: Kludex/uvicorn#2679

until uvicorn itself fixes what it tries to do with stdin, just use 1 worker instead of 2 to avoid the codepath that causes the break.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our pre-commits will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[mattp-]

@jason810496 curious - any reason you chose 2 workers specifically for this when you refactored?

[jason810496]

@jason810496 curious - any reason you chose 2 workers specifically for this when you refactored?

Airflow use 2 workers before I refactor from Flask to FastAPI with uvicorn.

https://github.com/apache/airflow/pull/52581/files#diff-4829755d9e5ddcb1bce89f1cc272181507d40cb38891ce6abd4537e97004d38cL191

[jason810496]

By the way, how can I reproduce this error?
I have run the components ( scheduler, trigger ) with server_logs enable successfully in container.

[jason810496]

Or maybe which version of uvicorn will cause the problem you described above?

[mattp-]

ah gotcha. i put a simple reproducer in Kludex/uvicorn#2679 which is just straight uvicorn
running airflow standalone on main right now also repros for me on rhel 9 with uvicorn 0.35.0

[jason810496]

running airflow standalone on main

Thanks for the reply.
Will it raise the same error for just released Airflow 3.0.4 standalone as well?

[mattp-]

just checked, no problem on 3.0.4

[jason810496]

Cool, thanks for the confirmation.
It would be great to raise another issue to track this behavior.

This means there might not be uvicorn issue but other changes in Airflow that cause this error.
Since there is not problem on 3.0.4 but there is error on latest main branch.

[mattp-]

mphillips81@mmp-pw-dev /tmp/af3/.venv/lib/python3.10/site-packages/airflow % grep -rsi 'uvicorn'
cli/commands/api_server_command.py:import uvicorn
cli/commands/api_server_command.py:        log.warning("Running in dev mode, ignoring uvicorn args")
cli/commands/api_server_command.py:                Running the uvicorn with:
cli/commands/api_server_command.py:        uvicorn.run(
Binary file cli/commands/__pycache__/api_server_command.cpython-310.pyc matches

i think uvicorn is only being hit from api-server in the released 3.0.4 - the problem in main is the uvicorn.run is happening wrapped by a multiprocessing.Process when log-server is spawned - those changes aren't in 3.0.4

[jason810496]

Thanks @mattp- for helping out!

Sorry for the delay response, and this issue was just addressed in Fix scheduler serve_logs subprocess file descriptor errors #55443 so I will close your PR.
Thanks for your understanding.