Tasks get stuck when using Airflow base url parameter

[inkerinmaa]

Apache Airflow version

3.0.0

If "Other Airflow 2 version" selected, which one?

No response

What happened?

Hi. I use docker-compose and want to run Airflow with subpath, like
AIRFLOW__API__BASE_URL: ' http://mycompany.com/airflow'
UI is loading fine, with subpath provided, but when I trigger any dag it stucks in queued state. Worker logs are empty - just info that it is ready
2025-04-24 14:12:55.240510 [info ] celery@00da0db4ba2e ready. [celery.apps.worker]
Then after a while it gets ServerResponseError: Not Found

err = ServerResponseError('Not Found')
response = <Response [404 Not Found]>
ServerResponseError: Not Found

What you think should happen instead?

No response

How to reproduce

Use official docker compose and add AIRFLOW__API__BASE_URL: ' http://mycompany.com/airflow' env variable

Operating System

Debian

Versions of Apache Airflow Providers

No response

Deployment

Docker-Compose

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[tirkarthi]

Is 404 seen for all API calls? What is the value of conf.get("core", "execution_api_server_url") in airflow configuration?

airflow/providers/celery/src/airflow/providers/celery/executors/celery_executor_utils.py

Lines 161 to 169 in ed06d99

	supervise(
	# This is the "wrong" ti type, but it duck types the same. TODO: Create a protocol for this.
	ti=workload.ti, # type: ignore[arg-type]
	dag_rel_path=workload.dag_rel_path,
	bundle_info=workload.bundle_info,
	token=workload.token,
	server=conf.get("core", "execution_api_server_url"),
	log_path=workload.log_path,
	)

[kaxil]

aah yes, we need the AIRFLOW__CORE__EXECUTION_API_SERVER_URL!!

[pierrejeambrun]

As mentioned by Kaxil, indeed, when you set AIRFLOW__API__BASE_URL you'll also want to set AIRFLOW__CORE__EXECUTION_API_SERVER_URL. Because execution api and core api are not necessarily served together.

[pierrejeambrun]

Closing, feel free to reopen if needed.

[inkerinmaa]

Closing, feel free to reopen if needed.

@pierrejeambrun Can you reopen it, please, since provided solution doesn't work. I have set that variable but now all tasks stuck in queued state and worker logs are empty

[pierrejeambrun]

@inkerinmaa can you provide more context please this is not expected. Do you still see the same type of logs in the task sdk ?

Sounds like the task sdk still cannot reach the the execution api server.

[inkerinmaa]

@pierrejeambrun During my experiments with official docker compose I've got to the point when sign in goes to infinite loop, but this is another issue already registered here. So, I stopped using base url for now, but without base url I also see some unconsistent behavior - sometimes dag runs without any problems, sometimes - stuck in queued state and sometimes "Clear run" helps, sometimes - not. But I will ask in Slack first, maybe, I am doing sth wrong. Thanks

[pierrejeambrun]

Thanks for the feedback. Yes this is a different issue #49495 you should not experience this, there's already a patch for it, maybe wait for 3.0.1 or you can apply the patch to your image and see if the issue persists. (more info in the #49495 thread).

Indeed it sound like a misconfiguration, let us know when you have more information.

[inkerinmaa]

@pierrejeambrun I tried to explain what's going on in Slack https://apache-airflow.slack.com/archives/CCQ7EGB1P/p1745657909554469?thread_ts=1745657909.554469&cid=CCQ7EGB1P