Skip to content

Got UnboundLocalError when I add EdgeExecutor into executor in Airflow3.0.2 #52326

@yanshil

Description

@yanshil

Apache Airflow Provider(s)

edge3

Versions of Apache Airflow Providers

apache-airflow-providers-edge3/1.1.1

Apache Airflow version

3.0.2

Operating System

Docker Image FROM apache/airflow:3.0.2-python3.9 (Debian GNU/Linux 12 (bookworm) )

Deployment

Official Apache Airflow Helm Chart

Deployment details

Using custom image as below

# 3.0.2
FROM apache/airflow:3.0.2-python3.9
# --- core ---
RUN pip install 'apache-airflow[sentry]==3.0.2'
RUN pip install 'apache-airflow[otel]==3.0.2'
RUN pip install 'apache-airflow[standard]==3.0.2'
RUN pip install 'apache-airflow[graphviz]==3.0.2'
RUN pip install 'apache-airflow[async]==3.0.2'
RUN pip install 'apache-airflow[s3fs]==3.0.0'

RUN pip install 'apache-airflow[edge3]==3.0.2'
RUN pip install 'apache-airflow[ldap]==3.0.2'
RUN pip install 'apache-airflow[postgres]==3.0.2'
RUN pip install 'apache-airflow[mysql]==3.0.2'
RUN pip install 'apache-airflow[celery]==3.0.2'
RUN pip install 'apache-airflow[apache-kafka]==3.0.2'
RUN pip install 'apache-airflow[ssh]==3.0.2'

and change settings in values.yaml from

executor: CeleryExecutor

to

executor: CeleryExecutor,airflow.providers.edge3.executors.EdgeExecutor

will reproduce this issue

What happened

[2025-06-27T07:08:13.177+0000] {cli_action_loggers.py:97} DEBUG - Calling callbacks: []
Traceback (most recent call last):
  File "/home/airflow/.local/bin/airflow", line 8, in <module>
    sys.exit(main())
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/__main__.py", line 55, in main
    args.func(args)
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/cli/cli_config.py", line 48, in command
    return func(*args, **kwargs)
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/utils/cli.py", line 112, in wrapper
    return f(*args, **kwargs)
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/utils/providers_configuration_loader.py", line 55, in wrapped_function
    return func(*args, **kwargs)
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/cli/commands/scheduler_command.py", line 52, in scheduler
    run_command_with_daemon_option(
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/cli/commands/daemon_utils.py", line 86, in run_command_with_daemon_option
    callback()
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/cli/commands/scheduler_command.py", line 55, in <lambda>
    callback=lambda: _run_scheduler_job(args),
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/cli/commands/scheduler_command.py", line 43, in _run_scheduler_job
    run_job(job=job_runner.job, execute_callable=job_runner._execute)
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/utils/session.py", line 101, in wrapper
    return func(*args, session=session, **kwargs)
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/jobs/job.py", line 347, in run_job
    return execute_job(job, execute_callable=execute_callable)
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/jobs/job.py", line 376, in execute_job
    ret = execute_callable()
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/jobs/scheduler_job_runner.py", line 1005, in _execute
    executor.start()
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/utils/session.py", line 101, in wrapper
    return func(*args, session=session, **kwargs)
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/providers/edge3/executors/edge_executor.py", line 109, in start
    self._check_db_schema(engine)
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/providers/edge3/executors/edge_executor.py", line 87, in _check_db_schema
    elif edge_job_command_len and edge_job_command_len != 2048:
UnboundLocalError: local variable 'edge_job_command_len' referenced before assignment

What you think should happen instead

The fix would be quite simple. !52328
I have already tested by replacing this file in my Dockerfile image build with

COPY edge_executor.py /home/airflow/.local/lib/python3.9/site-packages/airflow/providers/edge3/executors/edge_executor.py

How to reproduce

  1. Use prebuild image with [edge3] install
  2. Add airflow.providers.edge3.executors.EdgeExecutor to values.yaml executor
    For example, I got this error when I
executor: CeleryExecutor

to

executor: CeleryExecutor,airflow.providers.edge3.executors.EdgeExecutor
  1. Deploy and you will see the failure

Anything else

Actually I still got issue when deploying edge worker and got the following message when I try to start the worker

DEBUG - Calling callbacks: []
405 Client Error: Method Not Allowed for url: <AIRFLOW_HOST>/edge_worker/v1/worker/<WORKER-IP>

I will create another issue to describe it's behavior in Airflow 3.0.2 installation

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions