[CLI command only] Importing `Variable` from `airflow.sdk` causes an `ImportError: cannot import name 'SUPERVISOR_COMMS'` for top-level Variable when running `airflow dags reserialize`

[TJaniF]

EDIT: this only affects the CLI command airflow dags reserialize, the dag-processor handles the dag fine and it runs correctly.

Apache Airflow version

3.0.0

If "Other Airflow 2 version" selected, which one?

main

What happened?

This dag serializes and runs fine for from airflow.models import Variable but not for from airflow.sdk import Variable

from airflow.sdk import dag, Variable
from airflow.decorators import task 
# from airflow.models import Variable

# This one only works for imports from airflow.models, not from airflow.sdk
var1 = Variable.get("v1", "TOP_LEVEL_VAR")

@dag()
def test_var():

    @task 
    def get_var():
        print("TOP LEVEL VAR ", var1)

        # this one works for both imports, from airflow.sdk and from airflow.models
        var2 = Variable.get("v2", "TASK_LEVEL_VAR")
        print("TASK LEVEL VAR ", var2)
    
    get_var() 


test_var()

The error is only caused when the Variable is used at the top level of the dag, not when within a task.

Error:

Traceback (most recent call last):
  File "/opt/airflow/task-sdk/src/airflow/sdk/definitions/variable.py", line 53, in get
    return _get_variable(key, deserialize_json=deserialize_json)
  File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/context.py", line 185, in _get_variable
    from airflow.sdk.execution_time.task_runner import SUPERVISOR_COMMS
ImportError: cannot import name 'SUPERVISOR_COMMS' from 'airflow.sdk.execution_time.task_runner' (/opt/airflow/task-sdk/src/airflow/sdk/execution_time/task_runner.py)

What you think should happen instead?

The dag should serialize and run with the updated import.

How to reproduce

Add the above dag to your env, run airflow dags reserialize, see the error. Switch the import of Variable to airflow.models to see the dag work.

Operating System

MacOS

Versions of Apache Airflow Providers

None

Deployment

Other

Deployment details

breeze

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[vatsrahul1001]

@TJaniF I tried it on main and it works well. Can you try on latest main ?

[TJaniF]

Tried on the latest main and the issue seems to be limited to the airflow dags reseialize command. The dag processor can handle the dag fine. So not the most high priority issue in this case :)

The screenshot below shows the cli command output with the error and having processed the other 3 dags in my env, while the dag processor at the same time is happy to process all 4 dags including the one in this issue

[Prajwal7842]

Hi @vatsrahul1001 Since this is marked as low priority. Can I try this out and propose a fix, if this is still an issue in latest main ?

[potiuk]

Hi @vatsrahul1001 Since this is marked as low priority. Can I try this out and propose a fix, if this is still an issue in latest main ?

You can start from reproducing it and checking it - I don't think anyone will be able to answer your question without attempting to reproduce it. And also even checking and posting here "yep it looks fixed in main" is one of the best contributions you can make.