[CLI command only] Importing Variable from airflow.sdk causes an ImportError: cannot import name 'SUPERVISOR_COMMS' for top-level Variable when running airflow dags reserialize

EDIT: this only affects the CLI command airflow dags reserialize, the dag-processor handles the dag fine and it runs correctly.

Apache Airflow version

3.0.0

If "Other Airflow 2 version" selected, which one?

main

What happened?

This dag serializes and runs fine for from airflow.models import Variable but not for from airflow.sdk import Variable

from airflow.sdk import dag, Variable
from airflow.decorators import task 
# from airflow.models import Variable

# This one only works for imports from airflow.models, not from airflow.sdk
var1 = Variable.get("v1", "TOP_LEVEL_VAR")

@dag()
def test_var():

    @task 
    def get_var():
        print("TOP LEVEL VAR ", var1)

        # this one works for both imports, from airflow.sdk and from airflow.models
        var2 = Variable.get("v2", "TASK_LEVEL_VAR")
        print("TASK LEVEL VAR ", var2)
    
    get_var() 


test_var()

The error is only caused when the Variable is used at the top level of the dag, not when within a task.

Error:

Traceback (most recent call last):
  File "/opt/airflow/task-sdk/src/airflow/sdk/definitions/variable.py", line 53, in get
    return _get_variable(key, deserialize_json=deserialize_json)
  File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/context.py", line 185, in _get_variable
    from airflow.sdk.execution_time.task_runner import SUPERVISOR_COMMS
ImportError: cannot import name 'SUPERVISOR_COMMS' from 'airflow.sdk.execution_time.task_runner' (/opt/airflow/task-sdk/src/airflow/sdk/execution_time/task_runner.py)

What you think should happen instead?

The dag should serialize and run with the updated import.

How to reproduce

Add the above dag to your env, run airflow dags reserialize, see the error. Switch the import of Variable to airflow.models to see the dag work.

Operating System

MacOS

Versions of Apache Airflow Providers

None

Deployment

Other

Deployment details

breeze

Anything else?

No response

Are you willing to submit PR?

Yes I am willing to submit a PR!

Code of Conduct

I agree to follow this project's Code of Conduct

@TJaniF

@TJaniF I tried it on main and it works well. Can you try on latest main ?

Tried on the latest main and the issue seems to be limited to the airflow dags reseialize command. The dag processor can handle the dag fine. So not the most high priority issue in this case :)

The screenshot below shows the cli command output with the error and having processed the other 3 dags in my env, while the dag processor at the same time is happy to process all 4 dags including the one in this issue

@vatsrahul1001

Hi @vatsrahul1001 Since this is marked as low priority. Can I try this out and propose a fix, if this is still an issue in latest main ?

Hi @vatsrahul1001 Since this is marked as low priority. Can I try this out and propose a fix, if this is still an issue in latest main ?

You can start from reproducing it and checking it - I don't think anyone will be able to answer your question without attempting to reproduce it. And also even checking and posting here "yep it looks fixed in main" is one of the best contributions you can make.

@task

I am able to reproduce it locally with main development branch. However, having some difficulty in understanding how to fix this. As also mentioned above: SUPERVISOR_COMMS is getting initialized only inside the @task get_var method. Any recommendations on how to get started ? Thanks

@task

I am able to reproduce it locally with main development branch. However, having some difficulty in understanding how to fix this. As also mentioned above: SUPERVISOR_COMMS is getting initialized only inside the @task get_var method. Any recommendations on how to get started ? Thanks

Thanks for checking! I will check and add some pointers soon.

I encountered this issue on 3.0.1 when running a test that creates a dagbag. It went away when I stopped using the -n flag for pytest (from pytest-xdist). It seems to me like some race condition or IPC issue involving multiple instances of dagbag (or dag processors). Interestingly, -n 1 exhibits this issue, but -n 0 does not.

I seem to get this error in 3.0.1 as well when using Variable.get("my_variable") inside an external_python task. The variable is perfectly able to be retrieved from an ordinary task. The variable is also listed when executing the airflow variables list command from an python environment activated terminal. The latter suggests, that the virtual environment should be properly set up, in my opinion. This setup worked fine in airflow 2.10.5 (however using from airflow.models import Variable instead of from airflow.sdk import Variable)

I don't know if the cause of this issue is the same, but sounds similar to me.

I faced the same ImportError trying to evaluate LazyXComSequence(<unevaluated length>) in the reduce function during dynamic task mapping. Running with dag.test(), LocalExecutor, Airflow 3.0.1.

# Reduce function
@task
def send_notifications(data_to_notificate: list[tuple[Rule, ObjectStoragePath]], **context) -> None:
    for rule, input_file in data_to_notificate: # The error raises here
        ...

UPD: the same parameter is identified as LazySelectSequence([4 items]) in Airflow 2.11.0, and the DAG works properly.

@opeida

@opeida

Seconding the dag.test() note - I am able to replicate the same error trying to use Connection.get (similar to Variable.get in initial example) and running via dag.test() (apologies for non-task flow syntax)

from airflow.models.dag import DAG
from airflow.providers.standard.operators.python import PythonOperator
from airflow.sdk.definitions.connection import Connection

MSSQL_CONN_ID = "test_conn_id"

with DAG(
    dag_id="test_dag",
    catchup=False,
) as dag:
    PythonOperator(
        task_id = "foo",
        python_callable=Connection.get,
        op_args=[MSSQL_CONN_ID]
    )

if __name__ == "__main__":
    dag.test()

Running this DAG via dag.test() results in this error, running normally (via UI) is fine.

We're seeing this same error when running tests (pytest):

# Passes
def test_with_models_import() -> None:
    from airflow.models import Variable

    Variable.set("foo", "bar")
    assert Variable.get("foo") == "bar"


# Fails
def test_with_sdk_import() -> None:
    from airflow.sdk import Variable

    Variable.set("foo", "bar")
    assert Variable.get("foo") == "bar"

The seconds test fails with:

FAILED tests/dags/temp_test.py::test_with_sdk_import - ImportError:
cannot import name 'SUPERVISOR_COMMS' from 'airflow.sdk.execution_time.task_runner'
(<conda env dir>/lib/python3.12/site-packages/airflow/sdk/execution_time/...

Both on Airflow 3.0.1 and 3.0.2rc2.

Any news one that? We heavily use airflow dags reserialze and this error has significantly affected our workflow.

faced this issue in tests, when forget to update mock to airflow.sdk.Variables.get from airflow.models.Variables.get

I think the original issue in the thread is fixed. I am unable to reproduce this on latest main with airflow dags reserialize command. SS attached.
[

](url)

This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activity on this issue. Kindly asking to recheck the report against latest Airflow version and let us know if the issue is reproducible. The issue will be closed in next 30 days if no further activity occurs from the issue author.

Navigation Menu

Repository navigation

[CLI command only] Importing `Variable` from `airflow.sdk` causes an `ImportError: cannot import name 'SUPERVISOR_COMMS'` for top-level Variable when running `airflow dags reserialize` #48554

Apache Airflow version

If "Other Airflow 2 version" selected, which one?

What happened?

What you think should happen instead?

How to reproduce

Operating System

Versions of Apache Airflow Providers

Deployment

Deployment details

Anything else?

Are you willing to submit PR?

Code of Conduct

2 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[CLI command only] Importing Variable from airflow.sdk causes an ImportError: cannot import name 'SUPERVISOR_COMMS' for top-level Variable when running airflow dags reserialize #48554

Description

Apache Airflow version

If "Other Airflow 2 version" selected, which one?

What happened?

What you think should happen instead?

How to reproduce

Operating System

Versions of Apache Airflow Providers

Deployment

Deployment details

Anything else?

Are you willing to submit PR?

Code of Conduct

Activity

vatsrahul1001 commented on Mar 31, 2025

TJaniF commented on Mar 31, 2025

Prajwal7842 commented on Apr 1, 2025

potiuk commented on Apr 1, 2025

2 remaining items

Prajwal7842 commented on Apr 5, 2025

bugraoz93 commented on Apr 5, 2025

Dev-iL commented on May 21, 2025

fromm1990 commented on May 23, 2025

opeida commented on May 29, 2025

jablecherman commented on May 29, 2025

farkmarnum commented on Jun 6, 2025

alkismavridis commented on Jun 25, 2025

dshvedchenko commented on Aug 13, 2025

Prajwal7842 commented on Oct 18, 2025

github-actions commented on Oct 19, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

[CLI command only] Importing `Variable` from `airflow.sdk` causes an `ImportError: cannot import name 'SUPERVISOR_COMMS'` for top-level Variable when running `airflow dags reserialize` #48554