airflow dags test needs dag to be serialized

#50825 re-introduced the bug where we can't run dags.test or airflow dags test command without Serializing DAG to DB manually. This adds huge friction to local dev.

This was originally fixed in 3.0.2 via #50420

Example failure:

root@40de4aa411de:/opt/airflow# airflow dags test a_dag
[2025-07-23T20:19:51.594+0000] {manager.py:122} INFO - DAG bundles loaded: dags-folder, example_dags
[2025-07-23T20:19:51.594+0000] {dagbag.py:620} INFO - Filling up the DagBag from /files/dags/daga.py
Nothing to clear.
/opt/airflow/providers/databricks/src/airflow/providers/databricks/plugins/databricks_workflow.py:44 DeprecationWarning: The `airflow.utils.task_group.TaskGroup` attribute is deprecated. Please use `'airflow.sdk.definitions.taskgroup.TaskGroup'`.
Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 10, in <module>
    sys.exit(main())
  File "/opt/airflow/airflow-core/src/airflow/__main__.py", line 55, in main
    args.func(args)
  File "/opt/airflow/airflow-core/src/airflow/cli/cli_config.py", line 48, in command
    return func(*args, **kwargs)
  File "/opt/airflow/airflow-core/src/airflow/utils/cli.py", line 113, in wrapper
    return f(*args, **kwargs)
  File "/opt/airflow/airflow-core/src/airflow/utils/providers_configuration_loader.py", line 54, in wrapped_function
    return func(*args, **kwargs)
  File "/opt/airflow/airflow-core/src/airflow/utils/session.py", line 100, in wrapper
    return func(*args, session=session, **kwargs)
  File "/opt/airflow/airflow-core/src/airflow/cli/commands/dag_command.py", line 622, in dag_test
    dr: DagRun = dag.test(
  File "/opt/airflow/task-sdk/src/airflow/sdk/definitions/dag.py", line 1113, in test
    dr: DagRun = _get_or_create_dagrun(
  File "/opt/airflow/airflow-core/src/airflow/models/dag.py", line 2326, in _get_or_create_dagrun
    dr = dag.create_dagrun(
  File "/opt/airflow/airflow-core/src/airflow/utils/session.py", line 98, in wrapper
    return func(*args, **kwargs)
  File "/opt/airflow/airflow-core/src/airflow/models/dag.py", line 1581, in create_dagrun
    orm_dagrun = _create_orm_dagrun(
  File "/opt/airflow/airflow-core/src/airflow/utils/session.py", line 98, in wrapper
    return func(*args, **kwargs)
  File "/opt/airflow/airflow-core/src/airflow/models/dag.py", line 249, in _create_orm_dagrun
    raise AirflowException(f"Cannot create DagRun for DAG {dag.dag_id} because the dag is not serialized")
airflow.exceptions.AirflowException: Cannot create DagRun for DAG a_dag because the dag is not serialized

https://github.com/apache/airflow/pull/50825/files#r2226635628

Fix `get_dag` function in CLI

Initialize DAG bundles in CLI `get_dag` function

Initialize DAG bundles and sync dags in CLI `get_dag` function

Initialize DAG bundles and sync dags in CLI `get_dag` function (#53699)

Initialize DAG bundles and sync dags in CLI `get_dag` function (apach…

@ephraimbuddy

@ephraimbuddy can we close this, looks like we already have a PR?

I will verify and close this

@vatsrahul1001

@vatsrahul1001 is this verified?

I am not getting the above error now we can trigger now; however, getting the below error now running airflow dags test

[2025-09-01T07:24:25.611+0000] {dag.py:1349} INFO - [DAG TEST] end task task_id=sleep map_index=-1
[2025-09-01T07:24:25.611+0000] {dag.py:1292} INFO - [DAG TEST] starting task_id=templated map_index=-1
[2025-09-01T07:24:25.611+0000] {dag.py:1295} INFO - [DAG TEST] running task <TaskInstance: tutorial.templated manual__2025-09-01T07:24:23.295650+00:00 [scheduled]>
2025-09-01 07:24:26 [debug    ] Starting task instance run     hostname=33dafbd4e807 pid=55183 ti_id=01990429-a98e-7752-8c01-d72925f6d03f unixname=root
2025-09-01 07:24:26 [debug    ] Retrieved task instance details dag_id=tutorial state=up_for_retry task_id=templated ti_id=01990429-a98e-7752-8c01-d72925f6d03f
2025-09-01 07:24:26 [warning  ] Cannot start Task Instance in invalid state previous_state=up_for_retry ti_id=01990429-a98e-7752-8c01-d72925f6d03f
[2025-09-01T07:24:26.661+0000] {_client.py:1025} INFO - HTTP Request: PATCH http://in-process.invalid./task-instances/01990429-a98e-7752-8c01-d72925f6d03f/run "HTTP/1.1 409 Conflict"
[2025-09-01T07:24:26.662+0000] {dag.py:1343} ERROR - [DAG TEST] Error running task <TaskInstance: tutorial.templated manual__2025-09-01T07:24:23.295650+00:00 [queued]>
Traceback (most recent call last):
  File "/opt/airflow/task-sdk/src/airflow/sdk/definitions/dag.py", line 1305, in _run_task
    taskrun_result = run_task_in_process(
  File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/supervisor.py", line 1577, in run_task_in_process
    return InProcessTestSupervisor.start(what=ti, task=task)
  File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/supervisor.py", line 1484, in start
    ti_context = supervisor.client.task_instances.start(supervisor.id, supervisor.pid, start_date)
  File "/opt/airflow/task-sdk/src/airflow/sdk/api/client.py", line 209, in start
    resp = self.client.patch(f"task-instances/{id}/run", content=body.model_dump_json())
  File "/usr/python/lib/python3.10/site-packages/httpx/_client.py", line 1218, in patch
    return self.request(
  File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/supervisor.py", line 1440, in request
    return super().request.__wrapped__(self, *args, **kwargs)  # type: ignore[attr-defined]
  File "/opt/airflow/task-sdk/src/airflow/sdk/api/client.py", line 860, in request
    return super().request(*args, **kwargs)
  File "/usr/python/lib/python3.10/site-packages/httpx/_client.py", line 825, in request
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/usr/python/lib/python3.10/site-packages/httpx/_client.py", line 914, in send
    response = self._send_handling_auth(
  File "/usr/python/lib/python3.10/site-packages/httpx/_client.py", line 942, in _send_handling_auth
    response = self._send_handling_redirects(
  File "/usr/python/lib/python3.10/site-packages/httpx/_client.py", line 999, in _send_handling_redirects
    raise exc
  File "/usr/python/lib/python3.10/site-packages/httpx/_client.py", line 982, in _send_handling_redirects
    hook(response)
  File "/opt/airflow/task-sdk/src/airflow/sdk/api/client.py", line 174, in raise_on_4xx_5xx
    return get_json_error(response) or response.raise_for_status()
  File "/opt/airflow/task-sdk/src/airflow/sdk/api/client.py", line 170, in get_json_error
    raise err
airflow.sdk.api.client.ServerResponseError: Server returned error
[2025-09-01T07:24:26.670+0000] {dag.py:1349} INFO - [DAG TEST] end task task_id=templated map_index=-1
[2025-09-01T07:24:26.672+0000] {dagrun.py:1171} INFO - Marking run <DagRun tutorial @ 2025-09-01 07:24:10.616100+00:00: manual__2025-09-01T07:24:23.295650+00:00, state:running, queued_at: None. run_type: manual> failed
Dag run  in failure state
Dag information:tutorial Run id: manual__2025-09-01T07:24:23.295650+00:00 Run type: manual
Failed with message: task_failure
[2025-09-01T07:24:26.672+0000] {dagrun.py:1266} INFO - DagRun Finished: dag_id=tutorial, logical_date=2025-09-01 07:24:10.616100+00:00, run_id=manual__2025-09-01T07:24:23.295650+00:00, run_start_date=2025-09-01 07:24:10.616100+00:00, run_end_date=2025-09-01 07:24:26.672732+00:00, run_duration=16.056632, state=failed, run_type=manual, data_interval_start=2025-09-01 07:24:10.616100+00:00, data_interval_end=2025-09-01 07:24:10.616100+00:00,
DagRun failed

we should create a new issue for the new error as dags are now serialized

@ephraimbuddy

@ephraimbuddy yeah right. Let me close this one and open up a new one.

@vatsrahul1001

@vatsrahul1001 you did not open the ticket you offered to open, or at least I can't find it in the Airflow issues list.

This issue still happens for dag.test(). I just created a new ticket to track this:
#56657

Navigation Menu

Repository navigation

`airflow dags test` needs dag to be serialized #53682

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

airflow dags test needs dag to be serialized #53682

Description

Activity

phanikumv commented on Aug 25, 2025

vatsrahul1001 commented on Aug 25, 2025

phanikumv commented on Sep 1, 2025

vatsrahul1001 commented on Sep 1, 2025

ephraimbuddy commented on Sep 1, 2025

vatsrahul1001 commented on Sep 1, 2025

DFINITYManu commented on Oct 9, 2025

tatiana commented on Oct 15, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

`airflow dags test` needs dag to be serialized #53682