Skip to content

Show a clean error when airflow backfill create targets an unknown Dag - #72893

Closed
Eason09053360 wants to merge 1 commit into
apache:mainfrom
Eason09053360:fix-backfill-create-unknown-dag-traceback
Closed

Eason09053360 wants to merge 1 commit into
apache:mainfrom
Eason09053360:fix-backfill-create-unknown-dag-traceback

Conversation

@Eason09053360

@Eason09053360 Eason09053360 commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

airflow backfill create --dag-id <unknown> now prints a one-line error and exits 1 instead of dumping a DagNotFound traceback, in both the normal and --dry-run paths.

Why

Mistyping the Dag id is the most ordinary user error this command can see, and today it looks like a crash:

$ airflow backfill create --dag-id does_not_exist --from-date 2025-01-01 --to-date 2025-01-02
Traceback (most recent call last):
  ...
  File ".../airflow/models/backfill.py", line 641, in _create_backfill
    raise DagNotFound(f"Could not find dag {dag_id}")
airflow.exceptions.DagNotFound: Could not find dag does_not_exist

The model layer raises DagNotFound on purpose (it serves both the REST API and the CLI), and the REST route already maps it to a 404. The CLI handler only caught NoBackfillRunsToCreate, so this exception escaped untranslated. Every other CLI "Dag does not exist" path (dags list-jobs, dags list-runs, dags state, ...) already reports it as a plain message on stderr.

The --dry-run path had a second wrinkle: _do_dry_run is a generator, so its body (including the Dag lookup that raises) only ran when the result was iterated, which happened after the create_session() block had exited and outside any try.

What

  • Catch DagNotFound at the CLI boundary and re-raise it as SystemExit(str(e)), which writes the message to stderr and exits 1, the same idiom the other CLI "not found" paths use. The exit code is unchanged from the traceback case, so scripts checking $? see no difference.
  • In the dry-run branch, consume _do_dry_run(...) with list(...) inside the session block and the try. Without this the new handler would be unreachable, because the exception surfaces on iteration, not on the call. As a side effect the generator now runs while the session it was given is still open; previously it ran after session.close().

Deliberately out of scope: the sibling validation errors from the same two model calls (AlreadyRunningBackfill, DagNonPeriodicScheduleException, DagRunTypeNotAllowed, InvalidBackfillDateRange, ...) still surface as tracebacks. They share the root cause but each needs its own message and test; this PR only fixes the missing-Dag case. Happy to follow up with the rest if reviewers prefer one PR.


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Fable 5.1)

Generated-by: Claude Code (Fable 5.1) following the guidelines

The command let DagNotFound escape from the model layer, so a mistyped
--dag-id produced a full traceback rather than a one-line message. In the
--dry-run path the exception surfaced even later, because _do_dry_run is a
generator whose body only runs on iteration, after the session block had
already exited. Catching it at the CLI boundary and consuming the generator
inside that block reports the missing Dag the same way the other CLI
"not found" paths do, with the exit code unchanged.
@potiuk

potiuk commented Sep 25, 2026

Copy link
Copy Markdown
Member

Hello @Eason09053360 - thank you for your contributions to Apache Airflow!

The Airflow community has introduced a limit of 5 open pull requests at a time for contributors without write access to the repository. You currently have 33 open pull requests, so - as a one-time step of introducing the limit - we closed the ones where maintainers have not engaged yet:

These pull requests stay open because maintainers are already engaged in them - they count towards your limit:

This is not a judgement of you or of your changes. We never told contributors before that opening many pull requests at once was a problem, so there is nothing to feel bad about - and nothing is lost: your branches, commits and the review history stay where they are.

What we ask you to do is to make your first prioritization decision: choose which of the pull requests above matter most to you, and reopen them (up to 5 open at a time, including the ones still open) with the "Reopen pull request" button or gh pr reopen <PR_NUMBER> --repo apache/airflow. Reopen the ones you are ready to follow through - keep them rebased, respond to review comments and fix failing checks.

While your pull requests are waiting for review, the most valuable thing you can do is help in other ways - reviewing other contributors' pull requests, helping with issues, and taking part in the discussions on the devlist and Slack.

Why we introduced the limit, what it means for you and how to reopen or restore a pull request is explained in https://github.com/apache/airflow/blob/main/contributing-docs/32_open_pull_request_limit.rst.


Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:CLI closed because of open PR limit Closed as a one-time step of introducing the open pull request limit

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants