Keep scheduled tasks when serialized Dag is missing

[deepujain]

When a serialized Dag is briefly missing, the scheduler marks every scheduled task for that Dag as failed. Users then have to retry tasks that never ran. This change keeps those tasks scheduled and excludes the Dag from the current scheduling batch, allowing other Dags to use the available slots.

For example, if a Dag has two scheduled tasks and its serialized record is unavailable, both previously became FAILED. With this change, both remain SCHEDULED for a later scheduling attempt.

Validation:

• Earlier validation recorded in this PR: Breeze pytest airflow-core/tests/unit/jobs/test_scheduler_job.py::TestSchedulerJob::test_queued_task_instances_remain_scheduled_with_missing_dag -q (1 passed on the rebased branch), git diff --check, and Ruff lint and formatting checks on both changed Python files.
• Hosted checks on head f81b91cf2316cc18ebb316c8cb3b3bf5b4be9ad2: 43 successful, 34 skipped, and CodeQL neutral; no failed or pending checks.
• This maintenance pass did not rerun local tests. The regression checks that tasks remain scheduled; it does not demonstrate recovery on a later tick or reproduce the multi-scheduler MWAA environment.

Fixes #62050

---

Was generative AI tooling used to co-author this PR?

• Yes (OpenAI Codex (GPT-6))

Generated-by: OpenAI Codex (GPT-6) following the guidelines

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[deepujain]

The lowest-deps CI failure (test_form_parser_error_keeps_its_own_detail, 422 vs 400) traces to the starlette floor on this branch. With starlette>=1.0.1, the urlencoded form parser does not bound the field count, so the 2000-field test body parses cleanly and then fails pydantic validation (422) instead of raising starlette's own 400. Upstream addressed this in #73302 by raising the floor to starlette>=1.3.1, whose FormParser enforces max_fields and reports a 400.

I have cherry-picked #73302 onto this branch so the lowest-deps job resolves the bounded parser. No changes to the PR's own code; CI should go green on the rerun.

---

Drafted-by: Muse Spark (Meta) (no human review before posting)

[deepujain]

Rebased the missing-serialized-Dag fix onto current main. The scheduled-task regression passes in Breeze, with Ruff lint and formatting checks passing. CI is running on the updated head.

---

Drafted-by: OpenAI Codex (no human review before posting)

[deepujain]

Rebased onto main at b6538c3. The scheduled-task regression for a missing serialized Dag passes in Breeze, with clean Ruff lint and formatting checks. Fresh hosted CI is starting on the updated head.

---

Drafted-by: OpenAI Codex (no human review before posting)

[deepujain]

@ashb @XD-DENG Checks are green on the latest commit. Could you review this when you have a chance?