Skip to content

Drop the cncf.kubernetes provider dependency from airflow-core tests - #71868

Closed
rjgoyln wants to merge 5 commits into
apache:mainfrom
rjgoyln:core-tests-drop-cncf-kubernetes
Closed

rjgoyln wants to merge 5 commits into
apache:mainfrom
rjgoyln:core-tests-drop-cncf-kubernetes

Conversation

@rjgoyln

@rjgoyln rjgoyln commented Aug 20, 2026 •

Copy link
Copy Markdown
Contributor

Summary

airflow-core cannot currently run its own test suite in the core-only environment created by uv sync --project airflow-core. Four modules import the Kubernetes client or the cncf.kubernetes provider at module scope, causing collection failures in unit/cli and unit/serialization. Several executor, CLI, and logging tests also require the provider to run.

This is the second slice of #71641, following #71677.

The tests are now split according to their actual dependency:

  • Core's executor_config and ExecutorConfigType tests use kubernetes.client directly and now skip when the client is unavailable.
  • Tests that exercise the cncf.kubernetes provider skip when the provider is unavailable.
  • unit/core/test_exceptions.py only verified that the provider re-exported two core exceptions, so it moves to the provider test suite.

pod_override is also removed from simple_dag. The pod encoding is independent of the serialized DAG version, so the existing v1/v2 fixtures provided no additional coverage beyond the new round-trip test.

A core-only test run is not fully green yet. test_providers_manager.py::test_cli and TestStringifiedDAGs::test_serialization depend on the full provider set and are left for #71637 and the test_example_dags.py slice. The remaining pyproject.toml dev-group dependencies will be removed in subsequent slices.

Related: #71641

Tests

With cncf-kubernetes and the kubernetes client removed from a uv sync --project airflow-core environment, the collection errors are gone and Kubernetes-dependent tests now skip with the appropriate reason. The same test files pass unchanged in a full workspace.


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 5)

Generated-by: Claude Code (Opus 5) following the guidelines

A scoped `uv sync --project airflow-core` is meant to give a core-only
environment, but airflow-core's own test suite cannot run in one: four modules
reach for the Kubernetes client or the cncf.kubernetes provider at module
scope, so whole directories fail to collect, and a dozen executor, CLI and
logging tests need the provider installed before they can pass.

Core's handling of Kubernetes pod objects is its own — `executor_config`
serialization and `ExecutorConfigType` reach for `kubernetes.client` lazily and
never for the provider — so the tests covering it now skip when that client
library is absent instead of requiring it. The two tests that only checked the
provider's exception re-exports belong with the provider, where the pairing
they assert can actually break.

Second slice of apache#71641.
@boring-cyborg boring-cyborg Bot added area:CLI area:dag-processor area:Executors-core LocalExecutor & SequentialExecutor area:providers provider:cncf-kubernetes Kubernetes (k8s) provider related issues labels Aug 20, 2026
rjgoyln and others added 2 commits August 22, 2026 18:29
The provider suite runs against released Airflow versions too, and
`airflow.exceptions` only re-exports the provider's pod exceptions from 3.0
onwards, so the pairing this test asserts does not hold on 2.x.
@rjgoyln
rjgoyln marked this pull request as ready for review August 23, 2026 13:10
@eladkal
eladkal requested a review from potiuk August 24, 2026 22:02
@rjgoyln rjgoyln changed the title Drop the cncf.kubernetes dependency from airflow-core tests Drop the cncf.kubernetes provider dependency from airflow-core tests Sep 1, 2026
@potiuk

potiuk commented Sep 25, 2026

Copy link
Copy Markdown
Member

Hello @rjgoyln - thank you for your contributions to Apache Airflow!

The Airflow community has introduced a limit of 5 open pull requests at a time for contributors without write access to the repository. You currently have 24 open pull requests, so - as a one-time step of introducing the limit - we closed the ones where maintainers have not engaged yet:

These pull requests stay open because maintainers are already engaged in them - they count towards your limit:

This is not a judgement of you or of your changes. We never told contributors before that opening many pull requests at once was a problem, so there is nothing to feel bad about - and nothing is lost: your branches, commits and the review history stay where they are.

What we ask you to do is to make your first prioritization decision: choose which of the pull requests above matter most to you, and reopen them (up to 5 open at a time, including the ones still open) with the "Reopen pull request" button or gh pr reopen <PR_NUMBER> --repo apache/airflow. Reopen the ones you are ready to follow through - keep them rebased, respond to review comments and fix failing checks.

While your pull requests are waiting for review, the most valuable thing you can do is help in other ways - reviewing other contributors' pull requests, helping with issues, and taking part in the discussions on the devlist and Slack.

Why we introduced the limit, what it means for you and how to reopen or restore a pull request is explained in https://github.com/apache/airflow/blob/main/contributing-docs/32_open_pull_request_limit.rst.


Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:CLI area:dag-processor area:Executors-core LocalExecutor & SequentialExecutor area:providers closed because of open PR limit Closed as a one-time step of introducing the open pull request limit provider:cncf-kubernetes Kubernetes (k8s) provider related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants