Skip to content

Add KubernetesPodOperator's name in templated fields #43480

Description

@anteverse

Description

Unless it's been kept out templated fields on purpose, I would like to add the param name of KubernetesPodOperator into template_fields.

My guess is that we wished to protect the pod name and guarantee the right syntax via KubernetesPodOperator._set_name. But I think we could have the best of both worlds by adding name to template_fields and have the protected name through something like self.pod_name = self._set_pod_name(name), or equivalent.

Would that be worth a shot? I'm willing to submit a short PR about this.

Thanks a lot!

Use case/motivation

The current use-case is when we explore pods from the Kubernetes side, it's easy to have pod_name=task_id but that's not sufficient when we have several DAGs with identical tasks in them.
Discriminant and dynamic information can be passed to labels or annotations since they're templated fields, but the pod name isn't.

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Activity

  1. potiuk commented on Oct 29, 2024

    @potiuk
    Member

    Sure. Go ahead.

  2. insomnes commented on Jan 30, 2025

    @insomnes
    Contributor

    @eilon246810 hi! I am very interested in this feature, so I have created my PR based on your previous one (with some extra tests).

    Let me know if you'd like to recreate it yourself.

  3. insomnes commented on Jan 31, 2025

    @insomnes
    Contributor

    Fixing the problem with tests led me to the bug in SparkKubernetesOperator tests, so I have fixed them in addition to this feature too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions