Support logs pulling in KubernetesJobOperator #40252

Description

In cncf-kubernetes provider, KubernetesPodOperator provides get_logs and log_containers arguments to support pulling logs generated from created Kubernetes pod into task logs, and these arguments are also available in KubernetesJobOperator through class inheritance.

However, KubernetesJobOperator does not implement logs pulling mechanism, so user has no way to transfer pod logs from the created job to task logs within this operator.

Use case/motivation

Supports get_logs and log_containers settings in KubernetesJobOperator when wait_until_job_complete is enabled (otherwise it's user's responsibility to manage logs/output generated from the job).

We can take similar approach used in KubernetesPodOperator to regularly fetch pod logs under the created job.

Related issues

• KubernetesJobOperator #17490
• Create GKEStartJobOperator and KubernetesJobOperator #36847

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct