Skip to content

Helm chart does not support multiple executors #43224

Closed

Description

@rcheatham-q

Official Helm Chart version

1.15.0 (latest released)

Apache Airflow version

2.10.2

Kubernetes Version

1.30.4

Helm Chart configuration

executor: CeleryExecutor,KubernetesExecutor

Docker Image customizations

n/a

What happened

First, the JSON schema specification failed as it has an allowlist of all known supported executors. Then, when I skipped schema verification, the deployment failed because the executor value is used as a label on at least one deployment, and commas are not allowed in label values.

❯ echo 'executor: CeleryExecutor,KubernetesExecutor' > values.yaml
❯ helm upgrade --install airflow apache-airflow/airflow --values values.yaml                                
Error: UPGRADE FAILED: values don't meet the specifications of the schema(s) in the following chart(s):
airflow:
- executor: executor must be one of the following: "LocalExecutor", "LocalKubernetesExecutor", "CeleryExecutor", "KubernetesExecutor", "CeleryKubernetesExecutor", "airflow.providers.amazon.aws.executors.batch.AwsBatchExecutor", "airflow.providers.amazon.aws.executors.ecs.AwsEcsExecutor"

❯ helm upgrade --install airflow apache-airflow/airflow --values values.yaml --skip-schema-validation
Release "airflow" does not exist. Installing it now.
Error: 1 error occurred:
	* Deployment.apps "airflow-scheduler" is invalid: metadata.labels: Invalid value: "CeleryExecutor,KubernetesExecutor": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue',  or 'my_value',  or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')

What you think should happen instead

The helm chart should deploy successfully when multiple executors are specified as described in the docs.

How to reproduce

echo 'executor: CeleryExecutor,KubernetesExecutor' > values.yaml
helm upgrade --install airflow apache-airflow/airflow --values values.yaml

and

helm upgrade --install airflow apache-airflow/airflow --values values.yaml --skip-schema-validation

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Activity

  1. potiuk commented on Oct 24, 2024

    @potiuk
    Member

    Sure this is good feature to add. If you would liek to do @rcheatham-q - you can add it, otherwise it will have to wait for someone to implement it.

  2. potiuk commented on Oct 24, 2024

    @potiuk
    Member

    Shall I assign you @rcheatham-q ?

  3. rcheatham-q commented on Oct 24, 2024

    @rcheatham-q
    Author

    I don't have time to do this right now, but if the opportunity presents itself I'm willing to take this on.

  4. amoghrajesh commented on Nov 1, 2024

    @amoghrajesh
    Contributor

    @rcheatham-q do you have some time to work on this? Otherwise I can pick this up

  5. rcheatham-q commented on Nov 2, 2024

    @rcheatham-q
    Author

    @amoghrajesh I do not have time to work on this

  6. jx2lee commented on Nov 2, 2024

    @jx2lee
    Contributor

    Can I take this issue?

  7. amoghrajesh commented on Nov 2, 2024

    @amoghrajesh
    Contributor

    @jx2lee feel free. Assigning to you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions