Skip to content

Support code packages in MWAA Serverless workflow operators - #72622

Merged
o-nikolas merged 2 commits into
apache:mainfrom
yamakazushi:mwaa-serverless-code-package
Sep 15, 2026
Merged

o-nikolas merged 2 commits into
apache:mainfrom
yamakazushi:mwaa-serverless-code-package

Conversation

@yamakazushi

@yamakazushi yamakazushi commented Sep 7, 2026 •

Copy link
Copy Markdown
Contributor

MWAA Serverless added PythonOperator and BashOperator support, which requires shipping the Python modules and shell scripts those tasks run separately from the YAML workflow definition, through the Code request parameter.

Neither MwaaServerlessCreateWorkflowOperator nor
MwaaServerlessUpdateWorkflowOperator exposed Code, and neither accepts a kwargs passthrough, so the parameter was unreachable. Workflows using those operators could not be created, and their code could not be updated, from Airflow.

Add a code parameter to both operators, named after the API field and passed through untouched so a future member of the Code union needs no operator change. prune_dict omits it when unset, so existing DAGs are unaffected.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

kiro


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

MWAA Serverless added PythonOperator and BashOperator support, which requires
shipping the Python modules and shell scripts those tasks run separately from
the YAML workflow definition, through the Code request parameter.

Neither MwaaServerlessCreateWorkflowOperator nor
MwaaServerlessUpdateWorkflowOperator exposed Code, and neither accepts a kwargs
passthrough, so the parameter was unreachable. Workflows using those operators
could not be created, and their code could not be updated, from Airflow.

Add a `code` parameter to both operators, named after the API field and passed
through untouched so a future member of the Code union needs no operator
change. prune_dict omits it when unset, so existing DAGs are unaffected.
@boring-cyborg boring-cyborg Bot added area:providers provider:amazon AWS/Amazon - related issues labels Sep 7, 2026

@o-nikolas o-nikolas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is nice good, short and to the point. Though there are still other params that these operators don't forward through to the API call, and adding more of this handling/testing seems not worth it. I'm okay with merging this one, but if you want to do a follow-up @yamakazushi adding a create_workflow_kwargs and update_workflow_kwargs where the lesser used inputs can be passed through (and any future additions) would be great.

@o-nikolas

Copy link
Copy Markdown
Contributor

CC @kars0508

@yamakazushi

Copy link
Copy Markdown
Contributor Author

o-nikolas, thank you for reviewing my commits. As you mentioned, it is good that other parameters will be added but this commit is my first commits excluding documentation fix. Therefore, I would like to focus on adding code parameter in this commit, and I will commit to add other parameters.

@o-nikolas

Copy link
Copy Markdown
Contributor

o-nikolas, thank you for reviewing my commits. As you mentioned, it is good that other parameters will be added but this commit is my first commits excluding documentation fix. Therefore, I would like to focus on adding code parameter in this commit, and I will commit to add other parameters.

Sounds good, I'm looking forward to the follow-up 😃

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants