Skip to content

Extend DEFAULT_SENSITIVE_FIELDS with webhook_url, bearer, dsn, auth_header, service_key - #66673

Merged
vatsrahul1001 merged 1 commit into
apache:mainfrom
potiuk:extend-default-sensitive-fields
May 15, 2026
Merged

vatsrahul1001 merged 1 commit into
apache:mainfrom
potiuk:extend-default-sensitive-fields

Conversation

@potiuk

@potiuk potiuk commented May 10, 2026

Copy link
Copy Markdown
Member

DEFAULT_SENSITIVE_FIELDS is the allowlist used by the secrets masker for
masking Variables and Connection extras. Several common field names used by
official Airflow providers and standard HTTP/database configurations are
not in the allowlist.

This PR adds five field names commonly used in connection extras and
provider configurations:

  • webhook_url — Slack provider webhook URL key
  • bearer — HTTP bearer-token auth key
  • dsn — database connection strings (which typically embed credentials, e.g. postgres://user:pass@host/db)
  • auth_header — custom HTTP auth header values
  • service_key — service-account-like keys

The matcher uses case-insensitive substring matching, so e.g. bearer covers
Bearer, bearer_token, auth_bearer, etc.

Related: https://github.com/airflow-s/airflow-s/issues/377

Test plan

  • uv run --project shared/secrets_masker pytest shared/secrets_masker/tests/ -xvs — secrets-masker tests pass
  • New parametrised test asserts should_hide_value_for_key(name) returns True for each of the five new field names plus substring variants (WEBHOOK_URL, slack_webhook_url, auth_bearer, AUTH_HEADER, custom_auth_header, my_service_key)
Was generative AI tooling used to co-author this PR?
  • Yes — Claude Opus 4.7 (1M context)

Generated-by: Claude Opus 4.7 (1M context) following the guidelines at
https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions

DEFAULT_SENSITIVE_FIELDS is the allowlist used by the secrets masker
for masking Variables and Connection extras. Several common field
names used by official Airflow providers and standard HTTP/database
configurations are not in the allowlist.

This commit adds five field names commonly used in connection extras
and provider configurations:

- webhook_url — Slack provider webhook URL key
- bearer      — HTTP bearer-token auth key
- dsn         — database connection strings (which typically embed
                credentials, e.g. postgres://user:pass@host/db)
- auth_header — custom HTTP auth header values
- service_key — service-account-like keys

Related: airflow-s/airflow-s#377

Generated-by: Claude Opus 4.7 (1M context) following the guidelines at
https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions
@potiuk
potiuk requested review from amoghrajesh and ashb as code owners May 10, 2026 22:11
@potiuk potiuk added this to the Airflow 3.2.2 milestone May 10, 2026
@vatsrahul1001 vatsrahul1001 added the ready for maintainer review Set by triage when all criteria pass, or by the fork CI reconciler when fork CI is green. label May 12, 2026

@amoghrajesh amoghrajesh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this.

One thing worth considering is that providers already declare which extra fields are sensitive via "format": "password" in provider.yaml conn-fieldsand ProvidersManager already tracks this as is_sensitive=True on each ConnectionFormWidgetInfo. If we can get the secrets masker to read that and add it in, this whole process could be automated.

That way any provider that correctly declares a password field gets it masked automatically, without needing manual additions to the core allowlist.

@vatsrahul1001

Copy link
Copy Markdown
Contributor

Thanks for adding this.

One thing worth considering is that providers already declare which extra fields are sensitive via "format": "password" in provider.yaml conn-fieldsand ProvidersManager already tracks this as is_sensitive=True on each ConnectionFormWidgetInfo. If we can get the secrets masker to read that and add it in, this whole process could be automated.

That way any provider that correctly declares a password field gets it masked automatically, without needing manual additions to the core allowlist.

This can be in follow PR right @amoghrajesh ?

@amoghrajesh amoghrajesh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally, rest for follow up

@vatsrahul1001
vatsrahul1001 merged commit 32ac8ad into apache:main May 15, 2026
78 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-2-test

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
✅ v3-2-test PR Link

github-actions Bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request May 15, 2026
…eld names (apache#66673)

DEFAULT_SENSITIVE_FIELDS is the allowlist used by the secrets masker
for masking Variables and Connection extras. Several common field
names used by official Airflow providers and standard HTTP/database
configurations are not in the allowlist.

This commit adds five field names commonly used in connection extras
and provider configurations:

- webhook_url — Slack provider webhook URL key
- bearer      — HTTP bearer-token auth key
- dsn         — database connection strings (which typically embed
                credentials, e.g. postgres://user:pass@host/db)
- auth_header — custom HTTP auth header values
- service_key — service-account-like keys

Related: airflow-s/airflow-s#377
(cherry picked from commit 32ac8ad)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Generated-by: Claude Opus 4.7 (1M context) following the guidelines at
https: //github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions
aws-airflow-bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request May 15, 2026
…eld names (apache#66673)

DEFAULT_SENSITIVE_FIELDS is the allowlist used by the secrets masker
for masking Variables and Connection extras. Several common field
names used by official Airflow providers and standard HTTP/database
configurations are not in the allowlist.

This commit adds five field names commonly used in connection extras
and provider configurations:

- webhook_url — Slack provider webhook URL key
- bearer      — HTTP bearer-token auth key
- dsn         — database connection strings (which typically embed
                credentials, e.g. postgres://user:pass@host/db)
- auth_header — custom HTTP auth header values
- service_key — service-account-like keys

Related: airflow-s/airflow-s#377
(cherry picked from commit 32ac8ad)

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
Generated-by: Claude Opus 4.7 (1M context) following the guidelines at
https: //github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions
Nishieee pushed a commit to Nishieee/airflow that referenced this pull request May 15, 2026
…pache#66673)

DEFAULT_SENSITIVE_FIELDS is the allowlist used by the secrets masker
for masking Variables and Connection extras. Several common field
names used by official Airflow providers and standard HTTP/database
configurations are not in the allowlist.

This commit adds five field names commonly used in connection extras
and provider configurations:

- webhook_url — Slack provider webhook URL key
- bearer      — HTTP bearer-token auth key
- dsn         — database connection strings (which typically embed
                credentials, e.g. postgres://user:pass@host/db)
- auth_header — custom HTTP auth header values
- service_key — service-account-like keys

Related: https://github.com/airflow-s/airflow-s/issues/377

Generated-by: Claude Opus 4.7 (1M context) following the guidelines at
https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions
shahar1 pushed a commit that referenced this pull request May 15, 2026
…eld names (#66673) (#66991)

DEFAULT_SENSITIVE_FIELDS is the allowlist used by the secrets masker
for masking Variables and Connection extras. Several common field
names used by official Airflow providers and standard HTTP/database
configurations are not in the allowlist.

This commit adds five field names commonly used in connection extras
and provider configurations:

- webhook_url — Slack provider webhook URL key
- bearer      — HTTP bearer-token auth key
- dsn         — database connection strings (which typically embed
                credentials, e.g. postgres://user:pass@host/db)
- auth_header — custom HTTP auth header values
- service_key — service-account-like keys

Related: https://github.com/airflow-s/airflow-s/issues/377
(cherry picked from commit 32ac8ad)


Generated-by: Claude Opus 4.7 (1M context) following the guidelines at
https: //github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 pushed a commit that referenced this pull request May 20, 2026
…eld names (#66673) (#66991)

DEFAULT_SENSITIVE_FIELDS is the allowlist used by the secrets masker
for masking Variables and Connection extras. Several common field
names used by official Airflow providers and standard HTTP/database
configurations are not in the allowlist.

This commit adds five field names commonly used in connection extras
and provider configurations:

- webhook_url — Slack provider webhook URL key
- bearer      — HTTP bearer-token auth key
- dsn         — database connection strings (which typically embed
                credentials, e.g. postgres://user:pass@host/db)
- auth_header — custom HTTP auth header values
- service_key — service-account-like keys

Related: airflow-s/airflow-s#377
(cherry picked from commit 32ac8ad)


Generated-by: Claude Opus 4.7 (1M context) following the guidelines at
https: //github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 pushed a commit that referenced this pull request May 20, 2026
…eld names (#66673) (#66991)

DEFAULT_SENSITIVE_FIELDS is the allowlist used by the secrets masker
for masking Variables and Connection extras. Several common field
names used by official Airflow providers and standard HTTP/database
configurations are not in the allowlist.

This commit adds five field names commonly used in connection extras
and provider configurations:

- webhook_url — Slack provider webhook URL key
- bearer      — HTTP bearer-token auth key
- dsn         — database connection strings (which typically embed
                credentials, e.g. postgres://user:pass@host/db)
- auth_header — custom HTTP auth header values
- service_key — service-account-like keys

Related: airflow-s/airflow-s#377
(cherry picked from commit 32ac8ad)


Generated-by: Claude Opus 4.7 (1M context) following the guidelines at
https: //github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
vatsrahul1001 pushed a commit that referenced this pull request May 21, 2026
…eld names (#66673) (#66991)

DEFAULT_SENSITIVE_FIELDS is the allowlist used by the secrets masker
for masking Variables and Connection extras. Several common field
names used by official Airflow providers and standard HTTP/database
configurations are not in the allowlist.

This commit adds five field names commonly used in connection extras
and provider configurations:

- webhook_url — Slack provider webhook URL key
- bearer      — HTTP bearer-token auth key
- dsn         — database connection strings (which typically embed
                credentials, e.g. postgres://user:pass@host/db)
- auth_header — custom HTTP auth header values
- service_key — service-account-like keys

Related: airflow-s/airflow-s#377
(cherry picked from commit 32ac8ad)


Generated-by: Claude Opus 4.7 (1M context) following the guidelines at
https: //github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions

Co-authored-by: Jarek Potiuk <jarek@potiuk.com>
sehaj-sandhu added a commit to acceldata-io/airflow that referenced this pull request Jun 30, 2026
Backport of apache#66673 (32ac8ad),
upstream fixed in 3.2.2.

Manually adapted to 2.8.x: set lives in airflow/utils/log/secrets_masker.py
(pre task-sdk layout); service_account already present, connection_string is a
pre-existing upstream entry not part of this PR.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready for maintainer review Set by triage when all criteria pass, or by the fork CI reconciler when fork CI is green.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants