Repository navigation
Conversation
Every task log line passes through the secrets masker and the JWT redactor in the supervisor, and the JWT redactor also runs in the task process. The masker re-normalised every key of every line with several regex substitutions and compared it against all sensitive field names, even though the answer only depends on the key name and the configured fields. The JWT redactor ran its regex on every string value, including the vast majority that cannot contain a token. With a high log volume the supervisor becomes the bottleneck and the task blocks while writing its logs, so this cost shows up directly in task duration.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every task log line goes through
SecretsMasker.should_hide_value_for_key, once per key (dag_id,ti_id,logger, ...), and #74840 made it heavier:The answer only depends on the name and the field contents, so it moves into a module-level
lru_cachekeyed by(name, tuple(sensitive_variables_fields))— reassigning or mutating the fields can't return a stale answer, and the cache is bounded because variable and connection-extra keys come from user data.redact_jwtran its regex on every string value; strings withouteyJnow skip it.In this logging-heavy workload, processing logs in the supervisor creates backpressure on the task, so reducing masking overhead reduces task duration. LocalExecutor, a task writing 100k lines, Linux x86_64; image
apache/airflow:3.4.0b1with these two files replaced by the versions from main and from this branch. Median of 5 fresh containers per side (each the mean of 2 runs), the two sides run back to back in randomised order:logging.infoprintEach of the 5 back-to-back pairs gave a ratio of 0.50–0.54 (Bash), 0.53–0.55 (
logging.info) and 0.52–0.60 (print).Tests. Two new tests cover the cache and the JWT shortcut; both fail if the source change is reverted. Comparing old and new code on 120k key lookups (including in-place edits of the field list) and 100k log events gave identical masking results. An unusual compatibility edge remains: an unhashable
strsubclass used as a name, or an unhashable sensitive-field entry, now raisesTypeErrorduring cache lookup.redactcatches this and returns<redaction-failed>. Airflow's configuration supplies ordinary strings.Local:
shared/secrets_masker174 passed (1 xfailed),shared/logging111 passed;test_http_access_log,test_decoratorsand the OpenLineage redactor tests pass; prek clean. In the four task-sdk modules I ran (test_log,test_operator,test_supervisor,test_task_runner), 1075 pass and 11 fail (secrets-backend count andTriggerDagRuncall assertions); the same 11 failures also occur on the base commit.related: #71721
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 5.5) following the guidelines
Used to profile the task log path, write the change and its tests. I reviewed every line and re-ran the benchmarks and tests myself.
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.