AI Guard: redaction tests for the full provider-bound context (multi-turn history) - #7595
Merged
cbeauchesne merged 5 commits intoAug 28, 2026
Merged
Conversation
…message APPSEC-68867 Backend RFC v1.1 and RFC v1.2 make the scope distinction normative: attack analysis targets the latest logical message, but SDS scans every model-visible string in the messages array of the current /evaluate call, so redaction_replacements may target the system prompt and historical user, assistant and tool messages as well as the latest one. Only REDACT_MIXED redacted a non-last message alongside the latest, so a tracer that redacted only the latest message passed 22 of the 24 existing scenarios while still shipping the whole history to the provider on turn 2. Adds Test_RedactionMultiTurnContext and nine generator scenarios: the RFC multi-turn example (historical SSN plus a new email, with an already redacted assistant message that must survive byte for byte), history-only redaction with a benign latest message, one replacement per role in a single call, historical tool call/result and content part, non-contiguous replacements in an eight-message conversation, a value restated across turns, and request-local paths replayed over three calls that grow then reorder the conversation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
|
|
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c0b0b6518b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
smola
approved these changes
Aug 27, 2026
Consolidate assertion logic duplicated across the redaction tests, with no change to what any test asserts. - Extract _assert_sdk_response_redacted: Test_RedactedMessagesInSDKResponse and Test_RedactionMultiTurnContext carried a byte-for-byte copy of the same "messages are the redacted ones and hold no sensitive value" block. - Extract _assert_tool_arguments_still_parse, shared by the two tests that check a redacted tool call keeps parseable arguments. It selects the .arguments replacement from the corpus, replacing a hardcoded path literal in test_redact_every_role_in_history that duplicated corpus data and would have drifted silently if the scenario were reindexed. - Dedupe the declared sensitive values in the generator. removed/retained are substring searches over the whole payload, so a value declared at two paths only repeated the identical assertion; REDACT_SAME_VALUE_ACROSS_TURNS was the only scenario affected. Cassettes are byte-identical and the test-ID inventory is unchanged, so no manifest entry is affected.
…-write claim Address review feedback on #7595. Historical assistant content was the one model-visible surface the corpus never redacted: the only assistant content replacement was REDACT_ASSISTANT_RESPONSE, where the assistant reply is the latest message. A tracer redacting the latest assistant output and the other historical roles but skipping historical assistant content passed the whole suite while still leaking it to the provider. The historical assistant message in REDACT_EVERY_ROLE_IN_HISTORY now carries content with its own SSN alongside its tool_calls, so one message exercises both surfaces. Scope the class docstring to what the tests actually assert. Every weblog deserializes a fresh message list per request and returns the evaluation rather than the caller's list, so an SDK mutating that list in place is invisible here. The suite verifies what the tracer sends and reports per call; the non-mutation half of copy-on-write would need a weblog endpoint holding one list across turns and echoing the original back. Drop the ai_guard.md subsection: it restated the class docstring and enumerated the class's test methods, which the docstrings already carry in context. The file is now identical to main.
avara1986
enabled auto-merge (squash)
August 27, 2026 15:40
cbeauchesne
disabled auto-merge
August 28, 2026 13:30
cbeauchesne
deleted the
avara/APPSEC-68867-ai-guard-redaction-multi-turn
branch
August 28, 2026 13:30
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Jira: APPSEC-68867
RFCs: RFC AI Guard Sensitive Data Redaction (v1.2) and RFC AI Guard Sensitive Data Redaction. Backend (v1.1)
Follow-up to #7385, which introduced the redaction corpus.
Both RFCs were revised to make one point normative:
This matters because redaction is copy-on-write: the tracer sends a redacted copy to the provider and leaves the caller's list alone, so an earlier message still holds its original value on the next turn.
The gap
Of the 24 redaction scenarios on
main, onlyREDACT_MIXEDredacted a non-last message alongside the latest one, andREDACT_SYSTEM_PROMPTredacted a two-message system prompt. A tracer that redacted only the latest message passed 22 of 24 and would still ship the entire conversation history to the provider on turn 2 — the exact failure the RFC revision was written to prevent.What this PR adds
Test_RedactionMultiTurnContext(9 tests) and 9 scenarios ingen_redaction_cassettes.py:REDACT_TURN_1+REDACT_HISTORY_AND_LATESTmessages[1]plus a new email atmessages[3], with the already-redacted assistant message atmessages[2]surviving byte for byte (no double redaction)REDACT_HISTORY_ONLYREDACT_EVERY_ROLE_IN_HISTORYREDACT_HISTORICAL_TOOL_CALLREDACT_HISTORY_CONTENT_PARTmessages[1].content[0].textin a historical multimodal messageREDACT_DEEP_HISTORYREDACT_SAME_VALUE_ACROSS_TURNSREDACT_REORDERED_CONTEXTtest_redact_paths_are_request_local, which replays turn 1 → turn 2 → reordered turn 2 across three calls. The reorder swaps the strings behindmessages[1]/messages[3], so a tracer reusing the previous response's paths writes the SSN replacement over the email message and failsPlus
test_redacted_history_in_sdk_response, asserting the list handed back to the caller — and therefore sent to the provider — has no history left. That is the guarantee redaction exists for.Keeping fixtures in sync
Unchanged from #7385: scenarios live in
SCENARIOSinutils/scripts/gen_redaction_cassettes.py, which writes both the cassettes andtests/ai_guard/redaction_scenarios.json, and cross-checks every declaredexpect_redacted/expect_removedagainst its reference implementation of the RFC algorithm../format.shruns it, so the fixtures in the repo are always the fixtures the script produces.Test results
./run.sh AI_GUARD tests/ai_guard/test_ai_guard_sdk.py::Test_RedactionMultiTurnContext→ 9 passed againstpython@4.14.0-rc1. Python'sredact_messagesis index- and cardinality-agnostic, so it passes unchanged — the new tests document a contract it already satisfies rather than asking for new work.Test_RedactionInSDKResponsegap).TEST_THE_TEST→ 437 passed../format.sh→ mypy, ruff, yamlfmt, yamllint, parser checks and the redaction-fixture drift check all clean.Notes for reviewers
Test_RedactionMultiTurnContextis activated for python (flask-poc: v4.14.0-dev) and declared at the existing redaction ref for nodejs (express4: *ref_6_13_0);missing_featurefor cpp_httpd, cpp_nginx, dotnet, golang, java, php, ruby.Test_Redactionon purpose: full-context scope is its own normative concern, so a tracer that implements last-message redaction first can be declared separately.docs/understand/scenarios/ai_guard.mdgains a subsection explaining the scope distinction and why the class exists.🤖 Generated with Claude Code