Skip to content

fix(buffs): preserve attempt attributes and isolate mutable state in _derive_new_attempt - #2198

Merged
jmartin-tech merged 1 commit into
NVIDIA:mainfrom
bodapatisaikrishna:fix/buff-preserve-intent-attributes
Oct 7, 2026
Merged

jmartin-tech merged 1 commit into
NVIDIA:mainfrom
bodapatisaikrishna:fix/buff-preserve-intent-attributes

Conversation

@bodapatisaikrishna

Copy link
Copy Markdown
Contributor

Description

When deriving new attempts in Buff._derive_new_attempt(), reverse_translation_outputs and intent were omitted from the Attempt(...) constructor call. As a result:

  1. Probes that set intent (such as IntentProbe, TAPIntent, GrandmaIntent, DirectIntent, DoNotAnswer, etc.) lost their intent metadata when processed by buffs (it reverted to None).
  2. Multilingual probes tracking reverse_translation_outputs lost their translation output mapping on derived attempts.
  3. Downstream components (such as intent-aware detectors and report generators) could not access the attempt's original intent.

In addition, notes, probe_params, targets, and detector_results were previously passed by reference directly into the new Attempt. When Buff._derive_new_attempt set new_attempt.notes["buff_creator"] = self.__class__.__name__, it mutated the dictionary belonging to source_attempt.notes.

Changes

  • Propagate intent and deepcopy reverse_translation_outputs in Buff._derive_new_attempt().
  • Deepcopy mutable structures (notes, probe_params, targets, detector_results) to ensure complete isolation between source and derived attempts.
  • Add regression tests in tests/buffs/test_buffs.py verifying that:
    • intent and reverse_translation_outputs are preserved.
    • Modifying derived attempt attributes does not mutate the source attempt.

Verification

  • Run pytest tests/buffs/test_buffs.py and ensure tests pass:
    pytest tests/buffs/test_buffs.py -k test_derive_new_attempt_preserves_attributes
    Result: 1 passed.
  • Run linter and formatting checks:
    ruff check garak/buffs/base.py tests/buffs/test_buffs.py
    ruff format --check garak/buffs/base.py tests/buffs/test_buffs.py
    Result: All checks passed.
  • Verify the thing does what it should: derived.intent matches orig.intent, derived.reverse_translation_outputs matches orig.reverse_translation_outputs.
  • Verify the thing does not do what it should not: Mutations to derived attempts do not leak back to source attempts.

@jmartin-tech

Copy link
Copy Markdown
Collaborator

Expands on #2156, these should land together.

@jmartin-tech jmartin-tech self-assigned this Sep 16, 2026
…_derive_new_attempt

When deriving new attempts in Buff._derive_new_attempt, reverse_translation_outputs and intent were omitted from the Attempt constructor call, causing derived attempts (such as from paraphrase or multilingual buffs) to lose their intent metadata and reverse translation outputs.

Additionally, notes, probe_params, targets, and detector_results were passed by reference without copying. When new_attempt.notes['buff_creator'] was set, it mutated source_attempt.notes in-place.

This commit:
- Propagates intent and deepcopies reverse_translation_outputs to derived attempts.
- Deepcopies notes, detector_results, targets, and probe_params to prevent mutating source attempts.
- Adds regression unit tests covering attribute preservation and mutation isolation.

Signed-off-by: bodapatisaikrishna <saikrishnabodapati@gmail.com>
@bodapatisaikrishna
bodapatisaikrishna force-pushed the fix/buff-preserve-intent-attributes branch from 33ddfde to 00a2109 Compare September 17, 2026 02:58
@bodapatisaikrishna

Copy link
Copy Markdown
Contributor Author

Thanks @jmartin-tech! Agreed completely.

PR #2156 covers deep-copying notes, detector_results, targets, and probe_params to fix state sharing (#2155), while this PR completes the propagation of the remaining Attempt attributes (intent and reverse_translation_outputs) so intent-aware probes and multilingual reverse translations survive buff transformations.

I have rebased this branch onto current main. If you land #2156 first, I will happily rebase on top of it, or feel free to merge/squash them together however is most convenient for the team!

@jmartin-tech
jmartin-tech merged commit ae3721b into NVIDIA:main Oct 7, 2026
19 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants