Skip to content

fix(engine): Match a declared Behavior however the model capitalises it - #233

Merged
omesser merged 1 commit into
mainfrom
fix-231-behavior-case
Sep 2, 2026
Merged

omesser merged 1 commit into
mainfrom
fix-231-behavior-case

Conversation

@omesser

@omesser omesser commented Sep 2, 2026

Copy link
Copy Markdown
Owner

Closes #231. ModelDirector::knows compared a proposed Behavior name to the declared ones with ==, and models write the name at the start of a line, so they capitalise it: every local model measured in #175 answered Prowl where the manifest declares prowl. The reply was refused, as_speech caught the whole thing, and it reached the user as dialogue with the Behavior name stuck on the front. The buddy talked and never acted.

What changed

declared(name) replaces knows(name): it compares without case and returns the Character's own spelling, which the proposal is then rebuilt with. Case-insensitive matching alone would not have been enough — the Engine looks a Behavior up by its declared name, so a proposal still carrying Prowl would have found nothing downstream. The test asserts the declared spelling, not merely that the reply was accepted.

say beside it was already matched with eq_ignore_ascii_case; this is the same rule for the name next to it, which is why this reads as removing an inconsistency rather than loosening a check. The duplicated speech-or-fail fallback is extracted into spoken_or_failed.

Measured, not just tested

The same 40-wake harness from #230, against the same two local models, before and after:

Model accepted before accepted after still genuine prose
gemma4:latest 8B 0% 92% 5%
llama3.2:1b 2% 78% 22%

The case-only bucket is empty in both runs. What remains is the models' real contract-break rate, unchanged by this PR — which is the number #144 should argue from.

It does not invent Behaviors

The looser comparison only ever matches something a Character declared. Pinned by a second test using Check, the exact invented name llama3.2:1b produced during the measurement, and visible in the runs above: gemma4's remaining prose reply was Sulks and the 1B's were delight and watch — none of them declared by cat, all correctly spoken rather than played.

Verify

  • cargo test --all-targets (514) and node --test tests/ (26) green.
  • a_declared_behavior_is_known_however_the_model_capitalises_it was written first and observed red against main.
  • Re-take the live numbers with test: Measure the reply-contract rate on local models, and what it exposed #230's harness: AI_BUDDY_DIRECTOR_BASE_URL=http://localhost:11434 AI_BUDDY_DIRECTOR_MODEL=gemma4 cargo test -p ai-buddy measure_the_reply_contract -- --ignored --nocapture

Once this and #230 are both in, the README's "contract kept" column is the accepted column, and that paragraph wants a sentence saying so.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@omesser
omesser merged commit 375ca58 into main Sep 2, 2026
2 checks passed
omesser added a commit that referenced this pull request Sep 2, 2026
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
omesser added a commit that referenced this pull request Sep 2, 2026
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A Behavior name in the wrong case is thrown away, so local models never act

1 participant