Description
Why
Two of the three ways you can click the buddy do nothing you can see or hear.
| Gesture | Verb | What it does today |
|---|---|---|
| Left click | Poke |
Plays the react Required Animation (engine.rs:831) |
| Double click | Poke + Summon |
Wakes a sleeping sprite. Nothing else |
| Right click | Menu |
Wakes a sleeping sprite. Nothing else |
| Pickup | Grab |
The sprite follows the cursor. No announcement |
| Drop | none | Nothing at all — see below, there is no drop verb |
engine.rs:2232 asserts that silence on purpose:
a_summon_or_a_menu_wakes_the_sprite_without_moving_it ends with
assert_eq!(addressed.animation, "idle", "and nothing is played for it"). Both
verbs are accepted and shown nothing, because the surfaces they open — the chat
surface (#17) and the tray menu (#18) — were the response.
So a double-click that lands is indistinguishable from one that missed the
sprite. There is no audio anywhere in the tree: git grep -i "audio\|sound"
over *.rs returns nothing but the word "sound" in a comment. The buddy is
silent, and most of the ways you can touch it are also invisible.
Each interaction should announce itself — its own visual cue and its own sound,
so the gestures are told apart by feel rather than by their consequences.
Why procedural, and not three new animations
This cannot be Character art, and that is a constraint rather than a preference.
ADR-0002 fixed the Required Animation Set at eight because "every required
animation is a tax on every Character that will ever exist". engine.rs:2226
records the same reasoning for the verb set: five verbs, so "no Character ever
has to grow another Animation for a sixth". Three per-interaction cues authored
as art would be six to nine new required frames per Character, retroactively,
for every package that already exists.
Procedural is the way out. The Shell draws and synthesizes the cue; no Character
Package changes, and no Primitive is added.
The precedent is already in the tree. src/bubble.js draws speech bubbles and a
thinking indicator — Shell-owned visuals, over the sprite, in the webview, no
Character art involved. A cue is the same kind of thing.
Audio needs no dependency either. The sprite renders in a Tauri webview
(src/index.html, src/main.js), so the Web Audio API is already there. An
oscillator plus a gain envelope is a few lines and no crate, no sample files,
nothing to ship per Character. Procedural in the literal sense: the sound is
computed, not loaded.
The seam
The frame event already carries one-tick pulses, which is exactly the shape a
cue needs. src/main.js:297 on dialogue: "Dialogue rides exactly one tick, and
latest keeps only the newest placement" — the bubble machine latches the pulse
where every delivery is seen.
A cue pulse rides the same path:
| Step | Where |
|---|---|
Decide the cue, from the tick's verbs and the Dragged transitions |
crates/core/src/engine.rs, beside addressed at :888 |
| Add it to the per-instance payload | src-tauri/src/main.rs:223 SpritePlacement, :330 Placed |
| Fill it in | src-tauri/src/frame_loop.rs:1188 |
| Latch it and start the cue | src/main.js, alongside view.bubbles.event(owned) |
The Engine decides which cue, not the webview. It is the only place that knows
both the verbs and the State transitions, it is pure, and the decision is
therefore testable without a window. The webview is told which cue to play and
plays it.
The cue vocabulary
Five cues, one per interaction. The verb set is fixed at five (#6), so the
vocabulary is the verb set and nothing has to be invented for a sixth.
| Interaction | Cue fires on |
|---|---|
| Left click | Verb::Poke |
| Double click | Verb::Summon |
| Right click | Verb::Menu |
| Pickup | Entering State::Dragged |
| Drop | Leaving State::Dragged |
A Throw is a drop that was moving. It gets the drop cue, played harder rather
than a sixth sound — say the same envelope with more gain and a higher pitch. If
that reads as too subtle in the hand, it becomes its own cue; that is a tuning
call, not a structural one.
Decided: a double-click emits Summon, not Poke and Summon
crates/core/src/input.rs:204 currently returns vec![Verb::Poke, Verb::Summon]
for the second click of a pair. That changes to vec![Verb::Summon], and the
comment above it — which argues for keeping the Poke — goes with it.
Two consequences the change has to carry, or it is a regression:
Summon has to start marking the buddy as addressed. engine.rs:834 sets
addressed = true inside the Verb::Poke branch, and nowhere else except the
cursor dwell at :644. addressed is what makes the next Director wake
reactive rather than ambient. Drop the Poke from a double-click and, as the code
stands, a double-click stops counting as the user addressing the buddy — the
loudest interaction there is would become invisible to the Director. Summon
must set addressed in the same way. Same for Menu, on the same reasoning:
engine.rs:2226 already calls both "the user reaching for the sprite".
Summon should play the react animation that the Poke used to bring. Today
the second click's Poke is what plays Primitive::React (engine.rs:831).
Without it, a double-click would visibly do less than a single click. Summon
plays React itself, so the sprite still notices you.
The first click of a pair still emits a Poke, and that cannot be fixed here.
A double-click is two releases: the first emits [Poke] at input.rs:206
before anything knows a second is coming, the second now emits [Summon].
Suppressing that first Poke would mean holding every left click for
DOUBLE_CLICK_MS (400 ms, input.rs:38) to see whether a partner arrives,
which makes every single click feel broken. So a double-click is a Poke cue
followed ~200 ms later by a Summon cue.
Handle it at the cue layer, not the verb layer: the Summon cue cancels a Poke
cue still in flight. The verb stream stays honest, the ear hears one cue, and
the cancel is a few lines in the webview where both cues live. This is why the
cue machine needs to own its own timing rather than fire and forget.
A drag emits no Poke today — that part already holds
Worth stating because it looks like a bug and is not: a press that becomes a
drag never emits a Poke. input.rs:177-181 moves Phase::Pressed straight to
Phase::Grabbing and returns vec![Verb::Grab], and the Poke only comes from
the release branch at :193. So pickup already does not fire the click cue.
Two mechanics the pickup and drop cues have to respect:
Verb::Grab arrives every tick, not on the press. engine.rs:67 says so
outright: "present on every tick the sprite is held". A cue keyed on the verb
being present would fire every frame of a drag. The pickup cue fires on the
transition into State::Dragged.
There is no drop verb, and adding one is not allowed. input.rs:216
returns Vec::new() when a grab is released without enough speed to be a
Throw — the Engine drops the sprite because the Grab stopped arriving, so
nothing announces it. The verb set is fixed at five by #6 and ADR-0002, so a
sixth Drop verb is exactly what the project forbids. The drop cue therefore
keys on leaving State::Dragged, which is also what makes drop and throw
distinguishable: both leave Dragged, and only the throw carries a velocity.
Decided
Settled, not open:
- Audio is on by default. Mute lives in Settings.
- Do Not Disturb silences the audio cue and keeps the visual one. Do Not Disturb: the buddy stays, and stops starting things #84 is
"quiet but not gone", and a visual cue cannot embarrass anyone in a meeting. - The cue vocabulary is global. No per-Character pitch or color, no new
manifest field. A Character-declared vocabulary would be its own issue and
its own ADR conversation.
Acceptance
- Left click, double click, right click, pickup, and drop each produce a
distinct visual cue and a distinct sound. - A double-click emits
Verb::Summonalone.input.rs:204no longer
returns a Poke with it. -
SummonandMenusetaddressed, so a double-click and a right-click
still make the next Director wake reactive. A test covers it — this is the
regression the verb change would otherwise cause. -
SummonplaysPrimitive::React, so a double-click is not visibly less
than a single click. - The Summon cue cancels a Poke cue still in flight, so one double-click is
heard as one cue and not two. - Every cue is edge-triggered. Holding the button through a drag fires the
pickup cue once, not once a tick. - The drop cue fires on leaving
State::Dragged, for a still drop and for a
throw alike. No sixth verb is added. - No Character Package changes. No new Primitive, no new Required Animation,
no new manifest field. - No new dependency for audio.
- Audio is on by default and mutable in Settings, and the mute takes effect
without a restart. - Do Not Disturb silences the audio cue and leaves the visual one.
- A hidden Character (Do Not Disturb: the buddy stays, and stops starting things #84
hidden) produces no cue of either kind. - The verb and state-transition mapping is unit-tested, in
input.rsfor
the verbs a gesture emits and inengine.rsforaddressedand the
Draggedtransitions. The drawing and the sound are not testable from
Rust, and the issue does not pretend otherwise. - Works on macOS and on Linux X11.
Out of scope
- The chat surface (Chat surface #17) and the tray menu (Tray and settings #18). This is the cue that the
gesture registered, not the surface it opens. - Per-Character cue vocabulary, and any Character Manifest field for one.
- A separate throw cue, unless the shared drop cue reads wrong in the hand.
- Windows.
docs/SPEC.md:569keeps the arms stubs.
Related: #6 fixes the verb set at five, #17, #18, #84 for Do Not Disturb,
ADR-0002 for why the cue cannot be Character art, ADR-0006 for the pixel-art
scaling a procedural visual has to sit beside without clashing.
All file:line references are pinned to main at 873c05f1.
— Claude Code, on @omesser's behalf.
Metadata
Metadata
Assignees
Labels
Projects
- StatusShow more project fieldsDone
Activity
omesser commentedon Sep 3, 2026
Body updated with the decisions from review: audio on by default with a mute in Settings, Do Not Disturb silences the audio cue and keeps the visual one, the cue vocabulary is global, and a double-click emits
Verb::Summonalone.Tracing the drag request turned up two things worth flagging before implementation:
SummonandMenudo not setaddressed.engine.rs:834sets it inside theVerb::Pokebranch only. Removing the Poke from a double-click would stop a double-click counting as the user addressing the buddy, so the next Director wake would be ambient rather than reactive. Both verbs now have to set it, andSummonalso has to playPrimitive::Reactor a double-click does visibly less than a single click.input.rs:216returns no verb when a grab is released too slowly to be a Throw — the Engine drops the sprite because the Grab stopped arriving. So the pickup and drop cues key on entering and leavingState::Draggedrather than on verbs, which keeps the verb set at five per The five interaction verbs #6 and ADR-0002.One constraint that cannot be designed away: the first click of a double-click has already emitted its Poke before anything knows a second is coming. Suppressing it would mean holding every left click for the 400 ms double-click window. The Summon cue therefore cancels a Poke cue still in flight, at the cue layer.
Relabelled
ready-for-agent.— Claude Code, on @omesser's behalf.
omesser commentedon Sep 3, 2026
Cue design approved. The set is now fixed, and
docs/design/cues.htmlin #282 is the spec — every duration, easing curve, frequency and envelope in itscuesCSS block andCuesscript block is signed off.The grammar is direction: Poke goes out, Summon is Poke reversed and doubled, Menu goes around, Pickup goes up, Drop goes sideways. That is what lets the cues be told apart without being loud. Every stroke is white on ink, the pair
src/main.cssalready uses for the speech bubble, so the overlay gains no new colors.The Menu cue went through one revision: it began as a thin three-quarter stroked arc, which read as low-resolution because a 2.5 px SVG stroke landed between pixels. It is now a conic gradient masked to a thin donut and snapped to whole pixels, which is crisp at any DPI.
Prerequisites #278 and #280 are merged. The implementation is in flight as PR 3 of 3.
— Claude Code, on @omesser's behalf.
docs: Add the interactive cue proposal for #277 (#282)