Skip to content

feat(input): the controller role follows the engaged device (ADR-034 Amendment 6) - #2460

Draft
dfattal wants to merge 2 commits into
mainfrom
feat/input-engagement
Draft

dfattal wants to merge 2 commits into
mainfrom
feat/input-engagement

Conversation

@dfattal

@dfattal dfattal commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Why

Presence answers "is the hardware there", not "is the user using it". A camera hand-tracking input provider that publishes left/right khr/simple_controller devices (select = pinch, grip/aim = palm) reports PRESENT whenever a camera exists. Under the presence-ranked walk (Amendments 1/3) it therefore held the controller roles permanently, and keyboard/mouse never got them back. With this change the role follows what the user is actually using.

What

ABI (append-only, API major stays 1): xrt_input_plugin_iface gains

int64_t (*get_engaged_ns)(struct xrt_input_plugin_instance *inst, uint32_t hand);

It returns the os_monotonic_get_ns time (CLOCK_MONOTONIC on POSIX, QPC on Windows) of that hand's last deliberate use (0 = left, 1 = right), or 0 for never. Deliberate means a pinch or a button press, never mere visibility. The slot is struct_size-gated through XRT_INPUT_PLUGIN_IFACE_HAS and announced by XRT_INPUT_PLUGIN_IFACE_HAS_GET_ENGAGED_NS. New ABI asserts pin the order of the appended slots and the size of the vtable.

Arbiter (target_input_arbiter.c), controller family only:

  • Per hand, the most recently engaged present candidate wins.
  • The keyboard is the default holder: a provider that implements the slot but has not engaged that hand since its latest return to PRESENT does not hold the hand while any other eligible candidate exists. The qwerty floor holds instead (never touched counts as the default holder), or an engaged or slot-less present provider. Such a provider holds by rank only when it is the only candidate (EmulatedControllers = 0). Ties keep rank order. (Changed in 2d8c1ef after the first live run: apps started on silent camera controllers until the user touched the mouse.)
  • A provider without the slot counts as engaged continuously while present. Today's behaviour is unchanged for it: a Leap keeps winning while plugged in.
  • An engagement older than a candidate's latest return to PRESENT does not count, so a provider that comes back does not reclaim the role on a stale pinch. The current leader refreshing its own engagement moves nothing.
  • Hard choices stay hard: a per-app pin, an app request, or a PreferredPlugin naming a controller provider turns switching off for the clients it applies to. Policy\Lock acts through the precedence it already shapes.
  • Engagement is read on the presence poll thread (input: no provider code on client threads — presence snapshot on a service timer; qwerty integrator gets an owner and a lock #958, never a client thread) at its ~250 ms rate. It is read for every present slot provider, including ones not holding the role.
  • It is read only while a client resolved roles in the last second, so a provider may treat the calls as demand and keep its camera open. An idle service stops calling.
  • A change of leader bumps the state epoch, so per-client cached roles recompute and generation_id moves.
  • The first engagement-driven move logs at WARN, later ones at INFO.

qwerty: qwerty_controller_last_input_ns() returns the latest of the controller's button, stick and trackpad edges and the movement/look keys and mouse deltas aimed at that controller. The qwerty head's own WASD/mouse-look does not count. The qwerty builder installs it with t_input_arbiter_set_floor_engagement.

Profile stability: the masquerade still applies. When the floor holds a hand by engagement while the provider is present, it reports the top provider's profile. The unit test asserts that left_profile stays the same across camera → keyboard → camera.

Policy ControllerSwitching = use (default) | presence (exactly today's walk):

  • Windows: InputProviders\Policy\ControllerSwitching, REG_DWORD 1 = use, 0 = presence. HKCU wins over HKLM, and HKCU is hidden under Policy\Lock, the same read as EmulatedControllers.
  • POSIX: a per-user controller_switching file containing use or presence.
  • CLI: displayxr-cli input switching use|presence [--machine].
  • input list --json reports controller_switching: {value, source}.
  • Live update through the component-policy watch: the Windows root is already covered, and on POSIX the file is folded into the input fingerprint.

sim-input: a new ENGAGE_MS setting makes it the in-tree reference for the slot. Each active hand engages every n ms. The default is 0, which leaves the slot NULL and keeps today's behaviour.

Docs: ADR-034 Amendment 6 (and its Status line), input-provider-discovery §2 / §3 / §4 (item 2b, and a policy table in item 3c), input-modality-switching §3.D.

Tests

  • New tests_input_arbiter_engagement (7 cases):
    • With nothing engaged, rank order holds.
    • Qwerty input takes the role; with the profile unchanged and the generation moved.
    • A pinch takes the role back.
    • On presence loss the role falls to the last engaged candidate, and an old pinch does not reclaim it on return.
    • A legacy provider without the slot keeps today's behaviour, and the slot is never called on it.
    • A per-app pin means no switching for that client.
    • A PreferredPlugin means no switching.
    • presence policy gives today's walk, and the slot is never called; a live flip back to use works.
    • The slot is called only while a client resolves roles: the calls stop after the demand window and resume when a client returns.
  • macOS: all input, arbiter, sim_input and qwerty tests pass (17/17).
  • MinGW compile check passes for aux_util, drv_qwerty, target_input_arbiter.c, target_input_plugin_loader.c and cli_cmd_input.c.
  • Windows N4 per-class conformance gains step 3f. It covers the store tiers, the input switching verb and input list --json.

Notes / deviations

  • Legacy beats slot: a present provider without the slot always beats a slot provider that is merely present, whatever their rank, because it is "continuously engaged". This is the literal reading of "engaged continuously while present"; it is documented.
  • With two controller providers, profile stability is guaranteed only against the top-ranked one, which is the one the masquerade mirrors.
  • Policy\Lock has no direct switch of its own. It only hides per-user pins, requests and the user default (existing precedence), and switching is off whenever that precedence names a controller provider.

🤖 Generated with Claude Code

…Amendment 6)

Presence answers "is the hardware there", not "is the user using it". A
camera hand-tracking provider that publishes simple_controller devices is
PRESENT whenever a camera exists, so under the presence-ranked walk it held
the controller roles permanently and the keyboard never got them back.

- xrt_input_plugin_iface: append the optional, struct_size-gated slot
  int64_t (*get_engaged_ns)(struct xrt_input_plugin_instance *inst,
  uint32_t hand) — os_monotonic_get_ns time of that hand's last deliberate
  use, 0 = never. API major stays 1; XRT_INPUT_PLUGIN_IFACE_HAS_GET_ENGAGED_NS
  announces it; ABI asserts pin the append order and the vtable size.
- Arbiter: per hand, the most recently engaged PRESENT controller candidate
  wins; ties / never-engaged keep rank order; a provider without the slot is
  engaged continuously while present (bit-identical to before); engagement
  older than a candidate's return to PRESENT does not count. A pin, a session
  request or a PreferredPlugin naming a controller provider turn it off for
  that client. Engagement is read on the presence poll thread (never a client
  thread), for every present slot provider, only while a client resolved
  roles in the last second (providers may treat the calls as demand); a
  change of leader bumps the state epoch.
- qwerty: qwerty_controller_last_input_ns() — the floor's engagement is the
  last keyboard / mouse input aimed at an emulated controller.
- Policy ControllerSwitching = use (default) | presence: Windows
  InputProviders\Policy\ControllerSwitching DWORD 1|0 (HKCU over HKLM, the
  EmulatedControllers read), POSIX per-user controller_switching file. CLI
  `input switching use|presence [--machine]`; `input list --json` carries
  controller_switching {value, source}; live through the policy watch.
- sim-input: ENGAGE_MS setting, the in-tree reference for the slot.
- Tests: tests_input_arbiter_engagement; Windows N4 step checks the store,
  the verb and input list --json.
- Docs: ADR-034 Amendment 6, input-provider-discovery §2/§3/§4,
  input-modality-switching §3.D.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…is engaged

First live run (camera hands filling get_engaged_ns): at generation 1 the
arbiter gave both hand roles to the provider although nothing had engaged,
because never-engaged ties fell back to rank — so the app started on
controllers that report nothing (no hand in view) until the user touched the
mouse.

Rule (same intent, the role follows what you use): a provider that
implements the slot and has not engaged that hand since its latest return to
PRESENT does not hold it while any other eligible candidate exists — the
qwerty floor (never touched = the default holder, not "never engaged") or an
engaged / slot-less present provider holds instead. It holds by rank only
when it is the only candidate (EmulatedControllers = 0). Among engaged
candidates the most recent still wins; ControllerSwitching = presence is
unchanged.

Tests: nothing engaged -> qwerty holds from generation 1; first pinch ->
provider (that hand only); no floor -> slot provider holds by rank; the pin
case now starts the unpinned client on the keyboard. ADR-034 Amendment 6,
the spec §4 item 2b, the slot doc and the roadmap updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant