Skip to content

feat: add full-duplex voice chat demo - #1365

Open
maciejkucmierz-swm wants to merge 1 commit into
software-mansion:mainfrom
maciejkucmierz-swm:feat/voice-chat-demo
Open

maciejkucmierz-swm wants to merge 1 commit into
software-mansion:mainfrom
maciejkucmierz-swm:feat/voice-chat-demo

Conversation

@maciejkucmierz-swm

Copy link
Copy Markdown
Contributor

Demonstrate upstream communication sessions, voice processing and queued Realtime playback with barge-in and lifecycle recovery in FabricExample.

Include a server-only token endpoint, focused regression tests and AEC documentation links.

Related: #1323, #1324, #1317.
Companion documentation: #1344.
No issue to close.

⚠️ Breaking changes ⚠️

  • None. No public API or library implementation changes.

Introduced changes

  • Add a Voice Chat demo showcasing RNAA communication sessions, voice processing and streamed playback with barge-in.
  • Handle audio interruptions and session cleanup.
  • Include a local OpenAI token server, setup instructions and AEC documentation links.
  • Add focused tests for turn-taking, playback and lifecycle handling.

Validation:

  • 25 focused app tests and 3 mocked token-server tests pass.
  • TypeScript, lint, formatting, Android bundling and Android debug build pass.
  • yarn validate:fast passes, including 549 C++ tests and 117 library JS tests.
  • Physical Android and iPhone testing completed successfully.

Checklist

  • Linked relevant issue — related PRs linked; no issue to close
  • Updated relevant documentation
  • Added/Conducted relevant tests
  • Performed self-review of the code

Demonstrate upstream communication sessions, voice processing and queued
Realtime playback with barge-in and lifecycle recovery in FabricExample.

Include a server-only token endpoint, focused regression tests and AEC
documentation links.
@maciejkucmierz-swm
maciejkucmierz-swm marked this pull request as ready for review October 26, 2026 14:38
@github-actions

Copy link
Copy Markdown

WPT non-regression comparison

PASS — no regressions · 0 improved section(s) · overall 3427 → 3427 (0)

Unchanged sections (28)
Spec section Base pass Head pass Delta
Processing model 1 1 0
Other 56 56 0
AnalyserNode 138 138 0
AudioBuffer 140 140 0
AudioBufferSourceNode 220 220 0
AudioContext 58 58 0
AudioNode 261 261 0
AudioParam 680 680 0
BiquadFilterNode 275 275 0
ChannelMergerNode 30 30 0
ChannelSplitterNode 7 7 0
ConstantSourceNode 64 64 0
ConvolverNode 178 178 0
DelayNode 127 127 0
DestinationNode 0 0 0
DynamicsCompressorNode 4 4 0
GainNode 15 15 0
IIRFilterNode 87 87 0
MediaElementAudioSourceNode 0 0 0
MediaStreamAudioDestinationNode 1 1 0
MediaStreamAudioSourceNode 0 0 0
OfflineAudioContext 35 35 0
OscillatorNode 104 104 0
PannerNode 727 727 0
PeriodicWave 33 33 0
ScriptProcessorNode 0 0 0
StereoPannerNode 104 104 0
WaveShaperNode 82 82 0

Baseline: a07241cbc70ac16a4c107e1125d74fc8ca63eb0e · Candidate: fd9be3a7683c6d4e6245dd0d5ca611dc33971bb8

Workflow run · this comment is updated on every push.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant