You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix: stream voice transcription into chat composer - #866
full desktop frontend suite: 296 files / 3416 tests passed; one unrelated PDF resource suite could not start because the test runtime does not provide DOMMatrix
Each live onLine dispatches a whole new composer-draft state (setComposerDraftForScope β dispatch(composerDraftUpdated)), which re-renders this very large page. The streaming transcriber emits partial events continuously (several per second, unlike the old ~6s chunked lines), so a full page re-render plus a requestAnimationFrame resize per partial can make dictation janky. Consider coalescing live updates, e.g. accumulate the latest text in a ref and flush at most once per animation frame (or debounce ~100ms) instead of dispatching on every partial.
When the recorded provisional range no longer matches (user edited before it, a slash command rewrote the draft, or the draft was cleared because the user hit Enter to send mid-dictation), this fallback appends next via mergeVoiceTranscript(current, nextText) β but next is the entire accumulated live transcript, not just the new remainder. All previously dictated words therefore appear a second time in the composer (and the final transcript replaces only that appended block, leaving the duplicate behind). Suggest trying to re-locate previous.text in current to recover the offset, and otherwise insert only the part of next that is not already present instead of re-merging the full text.
The recording is pinned to the scope that started it, but nothing cancels or reports it when chatScopeKey changes during dictation (e.g. the user switches conversations). The mic button in the new chat still renders as active because asrRecorder.isRecording is global, so pressing it stops the old recording and writes the whole transcript β plus any later live lines β into the previous chat's draft and error notices, invisible to the user. Consider cancelling voice input when the scope changes, alongside the existing useEffect(() => { setTaskRecording(null); }, [chatScopeKey]), or at least surfacing that recording continues in another chat.
These slash-menu/command-selection resets run unconditionally, while resize/focus just below are gated on scopeKey === chatScopeKey. When the recording belonged to a different chat scope, the current chat's slash-menu state is reset for no reason. Move them inside the same scope guard for consistency.
The live option supplies only onLine, so live-transcription failures are completely silent here. recording-entry.tsx passes onError for exactly this reason: when the relay cannot be opened or a segment request fails, the user keeps speaking while the composer stays blank (and the segmented fallback can fail too), with no hint that dictation is not being transcribed. Consider adding an onError that reports the first failure (e.g. via setComposerMediaErrorForScope), suppressed/throttled so the 6s fallback retries do not spam toasts (see isLiveAsrTimeout for the timeout case).
mergeLiveLines de-duplicates by startMs, which only holds while offsets are absolute from the start of the recording. When the streaming transcriber fails, useAsrRecorder silently swaps in createChunkedLiveTranscriber, whose consumedSamples restarts at 0, so its lines (0, 6000, 12000β¦) collide with and get sorted among the earlier stream lines. The merged text written into the composer is then re-ordered and partially overwritten (earlier sentences dropped, later ones shuffled), and because that whole string replaces the provisional range, the corruption is user-visible in the draft. Consider tracking revisions against the most recent line (replace when the offset does not advance, append when it does) instead of a global offset-keyed merge, or reset the accumulated lines when the recorder falls back.
Generated by cloud-assistant via Open Code Review.
Tested source: 71a43b79c0d384eeba5c8abbcd26e27232297713
Tested base: 1f87dcf385db4339da63f7018339cdd63f781dd7
Tested commit: 71a43b79c0d384eeba5c8abbcd26e27232297713
These results apply only to this immutable source/base snapshot.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Validation
@memmy/frontend-desktoptypecheck passedDOMMatrixorigin/v1.2.7(1 commit, 2 changed files)