Repository navigation
Conversation
When using Telnyx as the phone provider, you can now use Telnyx for speech-to-text and text-to-speech too, reusing the same API key. No separate OpenAI account needed for speech services. STT: streaming transcription via wss://api.telnyx.com/v2/speech-to-text/transcription - Accepts mu-law audio directly (8kHz mono) from the phone bridge, no conversion needed - Configurable engine (Telnyx, Deepgram, Google, Azure) and input format - Auto-detected as default STT when CALLME_PHONE_PROVIDER=telnyx TTS: streaming synthesis via wss://api.telnyx.com/v2/text-to-speech/speech - Returns raw PCM audio (matches the existing TTSProvider interface) - Voice catalog at GET /v2/text-to-speech/voices (AWS Polly, Azure, Google, etc) - Default voice: AWS.Polly.Matthew-Neural (en-US male) Both providers fall back to CALLME_PHONE_AUTH_TOKEN when CALLME_TELNYX_API_KEY is not set, so zero extra config is needed when already using Telnyx for phone.
Three critical bugs that would break the TTS path at runtime, plus
three medium-severity issues in the STT path.
TTS (server/src/providers/tts-telnyx.ts):
- audio_format 'pcm' is not a valid Telnyx value. The server silently
rejects it, which is almost certainly why live verification got
zero audio back. Switch to 'linear16' (16-bit signed LE PCM at
24kHz mono), which is what the existing TTSProvider contract and
the resampler in phone-call.ts already expect.
- Stop frame was { force: true }, which the Telnyx AsyncAPI defines as
an interrupt-mid-stream signal, not a completion signal. The
documented stop frame is { text: '' }. Sending force:true leaves
the server waiting for more text and never closes the session.
- 30s setTimeout was never cleared on resolve/error/close, leaking a
timer per synthesize call. Wrapped all settle paths in a single
settle() helper that clears the timer, removes WS listeners, and
closes the socket exactly once.
STT (server/src/providers/stt-telnyx.ts):
- connect() had no timeout. A hung WebSocket would block
initiateCall() forever. Added a 10s connect timeout that matches
the OpenAI Realtime provider.
- waitForTranscript cleared pendingTranscript on entry, silently
dropping any is_final events that arrived before listen() started
(e.g. user spoke during TTS playback). Now returns the buffered
transcript immediately if any finals are pending.
- Removed dead reconnectAttempts / maxReconnectAttempts /
reconnectDelayMs fields. They were declared and reset but never
used; reconnect logic was never implemented for Telnyx STT.
Provider factory (server/src/providers/index.ts):
- validateProviderConfig OpenAI-key check was
'sttProvider != telnyx AND ttsProvider != telnyx', which only
errored when BOTH were non-Telnyx. Mixed configs like
openai-realtime STT + telnyx TTS passed validation and then failed
at runtime. Replaced with needsOpenAI = stt openai-realtime OR
tts openai.
Verified: tsc --noEmit clean, bun smoke test confirms URL params
(linear16, 24kHz, mulaw, 8kHz) and stop frame { text: '' }.
Refs ZeframLou#39
QA pass + fixes pushedI did a thorough QA pass on this PR (cloned, typechecked, verified the protocol against the official Telnyx WebSocket docs at Critical (would have broken TTS at runtime)1. 2. Stop frame was 3. Medium4. TTS 30s timeout timer was never cleared. Each 5. STT 6. STT Cleanup7. Dead reconnect fields. Verification
The original commit's honesty about not being able to verify TTS audio was the right call. The root cause was the invalid |
|
Circling back on this PR — the QA round with the six fixes has been sitting for ~3 weeks now (last push on 2026-07-19), and I want to make sure it isn't blocked on me. Quick state:
If anything in the diff needs rework for the maintainer's taste — naming, file layout, error handling, comments — happy to push a follow-up commit. Otherwise, if the right call is to keep this parked or close it out, let me know and I'll stop nudging. No pressure either way. Thanks for the time on this. |
|
One more bump — about 6 weeks since this PR opened and ~3 weeks since the last ping. If the queue is just long, no worries. If anything in the diff needs rework on your end (naming, file layout, error handling, comments), happy to push a follow-up commit. If I don't hear back by mid-September, I'll close this out rather than let it linger. |
What this changes
Adds Telnyx STT and TTS providers so that when you're already using Telnyx as your phone provider, you can use Telnyx for speech-to-text and text-to-speech too, with the same API key. No separate OpenAI account needed for speech services.
CallMe already recommends Telnyx as the default phone provider ("50% cheaper"). This PR completes the "Telnyx everywhere" story: phone, STT, and TTS all on one account, one key, one bill.
Files
server/src/providers/stt-telnyx.tsTelnyxSTTProviderimplementingRealtimeSTTProvider. Streaming STT via WebSocket.server/src/providers/tts-telnyx.tsTelnyxTTSProviderimplementingTTSProvider. Streaming TTS via WebSocket.server/src/providers/index.tsSTTProviderType,sttProviderconfig field, Telnyx-specific env vars. Auto-detecttelnyxas default STT whenCALLME_PHONE_PROVIDER=telnyx. Relax OpenAI key requirement when using Telnyx for both STT and TTS..env.exampleREADME.mdHow it works
STT (
wss://api.telnyx.com/v2/speech-to-text/transcription):transcription_engine,input_format,sample_rate,language).Telnyx,Deepgram,Google, orAzure.TTS (
wss://api.telnyx.com/v2/text-to-speech/speech):TTSProvider.synthesize()interface.GET /v2/text-to-speech/voices(AWS Polly, Azure, Google, ElevenLabs, Resemble, xAI). Default:AWS.Polly.Matthew-Neural.Both providers fall back to
CALLME_PHONE_AUTH_TOKENwhenCALLME_TELNYX_API_KEYis not set, so zero extra config is needed when already using Telnyx for phone.How I tested it
tsc --noEmitpasses (no type errors)wss://api.telnyx.com/v2/speech-to-text/transcription?transcription_engine=Telnyx&input_format=linear16&sample_rate=8000, sent raw PCM binary frames, received{"transcript":" you","confidence":null,"is_final":true}.{"text":" ","voice_settings":{"voice_speed":N}}, text frames{"text":"..."}, audio frames{"audio":"<base64>"}, final frame{"isFinal":true}).wspackage for WebSocket connections.Environment variables
CALLME_STT_PROVIDERtelnyx*openai-realtimeortelnyx. *Auto-defaults totelnyxwhenCALLME_PHONE_PROVIDER=telnyxCALLME_TTS_PROVIDERopenaiopenai,kokoro, ortelnyxCALLME_TELNYX_API_KEYCALLME_PHONE_AUTH_TOKEN)CALLME_TELNYX_STT_ENGINETelnyxTelnyx,Deepgram,Google,AzureCALLME_TELNYX_STT_INPUT_FORMATmulawmulaw,linear16,alawCALLME_TELNYX_STT_LANGUAGEen-USCALLME_TELNYX_TTS_VOICEAWS.Polly.Matthew-Neural/v2/text-to-speech/voices)