Allow per-prompt model/provider selection for rewording prompts

[kostrse]

Hi! First of all, thank you for new features in the Dictate app.

I would like to suggest adding an optional per-prompt model/provider override for AI rewording prompts.

Motivation

Different prompt presets can have very different cost, latency, and quality requirements.

For example, prompts that run automatically after every transcription often need to be very fast and cheap:

• punctuation cleanup
• light grammar correction
• speech-to-written-text cleanup
• filler-word removal
• simple formatting normalization

For these tasks, a small or inexpensive model is often good enough, especially if the prompt is auto-applied after every dictation. Using a larger model for every small cleanup step can add unnecessary latency and cost.

At the same time, other prompt presets may genuinely benefit from a stronger or more specialized model:

• sophisticated rewriting
• style/tone transformation
• long-form text restructuring
• summarization or analysis
• specialized translation or domain-specific editing

For those tasks, it would be useful to select a more capable model even if it is slower or more expensive.

It would be great if each saved rewording prompt could optionally choose its own model/provider, while still defaulting to the global rewording configuration when unset.

Example workflows:

• "Auto cleanup" → cheap/lightweight model, auto-applied after every transcription
• "Fix punctuation" → small local/open-source model
• "Rewrite professionally" → stronger cloud model
• "Analyze and restructure" → highest-quality model
• "Translate carefully" → specialized translation-capable model

Why this is different from per-prompt reasoning effort

The current per-prompt reasoning setting is very helpful for controlling how much effort a reasoning-capable model spends on a task. This request is slightly different: it is about routing different prompts to different models/providers entirely, not just changing reasoning effort on the same selected model.

Possible UI/implementation shape

One simple approach could be an optional “Model” section in the prompt editor:

• "Default" — use the global rewording model
• "Custom" — choose provider/account and chat model for this prompt

Another approach could be reusable “execution profiles,” such as:

• "Cheap / fast"
• "Balanced"
• "Best quality"
• "Local/private"

Then each prompt could select a profile instead of duplicating provider/model settings.

Thanks for considering it!

[MyButtermilk]

I’ve had great success using Gemma 4 31B via Cerebras for transcript cleanup. It is extremely fast, has very low initial latency, and the high tokens/sec noticeably improves the overall dictation experience. For a keyboard app, this matters a lot: even small delays after every transcription make the app feel slower.

Suggested default profile:

• Provider: Cerebras
• Model: gemma-4-31b
• Temperature: 0–0.2
• Streaming: enabled
• Max output tokens: limited
• Fallback: user’s global/default model if Cerebras is unavailable or rate-limited

This would be a great default for:

• punctuation cleanup
• filler-word removal
• light grammar correction
• speech-to-written-text cleanup
• number/unit formatting

Per-prompt model ist probably not needed because Cerebras as a default is so fast that you will not gain anything on top.

Considering that it is multi modal it could be used for one shotting transcript + post processing as well. However I imagine that the transcription quality is significantly worse than other dedicated stt models. However I tested it for post-processing only where it excels.

Blog Release:
https://www.cerebras.ai/blog/gemma-4-on-cerebras-the-fastest-inference-is-now-multimodal

Implementation links:

1 Cerebras OpenAI-compatible API: https://inference-docs.cerebras.ai/resources/openai
[2] Chat completions API: https://inference-docs.cerebras.ai/api-reference/chat-completions
[3] Streaming responses: https://inference-docs.cerebras.ai/capabilities/streaming
[4] Model selection / Gemma 4 31B: https://inference-docs.cerebras.ai/models/choose-a-model
[5] Rate limits / fallback handling: https://inference-docs.cerebras.ai/support/rate-limits

[DevEmperor]

Thank you for the really thoughtful, well-structured request — and for the Cerebras tip!

I've decided not to add per-prompt model/provider selection, and I want to be transparent about why:

• The prompt editor is a keyboard settings surface where simplicity matters a lot, and it already carries name, prompt text, selection/auto-apply flags and per-prompt reasoning effort. A full provider + account + model picker per prompt (plus the fallback handling this needs when a provider is rate-limited) would make it noticeably heavier for a feature that, realistically, only a small number of power users would use.
• Most of the practical benefit is already reachable: pick a fast, inexpensive global rewording model, and use the existing per-prompt reasoning effort to dial effort up or down per prompt. For short cleanup prompts the model-to-model differences in latency/cost tend to be small once the provider is fast.
• On Cerebras specifically: it's OpenAI-compatible, so you can already use it today by adding it as a custom provider in Dictate (base URL from their OpenAI-compat docs) and setting Gemma as your rewording model — no per-prompt feature required. It does look great for fast cleanup, thanks for the pointer.

I'm keeping this in mind if demand grows, but for now I'd rather keep the prompt editor simple. Really appreciate the detailed write-up!