Teen dev replaces Whisper + LLM dictation pipeline with one 1.7B model, 0.5s vs 36s
Mehmet_dedeler · reddit · 2026-11-14
A 14-year-old developer replaced the classic Whisper + LLM dictation pipeline on a 4GB RTX 3050 Ti laptop with a single model, and open-sourced the Typd dictation app (Windows NVIDIA / Apple Silicon).
Results: the two-model pipeline took 2.5–36s per dictation depending on pairing; the single model does 0.5s for short and 1s for long dictations, as a 2.2GB (Q8) file resident in 2.5GB VRAM.
Training: the pipeline exists only at training time — real openly-licensed speech, a large LLM writing final text from transcripts, filtering clips where content was lost, then training Qwen3-ASR 1.7B (Apache-2.0) to go audio→finished text; runs in llama.cpp.
Trade-offs: no prompt-based style control, weaker long-list rewrites due to the tiny internal LM, silent drops (heavily filtered in training). Key advantage: it decides what to keep while it still hears the audio, unlike a text-only cleanup LLM.
Related event: 14-Year-Old Dev Ships Single-Model Dictation App Running on 4GB VRAM(2 posts)→
More from Apps
- DeepSeek open-sources a static chat demo that talks to its API with no backend — rohanpaul_ai · 2026-11-14
- A single prompt: make AI enumerate and iterate client requirements until all satisfied — curious_vii · 2026-11-14
- Parallel launches startup credits program with YC and a16z speedrun, unveils Decisions API — floguo · 2026-11-14
- Grok Bot can now join meetings on Zoom, Google Meet and Teams — Polymarket · 2026-11-14
- Scope launches Competitors to track how AI agents evaluate and bypass your product — RaphaelDabadie · 2026-11-14
- Allie Miller: Anthropic merging Chat and Cowork is a smart enterprise sales move — alliekmiller · 2026-11-14