Distilling speech encoders on internal representations: shrinking on-device ASR without quality loss
burkov · x · 2026-11-12
On-device speech recognition must balance accuracy against tight memory, power, and latency budgets. The audio encoder runs continuously and shares device memory with the language model it feeds, so an oversized encoder limits capacity for the rest of the system.
The article evaluates compressing the encoder via distillation: training a smaller student to reproduce the teacher's internal audio representations rather than final token choices or transcripts. This target is shared by two audio-to-LM interface designs, making the method usable with either.
The approach was tested on six teacher-student pairings, covering encoders trained both before and after joint training with a language model, confirming its generality.
More from Research
- CogBattery benchmarks 54 open-weight LLMs against humans on 13 cognitive paradigms — Dengzhe Hou · 2026-11-12
- After 17 years in bio, a scaling researcher calls AI-biomed hype Theranos-level — zacharynado · 2026-11-12
- AI 2027 authors: intelligence explosion on track for 2027, takeover window 3-12 months — Puzzleheaded-King584 · 2026-11-12
- sokrypton: solving biology needs subatomic simulators or massive wet labs for AI — sokrypton · 2026-11-12
- Uncertainty-Aware RWM: taming world-model hallucination for real robot control — breadli428 · 2026-11-12
- Can You Guess All Four Gradient Distributions? A Viral ML Fundamentals Quiz — cneuralnetwork · 2026-11-12