Phoneme-guided initialization cuts LLM-based ASR error to 11.7% in low-resource settings
kastnerkyle · x · 2026-10-15
Researchers publishing at IEEE SLT 2026 propose phoneme-guided initialization, tackling the modality-matching problem between audio encoders and LLMs when paired speech-text data is scarce.
- Method: Pre-train the audio encoder on speech-to-phoneme (S2P) and the LLM on phoneme-to-grapheme (P2G) tasks, then connect them and fine-tune end-to-end on the target ASR task — bringing cascade-style benefits into an E2E framework
- Results: Matches or beats both the cascaded S2P-P2G baseline and vanilla E2E models on Japanese CSJ (CER 20.8%→11.7% vs 18.1% for cascade on 10h), Chinese AISHELL-1, and low-resource Tatar and Urdu
- Most effective for languages with a large phoneme–grapheme gap, where paired data is limited
More from Research
- Goodfire cofounder's essay: neural nets are far more interpretable than we thought — burny_tech · 2026-10-16
- Cornell and Amazon revisit the 16x pretraining compute gap: diffusion LMs need only 3-8x — LucaAmb · 2026-10-16
- Interpretability debate: Nick Cammarata says most neurons are legible, Liv Gorton pushes back — sethlazar · 2026-10-16
- Interpretability pioneer: most neurons in trillion-parameter models are now legible — natesiggard · 2026-10-16
- AI trained on the fraction of human intelligence that fit the page — r0ck3t23 · 2026-10-16
- Quotient Recurrent Reasoners hit 97.9% on Sudoku-Extreme with 78% fewer loops — burny_tech · 2026-10-16