MIT's differentiable acoustic simulator reverse-engineers vocal tract from audio
maier_ak · x · 2026-11-12
A NeurIPS 2026 spotlight paper by MIT's Eric Ming Chen, Jin Woo Lee and Vincent Sitzmann reconstructs vocal tract shape from sound alone, mimicking how children learn speech by listening.
Method
- Builds on the classic source-filter model: vocal folds produce a buzz; throat, mouth and lips shape it into speech
- The tract is modeled as a tube with width varying along length and time, divided into 32 sections, simulated with standard acoustics equations
- Key novelty: a differentiable simulator fast enough for deep learning pipelines — guess a tube shape, simulate sound, compare spectrograms with the real recording, and backpropagate the mismatch gradient to refine the shape
Author-noted limitations
- The problem is ill-posed: different shapes can produce the same sound
- True anatomy validated on only six vowels; MRI videos come from a generator trained on the same speakers
- No nasal tract; no code released yet
More from Research
- StepFun's P²-MOPD beats multi-teacher distillation with half the training steps — teortaxesTex · 2026-11-12
- Stanford professor: virtual cell models don't need billion-parameter foundation models — tweaked scVI did fine — anshulkundaje · 2026-11-12
- 2048 TPUv4s and deep domain expertise: professor dissects how a gene-regulation model was trained — anshulkundaje · 2026-11-12
- Professor: gene-regulation model was built on open academic data from ENCODE, GTex and Cell x Gene — anshulkundaje · 2026-11-12
- Stanford professor hails new gene-regulation model as a real paradigm shift toward causal cis-trans models — anshulkundaje · 2026-11-12
- François Fleuret asks: are formal proof checkers actually unused in RL training? — francoisfleuret · 2026-11-12