Oscillator Language Models paper: phase-coded Kuramoto dynamics for language modeling
burny_tech · x · 2026-10-17
A new paper by Ali Kayam (BrainChip) proposes Oscillator Language Models (OLMs), where autoregressive computation happens via phase interactions in a learned Kuramoto oscillator system, extending sequence models from amplitude to phase coding. Key findings:
- Only learnable parts are a low-rank coupling (exact O(N) trigonometric factorization), frequency offsets, token-injection map, and linear readout
- Phase dynamics store and retrieve symbols reliably over long horizons (99.7% recall at lag 20); capacity scales with oscillator count
- At matched parameters, OLMs consistently beat amplitude-oscillator baselines on Tiny Shakespeare, enwik8 and WikiText-2 (2.79 vs 3.78 bpc), while trailing gated RNNs
- Real cost: nonlinear phase dynamics admit no parallel scan — an N=1024 LM trains 40x slower than a cuDNN GRU
The author frames OLMs as a path toward analog/neuromorphic implementations where computation is done by physics, not arithmetic — though no physical hardware was tested.
More from Research
- Mercor Post-Trains Nemotron with Targeted APEX Data, Gains Across Law, SWE, Accounting — NVIDIAAI · 2026-10-17
- Post-training Nemotron with Era data for $609 matches frontier models at 88.9%, 27x cheaper per correct answer — NVIDIAAI · 2026-10-17
- Researchers propose a Google Scholar-like index for async LLM agents that scales — shangbinfeng · 2026-10-17
- Vals Science Index: OpenAI Models Lead Overall, but No Model Sweeps Every Science Discipline — BenBlaiszik · 2026-10-17
- First-author PhD paper frames AI text detection as continual learning, accepted at NeurIPS 2026 — lucy3_li · 2026-10-17
- Regents Labs curates 17 AI/agent science papers weekly with AI-generated deep reads — seanwbren · 2026-10-17