COLM 2026 Accepts Multiple Papers on SSMs and LM Architectures
realDanFu · x · 2026-07-17
A research team announced that 4 of their papers have been accepted by COLM 2026, focusing on architectural innovations for efficient language models and State Space Models (SSMs): - **Parcae: Scaling Laws for Stable Recurrent Language Models**: Treats layer recurrence as a dynamic system to build stable recurrent LLMs, offering a new predictable scaling axis under constant memory. - **Exploring Document 'Soupability' in State Space Models**: Uses Mamba2 to encode documents separately, then averages their SSM states into a 'soup'. This method achieves better multi-document QA at extremely low inference costs and scales to handle 256 documents. - **Format Tax**: Research on measuring and mitigating the impact of formatting on model performance.
More from Research
- Perfect task routing beats the best single model by 15 points in pass@1 — ZainHasan6 · 2026-07-21
- A Matrix meme turns an LLM-solved-problems debate into a question of belief and access — prasanna_says · 2026-07-21
- More compute can materially improve frontier models’ cyber benchmark performance — peterwildeford · 2026-07-21
- Paper claims stochastic exploration fixes two 3D Gaussian Splatting optimization bottlenecks — zhenjun_zhao · 2026-07-21
- SSR refines monocular geometry with sparse volumetric updates and sparse 3D U-Nets — zhenjun_zhao · 2026-07-21
- Survey of 300+ papers says better reasoning does not make LLMs more self-aware — blaizedsouza · 2026-07-21