Architecture chatter: value norm and no RMS gamma in a new model
SonglinYang4 · x · 2026-11-13
ML architecture researcher Songlin Yang discusses a new model's internals, noting its interesting value norm design and—importantly—the absence of RMS gamma. A brief technical observation with limited detail.
Related event: Researchers discuss novel asymmetric norm design in new model architecture(3 posts)→
More from Models
- Gemini 4 Carbon keeps saying 'we' instead of 'I' on PostTrainBench — maksym_andr · 2026-11-13
- Inkling 2 Small loses 9 Pokemon battles to learn the opponent's strategy, then wins on the 10th try — simonguozirui · 2026-11-13
- Paying Google to use its models is still surprisingly hard, dev says — xeophon · 2026-11-13
- A developer's personal coding benchmark ranks 19 models on 21 tasks — teortaxesTex · 2026-11-13
- LLMs' mental map of SF tested: best street-level score 0.32 vs 0.23 random — Evening_Hawk_7470 · 2026-11-13
- AI routers are underrated: policy and data-retention dims matter for model choice — typewriters · 2026-11-13