Researchers discuss novel asymmetric norm design in new model architecture

Researchers including Songlin Yang discuss a new model's asymmetric design: QK-norm retains learnable RMS gamma while V-norm omits it, with its split q/k/v attention residual also drawing praise.

2026-11-13 ~ 2026-11-13 · 3 related posts

• Architecture chatter: value norm and no RMS gamma in a new model — SonglinYang4 · 2026-11-13
• New architecture's asymmetric QK-Norm with gamma but no V-Norm gamma draws attention — stochasticchasm · 2026-11-13
• Q/K norm gets gamma but V norm doesn't: attention residual split draws praise — stochasticchasm · 2026-11-13