Early reproduction of Kimi's Attention Residuals in a physics transformer shows faster convergence
burny_tech · x · 2026-10-27
Developer burnytech reports early success adding Kimi Team's Attention Residuals to the General Physics Transformer, with loss decreasing faster. A reproduced comparison table shows Block AttnRes cutting radiative-layer 12-step NMSE from 0.39277 to 0.28917. Same event as the fuller thread; kept for cross-reference.
More from Research
- FSPO Beats GRPO and SDPO on Reasoning With Fewer Tokens via Temporal Self-Distillation — IITK · 2026-10-27
- DiscoDemo Uses RL to Generate Diverse Robot Demos, Boosting Real-World Success from 39% to 78% — DAVIAN-Robotics · 2026-10-27
- AI Evaluator Forum Releases AEF-1: Minimum Operating Conditions for Independent AI Evaluations — RishiBommasani · 2026-10-27
- Model breeding begins: neuron combo makes Qwen believe it's Cookie Monster — yacineMTB · 2026-10-27
- EMNLP paper uses sparse autoencoders for text-based causal confounding adjustment — anjalie_f · 2026-10-27
- Volta releases Blackwell kernel verifier letting LLMs generate PTX directly, skipping compiler layers — Azaliamirh · 2026-10-27