antirez replicates MIT weight-rewriting paper, finds visible radial patterns in first-layer weights
antirez · x · 2026-11-04
antirez (creator of redis) says he doesn't believe the MIT paper on LLMs understanding/writing weights was bad science — it's impressive and worth exploring, though the initial game-recognition tests seemed too straightforward.
So he ran his own ablation: PPO training on a simple cannon-vs-parachutes game, and the first-layer weights clearly show radial patterns in the heatmaps he posted, offering independent evidence that models learn interpretable spatial representations internally.
More from Models
- Kimi K3's compute load too heavy to serve economically, analysis argues — teortaxesTex · 2026-11-04
- Kimi K3.1 Pro leaked doc shows Gated DeepSeek Sparse Attention for long context — zephyr_z9 · 2026-11-04
- Nathan Lambert: Chinese open-model labs have never missed a weights release promise — natolambert · 2026-11-04
- Moonshot releases Kimi K3.1-Pro as open weights on Hugging Face — ApprehensiveAd3629 · 2026-11-04
- Moonshot open-weights Kimi K3.1 Pro: 2.8T-param MoE with 1M context — scaling01 · 2026-11-04
- GLM-5.3 goes into production on T-Systems' sovereign Industrial AI Cloud in Munich — jietang · 2026-11-04