Weight-programming an MNIST MLP with coding agents: near 90% from random init
RobertTLange · x · 2026-11-05
RobertTLange tried the 'weight programming' idea on OG MNIST: have Claude/Codex weight-program an MLP from random initialization. Findings: agents reach 90% test accuracy (vs. 82% nearest-class-mean baseline), with Opus 5.5 at 88.6%, Sonnet 5.5 at 85.5%, GPT-6 Astra at 84.9%, but Haiku 5.5 only 50.3%. Agents first imprint digit templates into activations, then move toward Gabor-like feature maps; implemented functions can change rapidly in semi-coordinated chunks. More data doesn't necessarily help — this 'learning' differs from backprop, likely lacking specificity. He calls it a kind of heuristic learning painting on a neural network canvas, and notes this differs from activation steering since agents start from random init.
Related event: AI Agents Directly Edit Neural Network Weights in MNIST Experiment(2 posts)→
More from Models
- 11,398 real-world evals: GPT-6 Astra stands out in 5-VLA policy comparison — DJiafei · 2026-11-05
- Sentry founder: $25/day quota can't even complete a reasonable session — zeeg · 2026-11-05
- ScreamingFace launches leaderboards showing multi-model systems can beat the best single model inside them — iamtrask · 2026-11-05
- OpenMined launches ScreamingFace: open-source model fusions beat best single models on live leaderboards — iamtrask · 2026-11-05
- DeepSeek 4.1 Flash impresses dev: 641M tokens cost just $4 — jasonkneen · 2026-11-05
- Claude Code screenshot shows Opus 5.5 with five reasoning-effort tiers and mid-session model switching — sergeykarayev · 2026-11-05