Control test with zeroed weights debunks claim that models read trained weights
kevrussell · x · 2026-11-04
kevrussell reports that a much-discussed experiment does not show models reading trained weights. Every fix that worked still worked with the trained weights zeroed out—the models wrote their own small standalone controller, and given the same task without trained weights Opus did at least as well (5/5 vs 4/5). Sample size: Opus 5 replies per arm, Astra 3 per arm (one server-error call, rest not run), one test scene. Audited by honesty.tools, an independent research observatory for representation inspection and control work.
More from Models
- DanielMiessler: direct weight editing achieves fine-tuning results in minutes, vastly underrated — JensHonack · 2026-11-04
- OpenAI's internal model reportedly solving problems humans couldn't — imjustnewatai · 2026-11-04
- Artificial Analysis ships CritPt v2 grading server, regrades all models with max 1-point shifts — ArtificialAnlys · 2026-11-04
- Opus 5.5 hailed as first model that 'disappears', rumors of Opus 5.6 surface — haider1 · 2026-11-04
- LLM 'fixes' to a driving network work even with weights zeroed: model wrote its own controller — kevrussell · 2026-11-04
- Leaker hints: no "Fable 5.5" or "GPT-6.2 Astra" today — but Thursday is the date — daniel_mac8 · 2026-11-04