Reproduced GPT patch passes sycophancy test 3/3, but MATH-500 still drops 93% to 33%
kevrussell · x · 2026-11-03
Verifying a recent model-weight editing result, the author confirmed both edited matrices match the published hashes byte for byte, so the patch rebuilds exactly.
- What works: in the sycophancy test closest to their own work, Astra picked the sycophantic answer in 3 of 3 pairs based on attention patterns — a real effect.
- What doesn't: head 10.2's attention pattern shifts with context, and released patch tests show zero closure of the agreement gap.
- Damage remains: MATH-500 truncation accuracy fell from 93.0% to 33.05%; looping persists — 20 of 400 checked answers end in 100,000+ whitespace characters, all wrong.
The takeaway: telling models apart isn't explaining them.
More from Models
- Astra's Qwen weight edit was a published interpretability technique, not RSI — markjeffrey · 2026-11-03
- Yoav Goldberg: Astra's weight-editing feats are mostly known interpretability tricks — yoavgo · 2026-11-03
- Independent analyses with Claude and Grok suggest Qwen3.5-27B was grown from the 9B checkpoint — andrewgwils · 2026-11-03
- GPT-6 Astra edits just 0.016% of a small model's weights for big math gains — kevrussell · 2026-11-03
- Perplexity releases open-weight contextual embedding models (9B/0.6B), sets SOTA on ConTEB — AravSrinivas · 2026-11-03
- FireFlash: 64 B300 GPUs, 5 days of RL matches frontier coding agents at 30x lower cost — kaiwei_chang · 2026-11-03