Audit picks the sycophant model 3/3 times, but "telling apart isn't explaining"
kevrussell · x · 2026-11-03
Replying to Neel Nanda: the author's audit of released weight files found that, based on attention patterns, Astra picked the sycophantic model in 3 of 3 pairs. But head 10.2's pattern shifts with context, and released patch tests show zero closure of the agreement gap. Key point: "Telling models apart isn't explaining them."
More from AGI Musings
- Nearly 50% of new Steam games in October disclosed AI-generated content, up from 1.7% — chrisfirst · 2026-11-03
- Goldman bans Claude for traders, AI secretly sways expert decisions: a looming markets question — oran_ge · 2026-11-03
- FT's Burn-Murdoch: productivity spike predates AI and only shows up in the US — koltregaskes · 2026-11-03
- DeepSeek's 'super agent' sparks debate: is the meat proxy era over? — gdechichi · 2026-11-03
- Dean Ball: RSI will come from scaling mundane-but-difficult techniques beyond imagination — deanwball · 2026-11-03
- Fine-grained brain simulation would deserve the same moral rights as humans — dioscuri · 2026-11-03