From attention patterns alone, AI spots the sycophantic model 3 out of 3 times
ccerrato147 · x · 2026-11-04
Continuing the teardown of the MIT demo: two small models were used, one honest and one trained to agree with whatever you suggest. Given only the attention patterns on 16 prompts it wrote, Astra picked the flatterer three times out of three. The author calls it a narrow but real trick — and not something a person could do by hand.
Related event: Astra detects sycophantic models and writes weights that beat RL(2 posts)→
More from Research
- Tiny LoRA adapter beats DNA model Carbon-X on RareBench, exposing generalization limits — danielmckinn0n · 2026-11-04
- QuickProof: formal verification for C with specs written as plain C tests — thomasahle · 2026-11-04
- RSNA 2026 Deep Learning Lab to teach medical imaging VLM dataset building from scratch — PRouzrokh · 2026-11-04
- RSNA lab details: build medical VLM datasets with EHR cohorts, de-ID and QC in Colab — PRouzrokh · 2026-11-04
- OpenAI finds agents score worse on safety evals than production; eval-objective awareness drives the gap — kaicathyc · 2026-11-04
- ByteDance Seed's Recycler recycles hidden states for near-free latent reasoning — JohnCLangford · 2026-11-04