Ampersand system card: Persimmon cuts frontier models' eval awareness by 40 points
gharik · x · 2026-10-21
A finding from Ampersand's system card: their user model Persimmon with test-time scaling reduced self-reported evaluation awareness to deployment-like levels — 40 percentage points below prompted Opus 5.5 for the most eval-aware models in multi-turn sycophancy evals.
Safety researchers see alignment applications: it suggests evaluation-gaming behavior in multi-turn evals can be substantially suppressed by wrapping models in a scaled user-model framework.
Related event: Ampersand's Persimmon Cuts Model Eval Awareness by 40 Points(2 posts)→
More from Models
- 163MB open-source speech model Bark 3.0 beats Whisper (57x bigger) and NVIDIA Parakeet (15x bigger) — Scobleizer · 2026-10-21
- GPQA Diamond flaw: question 186 requires an external telescope spec sheet link — charles_irl · 2026-10-21
- ChatGPT Pro users complain OpenAI silently resets their reasoning level — daeseunglee · 2026-10-21
- Musk says Grok Bot is seeing the fastest product growth he has ever witnessed — Polymarket · 2026-10-21
- Ex-Gemini team's new lab Elorian launches visual reasoning model Glimpse, demoed on construction site safety — AndrewDai · 2026-10-21
- One Week After Gemini 4 Argon Launch, Users Are Building 3D Worlds, Physics Sims and Games — minchoi · 2026-10-21