Persona prompt pins models at ~35% on SIMMER benchmark regardless of Claude or DeepSeek
amplifiedamp · x · 2026-10-29
- A core persona prompt called "Annette" appears to cap model performance at 35% on SIMMER, a cooking benchmark, regardless of the underlying model — Claude or DeepSeek.
- A chart of 2,200 confirmation trials shows DeepSeek official profile at 36.7% (220/600), Claude Code Fable 5.1 and Opus 5.5 profiles both at 35% (35/100 each), and GPT-6.1 Sol profile at only 19%.
- Neutral prompts (green points) score anywhere from 17% to 53%, suggesting the persona layer dominates and flattens model differences.
- Chart caveats: unadjusted 95% cluster-bootstrap intervals (overlap ≠ paired significance), Claude Code includes native fallbacks/advisors, and Opus is parser-sensitive.
More from Models
- ChatGPT Pro/Business subscribers can now claim up to $500/month in API credits — JeremyNguyenPhD · 2026-10-29
- hunch: GPU-free ~20ms Jev-like decisions built on EmbeddingGemma 2 — iamrobotbear · 2026-10-29
- Rumor: DeepSeek's official API will strip cyber capabilities pending regulatory audits — teortaxesTex · 2026-10-29
- Vitalik: AI is hyper-knowledgeable but unwise, giving surface-level readings of new texts — paulnovosad · 2026-10-29
- No fresh local-capable open models lately — is Qwen 27B still the best? — emax · 2026-10-29
- Leaker says next model after Carbon is ready, waiting on OpenAI and Anthropic — iruletheworldmo · 2026-10-29