Study finds most accurate models are not always the most preferred by users
soumitrashukla9 · x · 2026-08-27
A highlight of work led by Mina Lee finds that the most accurate models are not always the ones human users prefer the most.
For instance, Opus and Sonnet are great at performing tasks on their own but not in providing assistance, while GPT-5-Mini was strong in both dimensions. Gemini models were found to be stronger as assistants than as automators.
Related event: CentaurBench: The Strongest Models Aren't Always the Best Assistants(3 posts)→
More from Research
- Priming Models: Teacher-Free One-Step Generation Hits FID 1.19 With 12x Less Compute — du_yilun · 2026-10-15
- vLLM Achieves Bitwise-Consistent RL Training, Fixing KL Blowups and Score Collapse — vllm_project · 2026-10-15
- David Deutsch: Proofs Are No Longer a Useful Proxy for Mathematical Progress — francoisfleuret · 2026-10-15
- STEMBench launches: 6 STEM domains, zero full-target successes across all runs — mariofilhoml · 2026-10-15
- OpenAI gives $5M to Simons Foundation's $15M push to help mathematicians navigate AI-era results — OpenAI · 2026-10-15
- Using AI to reverse engineer hardware: a path to faster open-source drivers — ErenayDev · 2026-10-15