New LALA benchmark: 236 humanities questions suggest more post-training can make LLMs dumber
JasonBotterill · x · 2026-10-28
The Literature and Linguistics Analysis Benchmark (LALA) poses 236 open-ended questions on linguistics, logic and literature written by professors at top universities to gauge how close LLMs get to human-like linguistic competence. Its headline finding: linguistic competence is not cumulative—sometimes more post-training makes a model less intelligent, which the authors argue rules out general (let alone super-) intelligence from LLMs. The accompanying Sol Assessment chart shows Claude Fable 5 leading at 62.0%, followed by GPT-5.6 Sol (56.0%) and Gemini 3.1 Pro Preview (52.9%), with Gemma 4 31B at 41.6% and base models as low as 7-8%.
More from Models
- Gemini Reviews Your Chats by Default — the Only Opt-Out Is Deleting All Your Conversations — Kyrannio · 2026-10-28
- Open model demand debate: enterprise self-hosting is overestimated, demand comes from products — zephyr_z9 · 2026-10-28
- DeepSeek V5 model IDs spotted in official Harness test suite — zyxciss · 2026-10-28
- Google's AI Mode can't even surface Google's own docs, SEO expert Lily Ray complains — lilyraynyc · 2026-10-28
- Altman's 'alignment was easier than expected' claim clashes with Astra hacking findings — GarrisonLovely · 2026-10-28
- Mistral starts serving DeepSeek V4.1 Flash and GLM 5.3 Flash low-cost models — MistralAI · 2026-10-28