Dev: LLMs seem stagnant or worse on semi-specified math problems, citing disappointing Astra
LucaAmb · x · 2026-11-17
Luca Ambrosio reports that for semi-specified math problems, models don't seem to be improving and may even be getting worse.
- He was quite disappointed by Astra: it made an absurdly incorrect assumption, then proceeded as if that assumption were the original problem definition
- A first-hand observation of apparent behavioral regression on math reasoning tasks
More from Models
- Mistral makes Vibe free in Europe for 3 months, claims coding edge over GLM and DeepSeek — qtnx_ · 2026-11-17
- Study: 9 LLMs can write neural game agents, but they act like hand-coded programs — burkov · 2026-11-17
- Cheap tokens, expensive agent: Flash hits $11.97/task vs Luna's $0.05 in Databricks AI Index — krishnan · 2026-11-17
- Rumor: Gemini 4 to use element names, four tiers mapped to Claude lineup — koltregaskes · 2026-11-17
- Musk: Grok 4.8 rolls out this week after full C/C++ stack rewrite — iamfakhrealam · 2026-11-17
- Paper: Claude edits model weights directly, lifting Qwen3.5-0.8B MATH-500 by 51.4 points — udmrzn · 2026-11-17