DeepMind researcher finds Kimi 3.1 Pro strong on benchmarks but weak in a real test
SamuelAlbanie · x · 2026-10-25
DeepMind researcher Samuel Albanie shared a mixed first impression of Kimi 3.1 Pro: on a moderately hard prompt asking for benchmark-based visualizations to estimate the China-vs-West frontier AI gap, the model performed poorly — despite spending roughly two hours reasoning.
He caveats the sample size of 1, noting the model looks very strong on public benchmarks and he's keen to see others' reactions.
More from Models
- Self-host open models or risk bans: why 'safety-first' labs make own-hardware the safer bet — gnukeith · 2026-10-25
- CrowdGPT: a crowd-trained, datacenterless 1B open-source LLM, 6.9B tokens done — Vxtzq1 · 2026-10-25
- Miles Brundage asks why Argon isn't available in Google's Antigravity — Miles_Brundage · 2026-10-25
- Apples-to-apples rerun: DeepInfer tops DeepSeek's published throughput by ~9% on 4x B300 — charles_irl · 2026-10-25
- Grok UI shows image icon, hinting at upcoming image upload feature — faraz0x · 2026-10-25
- Polymarket odds put Kimi K4 release below 9% by end of 2026 — Polymarket · 2026-10-25