Inkling Small 2 evals: 97% flags flakiness, never implies causality
keirp1 · x · 2026-11-20
Thinky Machines says its non-frontier Inkling Small 2 punches near Fables and Astra on multi-step spatial reasoning and bespoke languages. Internal evals highlight its discipline: it flags flakiness 97% of the time, never implies causality, advises evidence-checking, drops useless steps by generation 5+, and never ritualizes behavior it wasn't shown (though only 89% performs the check when actually needed).
More from Models
- Apple releases Depth Pro 2: fast monocular depth for photos and video at any resolution — pcuenq · 2026-11-20
- Creating good evals may be the most important post-training job in coming years — joecole · 2026-11-20
- Voice receptionist demo shows GPT-Live-1.1 booking appointments and escalating urgencies — amos_gyamfi · 2026-11-20
- Leaked: X Preparing Black Friday Discount Bundling Premium with Grok and Cursor — nima_owji · 2026-11-20
- Daily AI brief: Anthropic +17% on Nasdaq debut, Genie 4 opens to developers — testingcatalog · 2026-11-20
- SenseNova-Loop 9B runs on 8GB VRAM at 22 t/s, uses half the tokens of Qwen3.8 — R_Duncan · 2026-11-20