DeepSeek V4.1 Flash 1017 doubles ARC-AGI-3 score to 5.81%, leads open models
teortaxesTex · x · 2026-11-03
ARC Prize reports DeepSeek V4.1 Flash 1017 on ARC-AGI (Verified): semi-private ARC-AGI-3 score doubled 2.68% → 5.81% (public fell 11.05% → 8.97%). It leads open-weight models on ARC-AGI-3 (5.81%) and ARC-AGI-2 (78.1%), ties best open score on ARC-AGI-1 (94.5%). Cost: $0.30/M input, $1.20/M output (Baseten pricing); it used 36-48.6% fewer first-action reasoning tokens than the original. Community notes: on par with GPT-6 Sol (High); Haiku 5.5 (Max) hits 2.8%, while 6 Sol jumps to 23% at max reasoning and 6.1 nearly saturates the benchmark.
More from Models
- Qwen3.5 "weight editing" hype debunked: it's just a steering vector plus trial and error — akbirthko · 2026-11-03
- MIT CSAIL shows GPT-6 editing another model's weights by hand, boosting math accuracy with a 0.016% edit — kevrussell · 2026-11-03
- Reproduced GPT patch passes sycophancy test 3/3, but MATH-500 still drops 93% to 33% — kevrussell · 2026-11-03
- Audit update: MATH-500 truncation errors drop from 93% to 33%, but looping persists — kevrussell · 2026-11-03
- Audit of 64,222 labelled answers confirms claimed eval gains: MATH-500 7.9% to 48.1% — kevrussell · 2026-11-03
- Apple reportedly ships a secret multimodal node-graph editor with every iOS copy — round · 2026-11-03