PlotGround benchmark: best multimodal model hits 87.5% at ±5% reading scientific plots
james_y_zou · x · 2026-10-20
James Zou's team at Stanford introduced PlotGround, an automated pipeline that builds plot digitization benchmarks from real scientific figures paired with author-released source data. It yields PlotGround-1k: 1,119 human-verified questions drawn from 1,066 bioRxiv preprints.
Evaluating 16 multimodal models, the best reaches 87.5% accuracy at ±5% relative-error tolerance — but tightening tolerance to ±2% drops every model by 11-24 percentage points, exposing the gap between approximate visual reading and precise quantitative recovery.
The paired figure-source structure also enables comparing how accurately the same values are recovered from figures versus source tables. Paper and dataset are public.
More from Models
- MiniMax M3.1 Flash accused of benchmaxxing on Artificial Analysis' private Omniscience bench — teortaxesTex · 2026-10-20
- From memory capacity to transformers: outer-product stores cap out near 2×key dimension — burny_tech · 2026-10-20
- Reddit users slam Google's shrinking Gemini usage limits, suspect compute shortage — JesseBell935 · 2026-10-20
- Training a LoRA locally in 30 minutes? Veteran says you're probably lying — the_bollo · 2026-10-20
- Gemini 4 Argon finds 74% of known vulnerabilities, top recall on dfbench — andreamichi · 2026-10-20
- Frontier models already saturate persimmon, a tiny human-distribution benchmark — noahdgoodman · 2026-10-20