Chartography: A New Benchmark for Pro Chart Understanding
echen · x · 2026-07-17
Existing chart benchmarks (like ChartQA) are nearing resolution on basic charts (80-90% accuracy) but fail to reflect the challenges of real-world professional scenarios.
The research team introduced Chartography, a benchmark designed specifically to test frontier models on understanding professional charts (e.g., oncologist survival curves, contour maps, candlestick charts, Sankey diagrams). Test results show that even the best-performing models score below 45%. The authors note that models often experience total computational collapse due to misreading grid lines, and excessive reasoning only prolongs these errors.
Related event: Chartography: A New Benchmark for Professional Chart Understanding(3 posts)→
More from Models
- Gemini 3.6 Flash goes live in Antigravity with 17% fewer output tokens — rseroter · 2026-07-22
- Moonshot’s Kimi K3 sets a new open-weights ECI record at 156 — scaling01 · 2026-07-22
- Nanbeige4.2-3B launches as a 3B Looped Transformer model that beats larger baselines — Wooden-Deer-1276 · 2026-07-22
- A post says six companies now beat Google’s best LLM, including two open-source models — soham_btw · 2026-07-22
- Gemini 3.6 Flash benchmark results reignite concerns that Google is slipping behind — minxio_ · 2026-07-22
- Google says Gemini 3.5 Pro is in testing and Gemini 4 is already pre-training — Wide-Ad1564 · 2026-07-22