Chartography: A New Benchmark for Pro Chart Understanding
echen · x · 2026-07-17
Existing chart benchmarks (like ChartQA) are nearing resolution on basic charts (80-90% accuracy) but fail to reflect the challenges of real-world professional scenarios.
The research team introduced Chartography, a benchmark designed specifically to test frontier models on understanding professional charts (e.g., oncologist survival curves, contour maps, candlestick charts, Sankey diagrams). Test results show that even the best-performing models score below 45%. The authors note that models often experience total computational collapse due to misreading grid lines, and excessive reasoning only prolongs these errors.
Related event: Chartography: A New Benchmark for Professional Chart Understanding(3 posts)→
More from Models
- Only 2 of 25 full ChatGPT rewrites flagged 100% AI by detectors — paulnovosad · 2026-10-26
- Qwen 4 Reportedly Spotted in the Arena, Hints at Alibaba's Next-Gen Model Testing — AIFlow_ML · 2026-10-26
- Nathan Lambert's first open models lecture: distillation, risks and the ecosystem explained — natolambert · 2026-10-26
- Nanbeige 4.3-33B flops real-world test: zero compilable output vs Qwen — julianharris · 2026-10-26
- Claude's safety classifier blocks SS7 defense research, pushing a security dev to an abliterated GLM — l4rz · 2026-10-26
- Black Forest Labs open-sources 5.6B FLUX 3 Image Klein, +91 Elo with 38% fewer params — dl_weekly · 2026-10-26