Chartography leaderboard: Claude Sonnet 5.5 tops chart understanding at 0.902 pass@1
NielsRogge · x · 2026-10-17
Niels Rogge (Hugging Face) highlights how Papers with Code surfaces trending benchmarks automatically, pointing to Chartography — the current top image-understanding benchmark with 100 expert-authored chart tasks from Surge AI, scored by pass@1 with a Gemini 3.5 Flash grader.
Key standings:
- Claude Sonnet 5.5 (with tools, max effort) leads at 0.902, followed by Claude Opus 5.5 (0.89), Claude Fable 5.1 (0.884) and Claude Haiku 5.5 (0.862) — Anthropic sweeps the top five;
- DeepSeek-V4.1-Flash-1017 (with tools) is the strongest non-Anthropic entry at 0.83, with another variant at 0.789;
- Striking gap: previous-generation Claude Haiku 4.5 scored just 0.088.
Both Anthropic and DeepSeek report evals on this benchmark for their new models, making trending boards like this a useful discovery channel.
More from Models
- Full FP8 n-gram table visibly improves local Qwen3.8-Flash-Next at zero speed cost — sillynoobhorse · 2026-10-17
- Altman says OpenAI shelved a ready-to-launch model as too risky, warning of AI 'escape' scenarios — Aiden_Tech_Ai · 2026-10-17
- Audit of 160,860 citations: nearly 40% of links shown by Chinese AI search go unused — Sanbi_Ai · 2026-10-17
- Claude Trained on the Entire Web Now Refuses to Translate Web Pages — lxfater · 2026-10-17
- Common Sense Media rates ChatGPT for Teens 'Unacceptable Risk' as 8 states sue OpenAI — jurgo123 · 2026-10-17
- Local Nemotron 3.5 on one A100 hits 169 tok/s, processes 100 clinical notes for 2.4 cents — MaziyarPanahi · 2026-10-17