ASCIITermDraw-Bench tests whether VLMs can generate and edit diagrams in ASCII
East-Muffin-6472 · reddit · 2026-07-22
A new benchmark, ASCIITermDraw-Bench, evaluates whether VLMs and LLMs can generate and edit diagrams in plain ASCII.
- The author argues ASCII and Mermaid are the two reliable channels for communicating initial architecture ideas between humans and AI assistants.
- The benchmark focuses on ASCII generation/editing across architecture sketches, cluster/topology diagrams, and image-conditioned edits.
- It includes 80 tasks in four groups: basic boxes and layouts, network topologies, software architecture diagrams, and image-conditioned editing.
- Scoring combines a structural check and an LLM-based semantic judge, run five times per task, with 95% confidence intervals over the final score.
- The current leaderboard is topped by Gemma-4-31B-IT at 73.8% ± 4.1, followed by Qwen3.7-Plus at 70.2% ± 4.6.
- Twelve examples and the methodology are published on Hugging Face.
Related event: ASCIITermDraw-Bench: Evaluating AI's Ability to Generate and Edit ASCII Art(2 posts)→
More from Research
- Weights & Biases’ interactive confusion matrix gets praise and a marimo comparison — _ScottCondron · 2026-07-22
- LTX2.3 PoleDance LoRA MK6 shows better results at step 1,000 of 12,000 — JahJedi · 2026-07-22
- Video lectures cover physics-informed machine learning for modeling, control and estimation — Vjeux · 2026-07-22
- New health-time-series benchmark shows LLMs lag behind classic ML baselines — yang_yuzhe · 2026-07-22
- AutoLab benchmark shows frontier models win long-horizon tasks by persisting, not guessing — rohanpaul_ai · 2026-07-22
- Report says Gemini Flash keeps parity while cutting per-task cost by 30–40% — sujingshen · 2026-07-22