Introducing ASCIITermDraw Bench: Testing VLMs on ASCII Architecture Diagrams
East-Muffin-6472 · reddit · 2026-08-04
A developer introduced ASCIITermDraw-Bench, a new benchmark specifically designed to evaluate the ability of Vision Language Models (VLMs) to draw and edit diagrams using plain text (ASCII).
Dimensions & Evaluation:
- Features 80 tasks covering basic layouts, network topologies, software architecture diagrams, and image-conditioned diagram editing.
- Evaluation combines a structural score (verifying labels, edges, and entity relationships) with a semantic score from an LLM judge (evaluated 5 times per task to reduce variance).
Current Leaderboard:
- Gemma-4-31B-IT: 73.8%
- Qwen3.7-Plus: 70.2%
- Kimi-K2.6: 61.8%
- MiniMax-M3: 59.5%
More from Research
- OpenMed Plans to Fine-Tune Qwen3 27B into the Best Local Medical AI Model — MaziyarPanahi · 2026-08-04
- Stanford & Big Tech Consensus: Why Even Infinite Compute Won't Kill RAG — blaizedsouza · 2026-08-04
- NeurIPS 2026 Competition: Build AI Agents for Bargaining and Persuasion — Old_Station_4584 · 2026-08-04
- Microsoft Researcher Discusses RL: Why Does the Industry Only Focus on Positive Reinforcement? — gerardsans · 2026-08-04
- AI Models Can Guide Brain Microstimulation to Alter Primate Behavior — dyamins · 2026-08-04
- Open 'Bindome' Database Releases 300k+ Protein Binders for 8k Targets — jueseph · 2026-08-04