aaru publishes 2,993-question simulation eval with 7.62% mean TVD and 3.53% MAE
marcbhargava · x · 2026-09-25
Simulation startup aaru shared results of a comprehensive accuracy evaluation across 2,993 questions, reporting an industry-leading mean TVD of 7.62% and an MAE of 3.53%.
- The company argues simulation can be transformative for prediction, but only if it is demonstrably accurate — this eval is their evidence.
- The full write-up is posted on their site.
It's a rare public, quantitative benchmark for AI-driven social/behavioral simulation.
More from Research
- Psychometric profiling of nine LLMs shows score vectors can identify model identity — anselm · 2026-09-25
- Snorkel AI commits $3M to Open Benchmarks Grants funding agentic AI evaluations — typewriters · 2026-09-25
- IROS 2026 workshop puts scalable tactile sensing at the core of robot dexterity — mangahomanga · 2026-09-25
- Why PSPACE-Completeness still matters: Fortnow revisits a complexity theory classic — fortnow · 2026-09-25
- FrontierSmith lands NeurIPS spotlight: AI-synthesized coding data beats expert curation — AccBalanced · 2026-09-25
- Surya Ganguli: "Physics of Intelligence" is emerging as a new subfield of physics — irinarish · 2026-09-25