EliseAI handles 5M calls a month: why real-world AI beats benchmarks in difficulty
a16z Podcast · rss · 2026-10-18
a16z podcast with EliseAI Head of Applied Research Mario Martone on why deploying AI in housing and healthcare demands far higher reliability than benchmark scores.
Key points:
- EliseAI processes roughly 5 million phone calls per month, where mistakes affect people's housing and healthcare
- LLMs aren't the right tool for every problem: the team combines language models, traditional ML, neural networks, and internally built evals to decide what works in production
- New models are tested against EliseAI's own benchmarks; open-source models are fine-tuned to automate increasingly complex workflows
- Martone, with 15 years of theoretical physics training, argues real-world AI problems can be as intellectually hard as string theory
More from AGI Musings
- Free models let independent researchers outpace AI labs in their own fields — gerardsans · 2026-10-18
- From company brains to company harnesses: a shift in enterprise AI thinking — jacob_posel · 2026-10-18
- Paper on hidden pitfalls of autonomous AI scientists tops PNAS weekly charts — Dr_Atoosa · 2026-10-18
- François Fleuret: In the AI era, the only freedom left is making your own suboptimal decision — francoisfleuret · 2026-10-18
- AI nerds cheer automating everything, then mourn when entry-level coding gets automated — Aryvyo · 2026-10-18
- Tim Dettmers: 80% of students fear joblessness as academia gears for a renaissance — code_star · 2026-10-18