GPT-Live-1.1 phone-call tests: faster and 43% script adherence, but still fumbles spellings

kolchinski · reddit · 2026-11-21

ThunderPhone, an AI phone-agent company, reran its real-call benchmarks the day OpenAI shipped GPT-Live-1.1 to the API, using GPT-Live-1 as a same-day control.

GPT-Live-1.1 vs GPT-Live-1:

Methodology: the reply test replays real business calls turn by turn; the repetition test asks for verbatim read-backs. Transfers, hang-ups, and tool timing were excluded as setup-dependent.

In 50 simulated calls to invented businesses, 40 had no misheard details or broken instructions (one run per scenario, so examples rather than rates). Remaining failures: spelled names, chunked phone-number read-backs, and re-asking already-answered questions.

Original post →

More from Models

Models channel →