GPT-Live-1.1 phone-call tests: faster and 43% script adherence, but still fumbles spellings
kolchinski · reddit · 2026-11-21
ThunderPhone, an AI phone-agent company, reran its real-call benchmarks the day OpenAI shipped GPT-Live-1.1 to the API, using GPT-Live-1 as a same-day control.
GPT-Live-1.1 vs GPT-Live-1:
- Median time to start talking: 1.0s vs 1.3s
- Gives the business-approved reply: 43% vs 34% (205 moments from real calls)
- Repeats caller speech exactly: 61% vs 56%; 75% vs 70% on the 115 clips containing numbers
Methodology: the reply test replays real business calls turn by turn; the repetition test asks for verbatim read-backs. Transfers, hang-ups, and tool timing were excluded as setup-dependent.
In 50 simulated calls to invented businesses, 40 had no misheard details or broken instructions (one run per scenario, so examples rather than rates). Remaining failures: spelled names, chunked phone-number read-backs, and re-asking already-answered questions.
More from Models
- ChatGPT's PC use only works in new chat sessions, fails silently in old ones — LivingHighAndWise · 2026-11-21
- Researcher flags Depth Pro 2 border artifacts: matting vs depth confusion, RL could fix it — giffmana · 2026-11-21
- Qwen 27B quant runs on 8GB VRAM: 200k context, 92.5% Top-1 vs BF16 — Brief-Tap-6616 · 2026-11-21
- Rumor: DeepMind rebuilds Gemini architecture every 6 months with 3 parallel teams, ~6 releases a year — DeryaTR_ · 2026-11-21
- Google launches Gemini 4 Hydrogen: a speed-focused everyday model now live in Gemini app and Search — GeminiApp · 2026-11-21
- CCORE-Web-Coder-4B preview: 4B local model builds sites as single HTML — ConfidentDinner6648 · 2026-11-21