Even in 2026, LLMs can't reliably extract verbatim quotes from transcripts
dylfreed · x · 2026-11-06
Dylan Freedman (Google) claims that in 2026, LLMs still cannot reliably extract a verbatim substring from a transcript without mutating it — not what skilled operators can achieve with structured outputs and programmatic checks, but what happens to normal people assuming chatbots are more reliable than they are. He cites multiple real 2026 incidents as evidence, a key warning for journalism and legal use cases requiring exact quotes.
More from Models
- FLUX 3 models beat FLUX.2 in all 19 Text-to-Image categories; Klein tops $9/1k price frontier — ArtificialAnlys · 2026-11-06
- Anthropic chief architect Sam McCandlish's first post hints at internal-acceleration report, possibly hitting 2x RSP threshold — inductionheads · 2026-11-06
- 6,662 decisions benchmarked: decision model Mammoth-1 averages 80.9%, beats best small LLM by ~3 points — pournasserian · 2026-11-06
- Hesamation: OpenAI Models Beat Anthropic's Only at Certain Kinds of Games — Hesamation · 2026-11-06
- Meta's new 'walrus' checkpoint reportedly matches crouton-alpha quality — ChrisGPT · 2026-11-06
- Quick test: Gemini 3.1 Pro knows the answer in its weights, Flash doesn't — giffmana · 2026-11-06