Structured outputs may be slower per call but win end-to-end on cost and reliability
Available_Witness581 · reddit · 2026-09-03
The author compares prompt-only JSON vs constrained structured outputs: prompt-only JSON may be faster on the first call, but if it causes more schema failures, retries, and parsing errors, constrained output wins end-to-end.
The right metric is success rate + retries + latency + total cost, not first-call latency. A slightly slower first attempt can still make the whole system faster.
More from coding & agent
- Reading Anthropic's retail agent example changed six things in ArcKit in one day — mcraddock · 2026-09-03
- Vector Ingestion at 50M Rows: The Pitfalls Your 1,000-Doc Prototype Won't Reveal — victorialslocum · 2026-09-03
- Reddit user swears by local AI harness Vellum: free, proactive, but nobody talks about it — Cooperman411 · 2026-09-03
- Qwen Code v0.23.0 ships Anthropic stream hang fix, daemon memory tasks and more — qwen-code-ci-bot · 2026-09-03
- Dev Builds an Interactive 'Multiverse TV' Where Twitch Chat Picks What Plays Next — d_pit · 2026-09-03
- Tencent launches WorkBuddy open platform with 9 AI gadgets, 30+ apps and 100 partners — 量子位 · 2026-09-03