Structured Output ≠ Usable Results
mattjcoles · reddit · 2026-07-14
This article discusses how structured output only guarantees the "shape," not the "content", and therefore cannot be directly used to decide whether to merge code.
The author's approach involves layering multiple components:
- Using Pydantic AI on Bedrock for structured output
- Using Pydantic evals for evaluation
- Pairing it with a calibrated LLM judge
The goal is to turn model outputs into results that can genuinely "gate merge," rather than merely checking if the JSON format is correct.
More from coding & agent
- Spring AI 2.1.0-RC1 adds tool-call confirmation, pausing agent loops before real-data access — therealdanvega · 2026-10-26
- 250 Builders Hack With Codex at Berlin Community Hackathon Ahead of OpenAI DevDay — paw_lean · 2026-10-26
- Building AI bots on raw APIs? Devs say models are 'dumb without the harness' — madeofAjala · 2026-10-26
- HarnessMM cuts agent tokens per task by up to 88% while raising success rates — omarsar0 · 2026-10-26
- Microsoft demos intent-classifying agents built with Copilot App and Foundry, deployable across Teams and Outlook — marlene_zw · 2026-10-26
- Looking for a wearable mic with best battery life that feeds recordings to personal agents via MCP — altryne · 2026-10-26