Structured Output ≠ Usable Results
mattjcoles · reddit · 2026-07-14
This article discusses how structured output only guarantees the "shape," not the "content", and therefore cannot be directly used to decide whether to merge code.
The author's approach involves layering multiple components:
- Using Pydantic AI on Bedrock for structured output
- Using Pydantic evals for evaluation
- Pairing it with a calibrated LLM judge
The goal is to turn model outputs into results that can genuinely "gate merge," rather than merely checking if the JSON format is correct.
More from coding & agent
- Devin adds e2b sandboxes for remote agent execution — badphilosopher · 2026-07-22
- Hermes Agent Refactoring Proposal: Decoupling via Event Bus and Monorepo Slicing — Promptmethus · 2026-07-22
- ty now reads Pydantic config keywords and field metadata — charliermarsh · 2026-07-22
- Pensar Launches AI Security Agent to Autonomously Discover and Patch 0-Days — andriy_mulyar · 2026-07-22
- ty adds first-class Pydantic support, including strict and lax field handling — charliermarsh · 2026-07-22
- Google launches Gemini 3.5 Flash Cyber for CodeMender, with limited access for governments — GoogleAI · 2026-07-22