Context Engineering Dictates Agent Success
alex_verem · x · 2026-07-19
This paper shifts the focus of agent failures from "model capability" to "context engineering".
The authors built an open-source harness that evaluates system prompts, tool schemas, retrieved knowledge, memory, guardrails, and untrusted inputs. It scores context across 7 dimensions: role definition, guardrail coverage, instruction consistency, tool schema quality, grounding, injection resistance, and token efficiency. Each dimension maps to specific failure modes, with multiple reviewers providing independent scores to avoid single-perspective bias.
They validated this method across 300 evaluations and 7,500 agent turns, keeping models fixed (gpt-5.5 and claude opus 4.8) while altering only the context. Poor context yielded an agent performance of 3.15 with 4.11 critical failures per evaluation; structured context boosted performance to 5.49 and reduced critical failures to 1.33—a 74% improvement and 68% reduction. A counterintuitive finding: the weakest contexts are often the cheapest and consume the fewest tokens, yet carry the highest risk.
Related event: Study: Context Engineering is Key to AI Agent Success(2 posts)→
More from coding & agent
- Plasma AI Open-Sources Fractal: A Tool for Hierarchical Agent Loops — rohanpaul_ai · 2026-07-22
- Anthropic says Claude Code helped its developers migrate 10 code packages in one month — trq212 · 2026-07-22
- Gemini 3.5 Flash Outperforms GPT-5.6 in Light Coding Tasks — Shick_hydro · 2026-07-22
- DIYing a Flight Stick into an AI Keyboard: A Hardcore Coding Agent Workflow — thorax · 2026-07-22
- Building a Secure AI Agent Gateway: Self-Hosting OAuth for Multiple SaaS Apps — Defiant_Cod_2654 · 2026-07-22
- Rowboat launches as an open-source, local-first AI coworker with memory — ycombinator · 2026-07-22