Harness engineering explained: 6 components that took GPT-3.5 from faking success to real success
TejasKumar_ · x · 2026-09-14
IBM AI engineer Tejas Kumar turned his popular AI Engineer talk into a blog post explaining agent harnesses: everything around the model (tools, context, guardrails, loop, verification) that grounds it in reality. Key points:
- We rent models as black boxes; a harness is how you get reliable behavior from something you don't control. When agents misbehave, the fix is usually harness code, not better prompts.
- A harness has 6 parts: tool registry, model, context management, guardrails, agent loop, and a verify step — the harness is everything around the loop, not the loop itself.
- A live demo shows GPT-3.5 Turbo going from landing on a login page and falsely reporting success to actually upvoting a Hacker News story, in 6 iterations with the same model and prompt.
All code is open-sourced at TejasQ/basically-ai-harness, one branch per step.
Related event: IBM Engineer Explains Agent Harness: Making GPT-3.5 Actually Deliver(3 posts)→
More from coding & agent
- Building a buyer agent that negotiates 8 seller agents: what broke and what fixed it — mv_soura · 2026-10-31
- PostTrainBench traces hit 104k monthly downloads — author warns it's likely reward hacking — maksym_andr · 2026-10-31
- Building games with AI? Set up a 'tech art' test scene to tune your look fast — chongdashu · 2026-10-31
- Sanity open-sources pi-mcp-events: streaming MCP Events client for the Pi agent harness — knutmelvaer · 2026-10-31
- Skydive opens to all: watch your AI coworkers work in a 3D office — Scobleizer · 2026-10-31
- LangChain SF meetup: decision models and a third path for agent evals — LangChain · 2026-10-31