Agent Harness Lessons: Every Fix for GPT-3.5 Turbo Was Code, Not a Prompt
tejaskumarlol · reddit · 2026-09-14
After a talk at AI Engineer Europe, the author recounts building an agent harness where GPT-3.5 Turbo drives a real browser via Playwright to upvote the top Hacker News story — with the prompt never changing across versions:
- v0 bare loop: hits the login wall yet reports success, because the loop trusts the model's "done".
- v1 guardrails: code-enforced iteration/message limits and naive context trimming — bounds damage but fixes nothing.
- v2 deterministic verify: plain code reads the tool call trace to check the upvote actually happened, up to 3 retries — still fails, but stops lying.
- v3 harness-side login: the harness detects the login page from the URL and logs in with credentials from env vars the model never sees; success in 6 iterations.
Takeaways: a model claiming completion is a claim, not evidence — verify in code; put secrets and fragile deterministic steps in the harness, not the prompt; the harness is everything around the agent loop, sometimes a loop around the loop.
Related event: IBM Engineer Explains Agent Harness: Making GPT-3.5 Actually Deliver(3 posts)→
More from coding & agent
- User builds 20-bot Grok suite to track wins, prep 1:1s and close promises — NicoVerderosa · 2026-10-30
- LangChain CEO opens Interrupt London with keynote on why orgs should own their intelligence — LangChain · 2026-10-30
- Simlock: open-source control plane gives parallel coding agents their own iOS/Android simulators — Scobleizer · 2026-10-30
- YC experiment: Claude agents formed democracies while GPT agents formed hives — soumitrashukla9 · 2026-10-30
- Keyfleet launches open canvas where personal AI agents team up with a shared USDC treasury — seanwbren · 2026-10-30
- Page-based project management for Grok Bots: keep chats short, history on Pages — AiJohnAllen · 2026-10-30