ByteDance Paper: Are AI Agents Actually Controllable?
rohanpaul_ai · x · 2026-08-18
A new ByteDance paper introduces Harness-IF, questioning whether AI agents are truly controllable or if they just pass tests that align with their defaults. The framework evaluates instruction following using rules that go against the model's natural behavior across surfaces like system prompts and tool descriptions. Across 12 frontier models and 60 multi-turn coding tasks, every model scored worse when instructions pushed against their defaults, highlighting a gap in current agent controllability.
More from coding & agent
- Meta open-sources Muse Spark Open agent model with day-0 TPU support in SGLang — BanghuaZ · 2026-10-16
- Saying "load-bearing" in your prompt makes Opus 5.5 stop holding back — zacharynado · 2026-10-16
- You.com partners with Paperclip to give every agent live web access — PolarBearby · 2026-10-16
- I priced every Grok Bot call: $20 to $200 plan in 5 days, long jobs eat 52% — jonathanmalkin · 2026-10-16
- GitHub is criminally underrated as a free data store, devs argue — blelbach · 2026-10-16
- Claude Desktop doesn't support MCP sampling — here's a workaround for two-way MCP apps — wuj · 2026-10-16