ByteDance Paper: Are AI Agents Actually Controllable?
rohanpaul_ai · x · 2026-08-18
A new ByteDance paper introduces Harness-IF, questioning whether AI agents are truly controllable or if they just pass tests that align with their defaults. The framework evaluates instruction following using rules that go against the model's natural behavior across surfaces like system prompts and tool descriptions. Across 12 frontier models and 60 multi-turn coding tasks, every model scored worse when instructions pushed against their defaults, highlighting a gap in current agent controllability.
More from coding & agent
- After Three Years With AI Agents: Don't Let Comfort Turn Into Cognitive Surrender — rseroter · 2026-08-18
- LangChain Webinar: How to evaluate voice agents across execution, outcomes, and experience — LangChain · 2026-08-18
- Study Finds 3.8M Agent Skill Files Across 282K GitHub Repos — dair_ai · 2026-08-18
- 300-Page Monograph: Engineering Reliable Coding Agents as Systems — heyneighbor · 2026-08-18
- Tsinghua and ByteDance's CUDA Agent Writes Better CUDA Than Human Experts — anselm · 2026-08-18
- Manual backward pass: Ditching Autograd for extreme memory optimization — YouJiacheng · 2026-08-18