Harness as a Language: A Minimalist Agent Framework With Maximal Expressivity
Zhening Li, Joshua Liu, Mateja Vukelic, Nicole Shen, Supriya Lall, Amitayush Thakur, Alex Zhang, Omar Khattab, Jonathan Light, Armando Solar-Lezama
cs.AI
2026-09-23
JAZ reduces the agent stack to recursive invoke; it hits 69.9% far-recall on StuLife vs Letta's 61.8% at half the cost, and 74.2% TGC on AppWorld vs ACE's 69.9%.
Language-model agents sit in an agent loop: call a tool, read the result, decide the next step. ReAct lets the model pick the action. CodeAct upgrades the action to executable code in a REPL. Recursive language models make subagents the default. Long-horizon memory and cross-task self-improvement still get built as extra harnesses around that loop: MemGPT/Letta for memory, ACE for prompt evolution.
Li, Liu, Vukelic, Solar-Lezama, Khattab and colleagues at MIT CSAIL ask whether a harness that is little more than the loop can do those jobs. The framework is JAZ. The primitive is one function: invoke.
Start from a language-level definition. Take any Turing-complete language and add invoke(...). The call serialises every named argument, asks an LLM to write the function body, and runs it. Prompts, tools, and ordinary data are the same kind of argument. Because the language already contains invoke, generated code can call invoke again; subagents are the default. Tail-recursive invoke is already a closed agent loop on paper. In Python that blows the stack and breaks prefix cache, so the harness inserts the tail call and turns it into a REPL. Two properties follow:
The framework adds dynamic scoping via scope() (tools flow to all descendant invokes), hooks for budgets, validation, context-window warnings and trajectory replay, and a Config(llm, repl, protocol). The default protocol treats the raw model output as code and truncates REPL output only when it is too long.
Long-horizon work uses tail-recursive delegation: when the window is nearly full, return invoke(..., prevhistory=old+history), passing history by reference rather than by summary. Experiments fire a built-in ContextWindowWarning that appends a user message with a code template once a token threshold is crossed.
Continual self-improvement makes the top-level invoke a meta-agent: it edits the solver's prompt and skill functions, runs the next batch, reads returned history plus the eval, and edits inputs again. The prompt describes the workflow and contains no code templates. Solver recursion depth is capped at 2.
CodeAct and CodeAct+subagents are controlled ablations inside JAZ: a hook strips prompt and history variables. Off-the-shelf implementations scored worse because of prompt and REPL defects.
StuLife is 1284 ordered tasks simulating a college semester. A typical episode is 7000–8000 environment steps and 3000–5000 LLM calls. Of 939 scored tasks, 207 need information delivered more than 50 tasks earlier. All methods use GPT-5.4 nano (high); three independent runs.
| Method | Pass (all) | Pass (far recall) | Cost |
| CodeAct (per-task, no memory) | 52.5% | 24.8% | $4.4 |
| CodeAct+subagents (JAZ reimplementation) | 60.0% | 32.0% | $13.1 |
| Letta Agent | 70.9% | 61.8% | $42.1 |
| JAZ invoke | 72.6% | 69.9% | $18.3 |
On far recall JAZ is about 8 points above Letta at less than half the cost. One final-exam item about fictional protocols depends on a lecture 497 tasks earlier. JAZ was at recursion depth 70; prevhistory still held the full trace, and a substring search recovered the lecture in one turn. CodeAct+subagents copied the user prompt into children by hand, dropped the "these protocols are fictional, look them up" instruction, and answered from real-world knowledge. Letta's conversationsearch mixed keyword and vector hits and returned the quiz itself or similarly named but different protocols; it has no exact substring match.
AppWorld test-challenge is 417 tasks. Solver GPT-5.4 nano, meta-agent GPT-5.4, six runs for self-improving methods. JAZ reaches 74.2% TGC and 51.1% SGC at $20.9. ACE on the same CodeAct backbone is 69.9% / 47.1% at $30.6, of which $17.1 is the meta-agent. ACE's reflector and curator cost about 10× more per task than JAZ's meta-agent, so ACE was allowed to learn on the first 42 tasks (10%) only. JAZ learned across the whole test set and was still cheaper: it batches tasks, reads traces on demand, and can edit skills, not only the prompt.
Memory systems and self-improvement systems are demoted from products to consequences of loop expressivity. If history is an object, it can be passed by reference and searched by program. If the parent controls every input to a child invoke, it is already a meta-agent. Anyone stacking MemGPT, playbooks, and skill libraries gets a clean contrast: ask whether the missing piece is a specialised harness, or whether the model cannot programmatically touch its own context.
This is a framework paper, not a new model. The numbers are tied to the GPT-5.4 family and two benchmarks. The ablation is clean: inside the same JAZ, removing "prompt and history are variables" drops far recall from 69.9% to 32%. The extra expressivity shows up in the score.
There is no standalone limitations section. Several gaps sit in the setup. The "prompt-only" long-horizon run still injects a code template via ContextWindowWarning; the model does not invent tail-recursive delegation unaided. Keeping full history by reference to depth 70 will bloat; memory and latency are not reported. AppWorld self-improvement sees full test feedback after every task, which production does not provide. Both benchmarks are simulators, with n of 3 to 6. Official CodeAct numbers were worse, blamed on prompt and REPL bugs, so the controls were rewritten inside JAZ; framework quality is itself a confound. Only the GPT-5.4 family was tested.