CoworkEnv: Synthesizing Training Environments for Long-Horizon Cowork Agents
Zhongwei Xie, Yiming Du, Jiaxin Bai, Baixuan Xu, Chunyang Li, Xiyu Ren, Wei Fan, Tianshi Zheng, Weizhi Du, Jingqiao Yu, Xingsu Li, Zhaowei Wang, Yangqiu Song
cs.AI, cs.CL
2026-10-18
CoworkEnv synthesizes executable coworker envs with requests, records, tools, and graders. SFT raises Qwen3-VL-8B from 46.6 to 64.1 on Claw-Eval; multi-turn gains 31.9 points.
An AI coworker has to read records, call tools, and change workspace state across many turns while obeying workflow rules. A support escalation depends on customer tier and ticket age, must update the right ticket, and must skip prohibited actions. A fluent final reply does not show that the update landed, or that a ban was respected.
The evidence is spread across services. Records for the same customer have to agree, tools have to apply the state change, and the grader has to read actions and audit logs. If those pieces do not fit, the sample cannot be executed. CoworkEnv turns a named capability into tasks that support both imitation and learning from interaction.
Each task is a tuple: user request, initial state, tool interfaces, and grading checks. A coverage plan fixes the service mix, difficulty, distractor records, permitted actions, and a reference date. A model drafts the four parts from service catalogs. Programs fill registered services and a fixed grader template. New scenarios change records and checks. The runtime stays.
One axis splits evidence across services. An escalation combines ticket age with customer tier, selects enterprise tickets open for more than 48 hours, and fetches the account manager address for an email. A fresh high-priority ticket is a distractor: it looks urgent and fails the age test. The safety axis separates looking something up from being allowed to act on it. A payment-change case places a suspicious email next to legitimate mail and transaction history. Gathering evidence scores points. Sending a message or submitting a report trips a safety gate. The generator writes the request and records. A discriminator, in a separate context, writes completion checks and forbidden actions from the records and a scenario note. That split keeps answer identifiers out of the prompt.
A failed check returns a diagnostic, and the whole spec is rewritten. The spec must parse. Services must be registered and carry initial records. Completion weights must sum to 1. Required and prohibited tools must not overlap. The request must not leak identifiers the agent is supposed to discover. A behavior gate tests the grader on synthetic records: a satisfying record must get full completion and safety scores, an empty record must score 0 on completion, and one extra prohibited call must set safety to 0.
A teacher runs tasks through a Claude Code adapter in Harbor. Scores come from tool arguments and service audit records. Consistently solved tasks contribute one demonstration per normalized prefix, so easy items are not repeated. A middle band keeps several qualified runs. The lowest band is dropped. A later filter removes trajectories that are too short, too tool-heavy, or over the character budget, and long tool observations are cut to a prefix. Supervised fine-tuning fits only assistant replies and tool calls, with equal weight on supervised tokens. Rewards choose which demonstrations to keep and do not reweight the loss. The student then acts in the same environment. Missing rewards abort the sample. Failed rollouts are retried before advantage estimation. SFT and RL share tasks, services, and graders. PocketWorker is a LoRA on Qwen3-VL-8B-Instruct. The stack used slime and Polar during development. The RL comparison covers GRPO, Dr.GRPO, GDPO, and GiGPO, which differ in reward processing and credit assignment. Training tasks were built apart from the test sets. A library snapshot lists 602 Claw-Eval task identifiers and 263 ClawMark identifiers, with 236 in both. The paper says those counts are not test-set or training-set sizes.
The main Claw-Eval numbers use 161 General tasks and 38 Multi-turn tasks, 199 in total, three trials each, on a 0-100 scale. Pass@3 is the share of tasks with at least one trial at or above 0.75. Pass3 requires all three trials to clear that bar.
| Model | Overall | Multi-turn | General | Overall Pass@3 / Pass3 |
| Qwen3-VL-8B-Instruct | 46.6 | 45.9 | 46.7 | 28.6 / 21.6 |
| PocketWorker-8B (SFT) | 64.1 (+17.5) | 77.8 (+31.9) | 60.9 (+14.2) | 50.8 / 37.2 |
| Qwen3-VL-4B-Instruct | 33.9 | 56.1 | 28.7 | 19.6 / 11.1 |
| PocketWorker-4B (dialogue SFT) | 42.2 (+8.3) | 66.0 (+9.9) | 36.6 (+7.9) | 22.6 / 15.6 |
On 8B multi-turn, Pass@3 moves from 34.2 to 78.9 and Pass3 from 13.2 to 50.0. Half of those tasks still fail to pass every trial. General rises by 14.2.
Published Overall scores from Claw-Eval v3 are 81.4 for Claude Sonnet 4.6, 80.4 for Opus 4.6, 78.4 for GPT-5.4, 74.2 for Qwen3.5-397B-A17B, 68.3 for Kimi K2.5, 67.5 for DeepSeek V3.2, and 44.4 for Nemotron 3 Super. PocketWorker-8B at 64.1 beats its base and Nemotron 3 Super, and trails most of the larger systems. Its multi-turn 77.8 sits next to Kimi K2.5 at 75.4, below Sonnet 4.6 at 81.9, and above DeepSeek V3.2 at 64.0. The paper says a claim of beating a reference model needs matched evaluation settings. Reported total and active sizes include 120B/12B, 1T/32B, and 397B/17B for those three open systems.
The 4B base scores 56.1 on multi-turn, above the 8B base at 45.9. Its checkpoint is dialogue-only SFT for two epochs, and General Pass@3 and Pass3 each fall by 0.6. The data do not match the 8B tool-trajectory run, so this is not evidence that the smaller model is better at multi-turn work.
ClawMark has 100 tasks in 13 professional domains. SFT alone moves the 8B model from 11.8 to 19.4. On the website Run 1, GPT-5.4 scores 56.7, Sonnet 4.6 scores 53.1, and Kimi K2.5 scores 56.0. The recipe transfers. The absolute level stays low.
SFT data were randomly subsampled inside each stage. Overall goes from 46.6 with no SFT to 50.8 at 20% of the data, 55.6 at 50%, and 64.1 at full data. Multi-turn scores are 45.9, 53.6, 64.9, and 77.8. The second half of the data adds another 12.9 on multi-turn.
RL starts from a shared supervised warm-up and runs for about three epochs. The comparison below is against the base. Full-data SFT at 64.1 is a different starting point.
| Setup | General | Multi-turn | Overall |
| Qwen3-VL-8B base | 46.7 | 45.9 | 46.6 |
| Warm-up + GRPO | 52.8 (+6.1) | 72.5 (+26.7) | 56.6 (+10.0) |
| Warm-up + GDPO | 53.0 (+6.3) | 74.9 (+29.0) | 57.2 (+10.6) |
Multi-turn is where RL gains concentrate, and GDPO leads GRPO there. On 101 multimodal tasks the base scores 20.7 and GRPO reaches 22.1. Dr.GRPO and GiGPO are in the same table, with gains that depend on task type. The prose does not list their cells the way it lists GRPO and GDPO. Full-data SFT, at 64.1 overall and 77.8 multi-turn, stays ahead. Training rewards rise. The paper treats that rise as preliminary evidence that the environments can support policy optimization.
The reusable piece is the executable package: request, records, tools, and grading code in one place. Imitation and online rewards do not need two environments. With limited GPUs, that is more useful than another advantage estimator.
The large gains are Claude teacher trajectories distilled into an 8B model. Multi-turn adds 31.9, General adds 14.2, and ClawMark adds 7.6. The last two are much smaller, so the Claw-Eval multi-turn number does not stand in for coworker skill as a whole. More supervision still helps: full data beats half. About three epochs of RL after warm-up do not pass full-data SFT. For a stronger 8B checkpoint, use the SFT result.
Scores measure only what the graders encode, so errors outside those checks can pass. Filtering on teacher success keeps redundant successes and can drop useful recovery. The paper says failed trajectories deserve study, and other work on iterative skills finds that feedback which includes failures can still change behavior. Regenerating the original teacher runs may require a proprietary model service.
The figures 602 and 263 count library identifiers. The main text does not give the number of training trajectories. RL starts from warm-up, a weaker point than full-data SFT, so beating the base does not show that interaction beats imitation. The SFT table omits multimodal tasks, and that slice barely moves under RL. On the 4B model, General pass rates dip. Reference scores are reproduced from elsewhere. The lessons also note that a different harness changes training outcomes, and that the training shell and the inference shell can disagree. These are simulated offices. Deployed without extra safeguards, the agents can take wrong or unauthorized actions.