FACET: Preserving Source Intent and Executable State in Terminal Task Synthesis
Kou Shi, Zun Wang, Qisheng Su, Shiting Huang, Ziao Zhang, Zhen Fang, Qingnan Ren, Jin Liu, Yu Zeng, Yiming Zhao, Lin Chen, Zehui Chen, Feng Zhao
cs.AI, cs.PL
2026-08-19
FACET builds the container first, then writes the task against that state. 1.2K trajectories lift Qwen3.5-27B from 40.82 to 47.57 on Terminal-Bench 2.1, 1.49 shy of 397B.
Training a terminal agent needs tasks that actually run and actually check. Each task is a bundle: an instruction, an initialized container, a reference solution, and an executable verifier. If any two of those assume a different world, the task is unsolvable or scored wrong. Existing pipelines mint tasks from skill packs, terminal recordings, and Q&A. After a few generation stages, goals, dependencies, intermediate states, and procedural constraints collapse into a thin description. The instruction names a file the container never created. The solution assumes a different schema. The verifier tests a state the task cannot reach.
FACET keeps more of the source structure, and it writes all four artifacts against one realized execution state.
Public skill packages are collected from OpenClaw, ClawHub, and GitHub. Unsafe, private-network, and unreadable records are dropped, leaving 71,341 skills. An extraction agent proposes application contexts; similar hypotheses are retrieved by embedding and judged for complementarity and terminal executability.
Skill pairs are not turned into tasks immediately. Stage 2 rebuilds a compositional scenario through five modules: skill analysis, scenario exploration, association and filtering, evolution and recovery, and information expansion. The recovered workflow is written out along five axes (goal, context, capability, state, I/O and tools), then fused into a full natural-language scenario C. A solution reference is written first, then an instruction reference, and a consistency check requires the same initial state and the same observable end state.
Stage 3 materializes the environment before any final artifact. An environment agent plans a manifest, then creates files, services, and dependencies inside a restricted base image. Public resources are downloaded and localized so evaluation needs no network. Environment repair is capped at three rounds. Once the container is up, instruction, solution, and verifier are generated in that order, each with read access to the same realized state. Verifiers prefer behavioral checks over exact command matching. Acceptance follows the Harbor contract: the image builds, the verifier fails on the untouched initial state, the reference solution runs, and the verifier passes on the final state. A router repairs only the blamed artifact, at most five task-level rounds. Failures past that budget are discarded.
The funnel starts at 7,852 scenario–skill seeds and ends at 6,078 validated tasks, with 7,504 successful environments. DeepSeek-V4-Pro with Terminus-2 rolls out on about 6K tasks. Teacher success is 1,270/6,066 (20.94%). 1,200 complete successful trajectories are used for full-parameter SFT of Qwen3.5-4B, 9B, and 27B.
The tasks are denser and harder than the comparison sets. Mean executable checks per task are 22.77, against 16.60 for Terminal-Lego and 6.18 for Nemotron-Terminal. Trajectories average 11.86 turns. The same solver scores 27.00 P@1 on FACET tasks, lower than on the other datasets, which matches the denser check suite: finishing the main workflow is not enough if a secondary deliverable is missing.
| Model | Base | FACET-SFT | Gain |
| Qwen3.5-4B | 17.60 | 24.72 | +7.12 |
| Qwen3.5-9B | 27.34 | 35.58 | +8.24 |
| Qwen3.5-27B | 40.82 | 47.57 | +6.75 |
The fine-tuned 27B reaches 47.57, 1.49 below Qwen3.5-397B at 49.06 under the same protocol, with roughly 15× fewer parameters. The 4B model has the largest relative gain, about 40.5%.
On 100 shared pairs, Forward starts valid at 46.5% and finishes 83/100. Reverse (verifier before solution) is 24.2% then 63/100. Joint is 37.5% then 65/100. Contract mismatch is 56.5% of Reverse failures. On 500 shared pairs end-to-end, FACET’s validated yield is 70.0%, a TerminalWorld-style reproduction is 27.8%, and a no-reconstruction baseline is 15.6%.
Among unsuccessful rollouts, 54% fail only one or two checks. Check-level pass rate is 89.40%; task-level success is 20.94%. The hard part is satisfying every residual requirement, not failing the whole workflow.
For terminal-task synthesis, building the container first and writing against that state is more reliable than inventing instruction, solution, and verifier from text alone. 1.2K trajectories move 4B through 27B by about 7 points on Terminal-Bench 2.1, so the supervision is data-efficient. Code, models, and data are public.
This is still SFT. Binary task reward zeros out many near-miss trajectories, a gap the paper itself measures. Using these tasks as an RL environment would need a finer process signal.
The teacher succeeds on only one run in five. SFT keeps the successes and drops failed rollouts that already passed most checks. Forward is allowed five repair rounds; Reverse and Joint get three, so final yields are not a matched-budget repair comparison. The TerminalWorld pipeline is a reproduction adapted to the same skill-pair inputs, not the original paper’s numbers. Skills come from the OpenClaw/ClawHub ecosystem; there is no external calibration against real user terminal work.
Under the same setting, Qwen3.6-27B already scores 53.93, above the fine-tuned Qwen3.5-27B. Closing in on 397B is a fair within-family comparison. It is not a claim about the 27B ceiling.