IQuest-Q1: Advancing Foundation Capabilities for Agentic CLI Systems
A. Jian Yang, An Huang, Bo Dan, Chen Mu, Chengran Yang, Chuan Shi, Feng Chang, Hongxin Li, Hongyuan Tao, Jinxing Zhang, Jujie He, Kun Wang, Lin Jing, Ming Yang, Qicheng Hu, Renyuan Li, Shark Liu, Shawn Guo, Shi Qing, Shi Zhong, Tianyu Zheng, Wei Liu, Wei Zhang, Wenrui Cai, Wenxiao Chen, Xuxing Chen, Yan Xing, Yi Xing, Yifan Luo, Yiran Xing, Yizhi Li, Yongsheng Kang, Yucheng Ding, Yue Song, Yun Qing, Yunqi Liu, Zelong Huang, Zhengmao Ye, Zhentao Fan, Zhiteng Li, Ziling Huang, Bihong Tang, Bingli Wang, Ce Jun, Changhao Chen, Chen Hu, Chen Xiao, Chi Liu, Chunjiang Ge, Danyang Li, Deyang Kong, Dian Qi, Ding Tan, Evan Yang, Fanglin Xu, Fei Bai, Gang Liu, Guochang Li, Hao Tang, Haoran Feng, Haowen Wang, Huahao Hu, Huanjin Yao, Huatong Song, Jia Deng, Jiajun Wu, Jialiang Cheng, Jiayi Liang, Jiayi Wu, Jie Deng, Jiejun Tan, Jing Yang, Jingda Xu, Junhang Cheng, Junxiao Liu, Kai Lv, Kaixiang Pan, Kecheng Lu, Kun Zeng, Kun Zheng, Kunbo Ding, Liang Qiu, Lingyi Sun, Luan Zhang, Mengxiao Lv, Ning Dong, Pei Wang, Peng Zhang, Qingsong Cai, Qixiang Yin, Qiyao Wang, Renbiao Liu, Renfei Dang, Ruihang Li, Ruirui Huang, Shanhui Yin, Shen Jian, Shengjie Fang, Shikai Li, Shinan Zhang, Shiqi Chen, Shuai Liu, Shuangqi Li, Shuo Tang, Shuqing Deng, Sijin Hao, Siwei Wu, Siyue Wang, Tao Hai, Tao Ren, TingYang Chen, Wei Shao, Weicheng Gu, Weihan Peng, Wenlong Miao, Xavier Yu, Xiangyuan Ji, Xiaocheng Li, Xiaodong Huang, Xiaohan Ge, Xiaoxiang Shi, Xiaoyang Zhang, Xiaoyu Yang, Xidong Wang, Xin Li, Xinyu Tang, Xiyuan Yang, Xudong Ma, Ya Tong, Yachao Wang, Yang Tian, Yaxin Du, Yeli Ruan, Yi Wang, Yi Yu, Yibin Feng, Yibo Tan, Yihang Wang, Yiming Ju, Yinda Chen, Yinda Chen, Yingjie Du, Yiwei Liu, Yizhi Zhou, Yuan Wei, Yuli Chen, Yuqi Zhu, Yuwen Li, Yuxi Dong, Yuxuan Zhang, Yuyang Song, Zhange Zhang, Zhaohan Zhang, Zhaoyi Li, Zhen Zhang, Zhengbo Zhang, Zhihang Hu, Zhiwei Yu, Zhixin Xie, Zhiyu Chen, Zhizhao Luo, Zhongyuan Wu, Zhou Fang, Zhu Lin, Zhuocheng Gong, Zixi Zhao, Ziyang Zeng, Ziyi Wang, Ziyu Zhao, Ran Tao, Mingjie Tang, Chuan Hao, Peihao Wu, Jianzhou Wang, Nan Jiang, Yuan Zhi, Bryan Dai
cs.CL
2026-10-22
An open 320B/15B MoE built for CLI agents: 1.93x pre-training efficiency vs a Qwen3-style baseline on identical data, and 84.5 on CyberGym, ahead of closed models.
The command line has become the most general interface between language models and the outside world: running code, touching files, calling tools all bottoms out in a terminal, so an agent's ceiling increasingly depends on how long a model can keep working inside one. Open-weight releases from DeepSeek, GLM, and MiniMax have pushed toward agents, but their pipelines mostly center on general chat or code completion. IQuest-Q1 takes the opposite bet: a 320B-total / 15B-activated open MoE whose data mixtures and RL environments are all built around CLI agents.
Three engineering problems shape the pipeline. Pre-training stability, since the margin between safe and optimal learning rates decides how large a model can be. Scarcity of agent trajectories, which concentrate in a few tasks and environments. And reward hacking in RL environments, where the model games the verifier and poisons the training signal.
The architecture assembles known parts: 88 layers, sliding-window and full attention interleaved 3:1, query-aware sink tokens absorbing excess attention mass, QKNorm with clipping, 256 experts with 8 activated per token, a dense FFN in the first layer to avoid early-layer expert imbalance, and two multi-token prediction layers trained from the start, one later rebuilt into a recursive drafter for speculative decoding.
Stable SplitNorm targets a known weakness of Pre-Norm: residual-stream variance accumulates with depth, so later layers' updates contribute relatively less and effective depth shrinks. IQuest-Q1 splits the strategy, normalizing the stream entering attention while leaving the stream entering the FFN unnormalized. The FFN update then travels an unscaled identity path, approaching Post-Norm depth utilization while keeping Pre-Norm stability; an extra RMSNorm on the attention output guards against activation blowup, and the first layer adds a SandwichNorm.
Optimization combines Muon with μP parameterization. The learning rate is searched once on a small proxy (896 hidden width, 18 layers) and transferred zero-shot to the target, validated across a 4x range in both width and depth. Pre-training plus mid-training totals 29T tokens; the second stage raises the code share from 40% to 70% in four steps, context grows from 32K through 128K to 512K, and mid-training builds over one million executable containerized instances from real repositories, rolling out agents inside them to harvest valid trajectories.
Post-training carries most of the weight:
Reward hacking defense runs three layers: environment preparation (cleaned Git history, removed solution files), execution-time interception (network restrictions, tool-call pre-checks), and trajectory auditing with rules plus LLM judgment. On stability, the paper borrows the Critical Batch Size idea to define a Critical Model Size, extrapolated to roughly 572B activated parameters (about 12T total), far above the current 15B.
| Benchmark | IQuest-Q1 | GLM-5.3 Flash | Claude Opus 5 | GPT-5.6 Sol |
| CyberGym | 84.5 | 81.6 | 76.2 | 77.6 |
| DeepSWE v1.1 | 64.6 | 63.4 | 73.7 | 73.0 |
| SWE-Bench Pro | 57.5 | 59.4 | 79.2 | 64.6 |
| Terminal-Bench 2.1 | 83.2 | 84.3 | 89.1 | 88.8 |
| Terminal-Bench 4.0 | 22.7 | 32.8 | 53.9 | 37.3 |
| HLE (no tools) | 39.2 | 39.9 | 54.9 | 49.5 |
Asterisks mark scores the IQuest team ran themselves. The open-source field stays tight: MiniMax M3's self-tested 75.0 on DeepSWE v1.1 beats IQuest-Q1's 64.6.
The efficiency claim comes from a controlled comparison on identical data against a Qwen3-30B-A3B-style baseline: the IQuest-Q1 architecture reaches the same training loss with about 52% of the compute, a 1.93x scaling efficiency.
Two real-world signals: natural-language prompts produce browser-runnable interactive apps, with examples spanning games, a 3D garden editor, and scientific visualization; and a three-day deployment pairing human security experts with the model produced 30 vulnerability reports, all independently validated and responsibly disclosed.
The open-weight field gains another 320B-scale model with an explicit CLI focus, released with weights, deployment configs, and agent-integration instructions. For most practitioners the reusable part is the pipeline, not the weights: multi-harness, multi-version RL trains a single policy across Claude Code and Codex versions, addressing cross-harness generalization; the three-layer reward-hacking defense and the rule that only policy-caused failures enter the gradient are among the few agent-RL engineering details written down clearly; the zero-shot μP learning-rate transfer is a directly copyable setup for compute-constrained teams.
The most interesting section is RSI: IQuest-Q1 participates in developing its own successors. In one documented case, the agent hit 35 tool-call errors in a long-horizon task, traced them across 6,054 tool calls, found the training data biased toward a single tool-call schema with preprocessing mismatches, and proposed cleaning plus mixed-schema augmentation. Human experts approved, and the fix ran as a 2.35B-token, 938.7-GPU-hour training job. Humans on the loop, model in the loop: a deliberately restrained take on recursive self-improvement.
The authors list: text-only input; explanations and code can be wrong, so changes need review or task-appropriate tests; tool calls use an IQuest-specific format requiring compatible serving parsers and harnesses; and in real CLI tasks the model may overlook constraints, repeat failed attempts, or leave issues unresolved.
Beyond that: many baseline scores in the comparison table are self-tested and mixed with public scores, which weakens comparability; IQuest-CLIBench is proprietary, so its 53.7 has no external anchor. The 1.93x efficiency baseline is a much smaller 3B-activated model, so the claim reads as this design beating that small baseline, with no direct evidence the gain carries to 15B-activated scale. The gaps are visible: 17.9/22.7 on Terminal-Bench 3.0/4.0 against GPT-5.6 Sol's 34.6/37.3, and 39.2 on HLE without tools against Claude Opus 5's 54.9, leaving long-horizon terminal work and expert-level reasoning well behind the frontier. The RSI fix was evaluated on held-out schemas, but no numbers are given.