Amazon's open eight-stage recipe lifts GLM-4.5-Air IFBench from 33.6 to 76.9

Rufus-Air: An Open LLM Post-Training Recipe

Chia-Yuan Chang, Renyuan Cheng, Rui Feng, Xiaotian Han, Yuan He, Hongye Jin, Linwei Li, Shiyang Li, Fenglin Liu, Xin Liu, Priyanka Nigam, Haoyang Wen, Zhenghao Xu, Zhuocheng Xu, Bing Yin, Qingyu Yin, Chao Zhang, Rongzhi Zhang, Zhihan Zhang, Zixuan Zhang, Zixuan Zhang, Tuo Zhao

cs.CL, cs.AI, cs.LG

2026-09-24

Amazon serializes SFT through RLHF on public GLM-4.5-Air-Base data; Rufus-Air hits 76.9 IFBench versus the vendor's 33.6, and 76.4 LiveCodeBench v6 versus 59.6.

What problem this solves

Open-weight bases made post-training reachable. The recipes did not. Most reports still read like system cards: a list of stage names, one leaderboard table, and almost none of the data mix, reward design, stage order, or infra that a team would need to rerun. Tülu 3 is a rare exception.

Rufus-Air is Amazon's attempt to write the missing document, on the public GLM-4.5-Air-Base checkpoint (106B total, 12B active). The stack is open-source. The data is public. There is no new human annotation and no in-house distillation teacher. The main comparison is the vendor's own post-trained GLM-4.5-Air, plus INTELLECT-3, which starts from the same base.

Method

Eight stages run in series, each initialized from the previous checkpoint: SFT, Reasoning RL, Coding RL, Instruction-Following RL, General Agent, Coding Agent, Search Agent, RLHF.

Two axes set the order. Capability goes from basic to advanced. Rewards go from hard, verifiable signals toward softer judges and a preference model. Instruction-Following RL uses a rubric judge yet sits before the agent stages, because the target is close to what the policy already does and the judge has little room to be gamed. The preference reward of RLHF is the one that can actually be hacked, so that stage runs last.

A few choices show up in almost every stage:

• SFT is treated as capability building, not a warmup. The mix is 9.01M samples and 27.04B supervised tokens across chat, STEM, math, code, and two agent families. Three epochs are trained, but held-out scores flatten inside the first epoch, so RL starts from step 3799 rather than the last checkpoint.
• Difficulty filtering keeps prompts in a productive band. Prompts the current policy already solves above 80% are dropped; most stages also drop prompts it never solves. Reasoning RL keeps filtering online, admitting only groups whose mean reward sits in (0, 0.8].
• Verifiable stages use GSPO or GRPO. On the MoE, Rollout Routing Replay caches expert routing from FP8 rollouts and replays it in BF16 training. Agent stages pass token IDs through the loop so the chat template is never reapplied to earlier turns.
• The coding sandbox is a Firecracker microVM, not a shared-kernel container. Search Agent trains against an internal search backend and is evaluated on Serper plus Jina, which the policy never saw in training.

SFT ran about 13 days on 64 nodes of 8×H200. Reasoning RL used 8 nodes, Coding RL 32, agent stages 16.

Results

SFT alone already beats the vendor's post-trained release on instruction following and contest math:

GPQA is the hole SFT leaves. Later RL stages close it. The finished checkpoint versus the official release:

The largest single-stage moves are Multi-challenge under IF RL (+24.7), Creative Writing under RLHF (+14.4), and Tau2-Retail under General Agent (+9.8). Coding Agent stopped at the compute budget: Terminal-Bench 2.1 +1.4, SWE-bench +2.2, and the SWE-bench gain does not survive to the shipped checkpoint (67.8 back to 65.6).

Why it matters

For a team with tens of H200 nodes that wants to post-train an open base, this is a stage-by-stage construction document, not a scoreboard. Four takeaways travel: SFT quality sets the RL floor; difficulty filtering moves more than swapping the RL objective; stage order should track how gameable the reward is, not the task name; infra (token fidelity, sandboxes, routing replay) belongs in the recipe.

Versus INTELLECT-3, the comparison isolates the recipe on the same base. Versus Nemotron-3-Super, the lead sits in instruction following and agents; math and science are close. The recipe scores where it spends training signal.

Limitations

The authors say this directly: the stage order came from earlier experiments and a compute budget, not from an optimality proof, and they did not try training domain experts then merging them. Coding Agent was not trained to convergence. No single benchmark suite was tracked through every stage, so movement on SWE-bench, Terminal-Bench, and Seal-0 between a mid-pipeline checkpoint and the final model cannot be attributed to one stage.

A few other cracks. SFT decontamination is 8-gram overlap plus retrieval; the paper admits rewritten leaks can remain. Search Agent only requires two successful tool calls for a positive reward, so a memorized answer with incidental search is still a residual shortcut. Creative Writing still trails the official release by 7.3 points; RLHF used a generic Skywork reward model with no style-specific training. Tau2 numbers come from a reimplemented harness and a Claude user simulator, not the reference stack.

Terms

• RLVR: Reinforcement Learning with Verifiable Rewards, where a program checks answers instead of a learned reward model.
• GSPO: Group Sequence Policy Optimization, which applies the importance ratio and clipping at sequence level, more stable on long MoE rollouts.
• Rollout Routing Replay: Cache MoE expert routing from rollout and replay it in the training forward pass, removing a log-prob gap from routing mismatch.
• learnability filter: Keep only prompts the current policy sometimes solves, dropping all-pass groups (no gradient) and all-fail groups (no signal).
• token-in/token-out: Pass token IDs through multi-turn agent RL so history is never re-tokenized.

Source

• arXiv 2609.29421v1
• PDF
• Hugging Face

What people are saying

• Amazon team's open Rufus-Air recipe post-trains GLM-4.5-Air on under 400 H200s JingfengY

Related papers

• JCA tops all nine hard slices and leads CoT by up to 20.6 points
• Lossy speculative decoding accepts 34% more tokens per round, then writes 54% more text
• CoDA beat expert Macro F1 after 115 reviews and two hours on homelessness notes
• ScaleMAS caps search at 8 messages and reaches 50.9% with 128 agents
• A perfectly plausible chatbot reply scores 0.0 BLEU: surveying how we evaluate text generation
• A 3B monitor cuts Llama-3.1 multi-turn harm from 86.36% to 9.85%, over-refusal 11.64%

All paper explainers