A 4B model matches Claude Sonnet 4.6 after RL on 5,144 tasks back-chained from real MCP APIs

Firefly: Illuminating Large-Scale Verified Tool-Call Data Generation from Real APIs

Yuxuan Lu, Ziyi Wang, Yingzhou Lu, Yisi Sang, Jiri Gesi, Xianfeng Tang, Yimeng Zhang, Zhenwei Dai, Hui Liu, Hanqing Lu, Chen Luo, Qi He, Benoit Dumoulin, Jing Huang, Dakuo Wang

cs.SE, cs.CL

2026-05-18

Firefly explores real MCP APIs first, then back-chains tasks from observed outputs: 5,144 verified tasks. A 4B model RL-trained on them matches Claude Sonnet 4.6 on the held-out set.

What problem this solves

Training a tool-calling agent needs three things at once: tasks, executable trajectories, and a checkable final answer. RL makes the appetite worse; even data-efficient recipes consume thousands of tasks, while public benchmarks offer tens to hundreds (50-115 per domain in Tau2-Bench, 127 in MCPMark). Synthetic pipelines scale but give up either realism or verifiability. Simulated environments let an LLM invent tool responses that drift from real API behavior. Forward generation, where the task is written first and the trajectory filled in afterward, produces calls that may not execute and answers with no ground truth to check against. Live APIs add a third problem: weather, stock quotes, and search indices change over time, servers rate-limit and disappear, so RL rollouts are expensive and unreproducible.

Method

Firefly inverts the pipeline. A strong LLM first explores real MCP servers, records the states it actually reaches, and only then are tasks synthesized backward from those observed outputs. Every label is correct by construction, because the answer records what real tools actually returned.

Three stages:

Training environment: every exploration call is cached in a retrieval-augmented simulator with three tiers. Exact input match replays the cached output; near match retrieves the closest historical calls and lets an LLM generate a response; unseen inputs return an error. The error tier is deliberately conservative, since ground-truth trajectories live in the cache, so calls far outside it are likely wrong and should earn no reward. RL and evaluation both run offline against this simulator.

Results

The pipeline yields 9,749 exploration DAGs and 9,275 candidate tasks, of which 5,144 pass validation. Tasks average 3.0 tool calls and 4.6 answer fields, and 38.5% require combining tools across servers. Generation used Claude Sonnet 4.5 batch inference, about 23.5B tokens, roughly $47K. During training, 42.2% of agent calls hit the cache exactly, 57.8% went through fuzzy LLM generation, and 0% fell into the no-data tier.

On the 200-task held-out set (Qwen3-4B base, GRPO training):

Modelpass@1pass@8
Qwen3-4B (base)28.1%38.4%
Qwen3-4B + Firefly RL41.5%52.8%
Claude Haiku 4.530.0%48.7%
Claude Sonnet 4.642.2%50.3%
Claude Opus 4.744.5%54.4%

That is +13.4 points for the 4B model, matching Sonnet 4.6 at pass@1 and passing it at pass@8; at pass@16 the 4B ties Opus 4.7 at 57.0%.

All seven public-benchmark results improve:

BenchmarkBaseAfter RL
Tau2-Bench Retail0.4910.627
Tau2-Bench Airline0.3650.525
Tau2-Bench Telecom0.1890.204
MCP-Atlas19.4%26.0%
MCPMark File System40.0%60.0%
MCPMark Postgres Easy70.0%80.0%
MCPMark Postgres Std9.5%13.3%

Why it matters

For teams building tool-calling agents, this is a copyable data recipe: realism and verifiable labels stop being a trade-off. Back-chaining is not a new idea, and execution-based filtering in code generation trod similar ground. The contribution here is showing it scales to a real MCP ecosystem of nearly 1,000 tools once a tool graph prunes exploration to semantically coherent neighborhoods, and that a replay simulator removes live APIs from the RL loop; $47K one-time buys unlimited offline rollouts. The transfer results carry more weight than the headline: Firefly is single-turn, yet gains show up on multi-turn Tau2-Bench and on real MCP execution in MCPMark, which suggests the model learns transferable behaviors (tool selection, grounding arguments in observations, chaining outputs) rather than memorizing cached trajectories. The paper is accepted at NeurIPS 2026, with code and data in an anonymous repository.

Limitations

The authors flag two: only a 4B model was trained, and every task is single-turn, with no multi-turn clarification or requirement changes. The small pass@1-to-pass@16 gap (41.5% to 57.0%) says a chunk of hard tasks stays out of reach for 4B at any sampling budget.

A few discounts apply when reading the headline. Matching Claude Sonnet is measured on Firefly's own held-out set, generated by the same pipeline, evaluated in the same simulator, with an LLM judge as reward fallback; the 4B plays on home turf while Claude is zero-shot. The simulator's fuzzy tier generates responses with an LLM, and 57.8% of training-time calls went through it, so grounding every response in real API outputs is not literally true for most calls. Telecom gains are small (0.189 to 0.204), so transfer is uneven. Tasks are reverse-written from existing outputs, which invites questions fitted to the answer; the 7.8/10 realism score is an LLM self-assessment, and no human-audit numbers are reported.

Terms

Source

What people are saying

Related papers

All paper explainers