Microsoft's FrogNano trains a 4B coding agent almost entirely with RL, no teacher model

ossm-me · reddit · 2026-09-10

A new report called FrogNano from Microsoft Research Montréal's Froggy Team, with collaborators from Mila and UC San Diego, refines Qwen3.5-4B into a small coding agent using reinforcement learning — with no human-labeled data and no larger teacher model providing answers.

Key points:

All learning comes from synthetic software engineering tasks generated and trained across roughly 1,500 environments over repeated rounds;
The core contribution is task difficulty curation: tasks that are too easy teach nothing, too hard yield no reward, and learning only happens in a narrow middle band;
The system keeps generating new tasks matched to the model's improving skill, and the authors argue difficulty matching matters more than raw task count.

The stated goal is a coding assistant that runs on limited hardware. It's an early report rather than a finished model, but clearly presented and carefully tested.

Original post →

Microsoft's FrogNano trains a 4B coding agent almost entirely with RL, no teacher model

More from coding & agent