OpenSIR: a self-play framework lets LLMs teach themselves math with zero training data

PMinervini · x · 2026-10-28

OpenSIR (Open-Ended Self-Improving Reasoner) is an open-ended self-play framework where an LLM improves its math reasoning without any external training data.

A single policy alternates between teacher and student roles, co-evolving:

Problem generation: the teacher proposes diverse, appropriately challenging problems that must be conceptually different from existing ones, stored in a problem pool;
Solution sampling: the student samples multiple step-by-step solutions per problem;
Scoring: rewards combine novelty (difficulty + diversity weighting, keeping problems challenging yet solvable while exploring new concepts) and correctness (majority-vote on sampled answers);
Update: the combined reward updates the model, looping into the next iteration.

The entire pipeline is unsupervised, with difficulty × diversity driving open-ended self-improvement.

Original post →

OpenSIR: a self-play framework lets LLMs teach themselves math with zero training data

More from Models